Building a Production-Ready Node.js Microservice (Express + Kafka + MongoDB)

✅

Building a Production-Ready Node.js Microservice (Express + Kafka + MongoDB)

A Practical Guide for New Developers

📌

Overview

This document introduces new developers to our standard Node.js microservice architecture.

It explains the folder structure, the responsibilities of each layer, and how we use shared utilities to enforce consistency across all services.

This pattern has the following goals:

• ✅ Clean separation of concerns

• ✅ Predictable flow of data (HTTP → service → event → audit → response)

• ✅ Reusable shared utilities (auth, logging, validation, Kafka, Mongo)

• ✅ Easy onboarding for new developers

• ✅ Scale: dozens of microservices following the same template

• ✅ Modern best practices for Node.js and Express

This is the foundation for every service we build going forward.

🧱

1. High-Level Architecture

Every microservice follows the same building blocks:

Request → Router → Controller → Service

              ↓             ↓

      (Validation)       (Kafka Event)

                                ↓

                      (Mongo Audit Log)

This gives us:

• Thin controllers

• Services that contain all business logic

• Middlewares that enforce cross-cutting concerns

• Utility modules for Kafka and Mongo that remain consistent across all services

📂

2. Folder Structure

src/

  app.ts

  server.ts

  config/

    env.ts

    db.ts

    express.ts

  api/

    <feature>/

      controller.ts

      service.ts

      routes.ts

  shared/

    utils/

      logger.ts

      kafka.ts

      mongodb.ts

    middleware/

      auth/

        index.ts

    validation/

      index.ts

Each part has one clear responsibility.

🚦

3. Routing Layer

Located at:

src/api/<feature>/routes.ts

This file wires together:

• Authentication middleware

• Validation middleware

• Controller method

Example:

router.post(

  '/',

  authHandler,

  validator('createPayload'),

  controller.create

);

✅ Routes remain declarative and simple

✅ Business logic never lives here

🎛️

4. Controller Layer

Located at:

src/api/<feature>/controller.ts

Controllers:

• Receive raw HTTP input

• Check validation results

• Call the service layer

• Map service result → HTTP response

Example:

const result = await service.create(req.body, req.headers);

return res.status(201).json(result);

✅ Controllers are intentionally boring—they should contain zero business logic

🧠

5. Service Layer

Located at:

src/api/<feature>/service.ts

The service layer is the brain of the microservice.

Typical responsibilities:

• Merge and prepare data

• Produce an event to Kafka

• Write an audit log to Mongo

• Handle correlation IDs

• Perform transformations

Services should NOT:

• Know anything about HTTP

• Perform authentication

• Parse the request

• Talk directly to Express

Example:

const eventId = await createEventAndSend(

  kafkaProducer,

  env.KAFKA_TOPIC,

  payload,

  correlationId,

  eventTime

);

await logEvent(getDB(), eventId, payload, eventTime);

return { eventId };

✅ Keeps all orchestration in one place

✅ Reusable patterns across all services

📡

6. Shared Utilities

Shared utilities live in:

src/shared/utils/

We provide reusable modules for:

✅ Logging (

logger.ts

)

Unified logging interface for all services.

✅ Kafka (

kafka.ts

)

Centralized logic to:

• Connect to Kafka once

• Manage a reusable producer

• Publish a consistent event shape

✅ Mongo (

mongodb.ts

)

One function to write an event log:

logEvent(db, eventId, payload, eventTime);

✅ Validation (

validation/index.ts

)

Register schemas once and reuse across routes.

✅ Auth Middleware (

middleware/auth

)

Verifies tokens, extracts identity, etc.

✅ All services share the same utilities → reduces bugs, increases velocity.

🛠️

7. Configuration & Environment

Configuration lives in:

src/config/env.ts

All services follow 12-factor principles:

• Use environment variables

• No hardcoding URLs or secrets

• Deploy anywhere (dev, test, stage, prod)

Example:

export const env = {

  PORT: process.env.PORT || 3000,

  KAFKA_URL: process.env.KAFKA_URL,

  MONGO_URL: process.env.MONGO_URL

} as const;

✅ Easy to deploy across environments

🍃

8. MongoDB (Database Layer)

The DB connection is initialized once at startup in:

src/config/db.ts

• Verifies connectivity

• Exposes getDB()

• Ensures consumers cannot use the DB before initialization

This improves:

✅ Stability

✅ Health checks

✅ Startup sequencing

🔊

9. Kafka (Event Layer)

Kafka usage follows these principles:

• Connect the producer once

• Reuse it for all events

• Always include:

  
  

  • correlationId

  • eventTime

  • eventId (UUID)

  • payload

• 
  

This ensures:

✅ Consistent event shape

✅ Traceability across systems

✅ Easy debugging

❤️

10. Health and Readiness

Implemented in:

src/app.ts

src/server.ts

We expose:

• /health → is the process alive?

• /ready → is DB connected? Kafka connected?

This allows Kubernetes to:

• Restart services automatically

• Avoid sending traffic too early

• Ensure zero-downtime deployments

✅ Production-grade operational readiness

🧰

11. The “Starter Kit” Philosophy

This architecture gives us:

✅ Repeatable microservices

✅ Consistency for new developers

✅ A platform mindset

✅ Faster onboarding

✅ Reduced bugs

✅ Less debate about folder structure and patterns

A developer can walk into this structure and instantly know:

• Where routes live

• Where controllers live

• How the service works

• How events are written

• How Mongo logging works

• How to use shared utilities

This is professional Node.js engineering.

🚀

12. Creating a New Microservice

With the starter kit, the process is:

1. Copy the template

2. Rename the service folder

3. Create new api/<feature> folder

4. Add:

   
   

   • routes.ts

   • controller.ts

   • service.ts

5. 
   

6. Update validation schema

7. Update Kafka topic & event names

8. Deploy

Coming soon:

npx create-eip-service myService

Which will auto-generate the entire structure.

✅

Conclusion

This architecture is intentionally clean, scalable, and easy to learn.

It lets us deliver multiple microservices quickly, with:

• Consistency

• Reliability

• Readability

• Strong operational guarantees

• Separation between HTTP, business logic, and infrastructure concerns

Every developer on the team should understand these patterns,

because mastering this structure means you can walk into any Node.js shop and build production services on day one.

Just say the word, dude.