API Design Lessons Learned Link to heading

After building several APIs this year; from simple REST services to more complex GraphQL implementations; I’ve accumulated a collection of hard-won insights about what works, what doesn’t, and what I wish I’d known from the start. These lessons come from real projects with real users, not theoretical examples.

Consistency Is King Link to heading

The most important lesson has been the value of consistency in API design. Users develop mental models of how your API works, and breaking those patterns creates friction and confusion.

URL Patterns Link to heading

Early in one project, I was inconsistent with URL patterns:

GET /users/123          # Good
GET /user/profile/123   # Inconsistent
GET /posts/456          # Good
GET /post/456/details   # Inconsistent

Standardising on a clear pattern made the API much more intuitive:

GET /users/123
GET /users/123/profile
GET /posts/456
GET /posts/456/details

Response Structures Link to heading

Initially, my API responses were inconsistent in their structure:

// User endpoint
{
  "id": 123,
  "name": "John Doe",
  "email": "john@example.com"
}

// Posts endpoint (inconsistent metadata location)
{
  "posts": [...],
  "total": 45,
  "page": 1
}

Adopting a consistent response envelope improved the developer experience:

{
  "data": {
    "id": 123,
    "name": "John Doe",
    "email": "john@example.com"
  },
  "meta": {
    "timestamp": "2021-10-23T11:30:00Z"
  }
}

Error Handling Done Right Link to heading

Good error handling is what separates professional APIs from amateur ones. I learned this the hard way when debugging production issues became a nightmare due to poor error responses.

Meaningful Error Codes Link to heading

HTTP status codes should be accurate and consistent:

// Bad - generic 500 for all errors
if (error) {
  return res.status(500).json({ error: "Something went wrong" })
}

// Good - specific status codes with context
if (error.code === "VALIDATION_ERROR") {
  return res.status(400).json({
    error: {
      type: "validation_error",
      message: "Invalid input data",
      details: error.details,
    },
  })
}

if (error.code === "NOT_FOUND") {
  return res.status(404).json({
    error: {
      type: "resource_not_found",
      message: "User not found",
      resource: "user",
      id: req.params.id,
    },
  })
}

Error Response Structure Link to heading

Consistent error structure makes client-side error handling predictable:

{
  "error": {
    "type": "validation_error",
    "message": "The request contains invalid data",
    "code": "INVALID_EMAIL",
    "details": [
      {
        "field": "email",
        "message": "Must be a valid email address",
        "value": "invalid-email"
      }
    ],
    "request_id": "req_123456789"
  }
}

Versioning Strategy Matters Link to heading

API versioning seemed theoretical until I needed to make breaking changes to a production API with active users. Having a clear versioning strategy from the start would have saved significant pain.

Header-Based Versioning Link to heading

I initially used URL-based versioning (/v1/users) but found header-based versioning more flexible:

app.use((req, res, next) => {
  const version = req.headers["api-version"] || "1.0"
  req.apiVersion = version
  next()
})

app.get("/users", (req, res) => {
  if (req.apiVersion === "2.0") {
    // Return v2 format
    return res.json({ users: users })
  }
  // Return v1 format (backward compatibility)
  return res.json(users)
})

Deprecation Communication Link to heading

Clear deprecation warnings help users migrate gracefully:

if (req.apiVersion === "1.0") {
  res.set("Warning", '299 - "API version 1.0 is deprecated. Please upgrade to version 2.0 by 2022-01-01"')
  res.set("Sunset", "Wed, 01 Jan 2022 00:00:00 GMT")
}

Pagination and Performance Link to heading

Pagination seems straightforward until you’re dealing with large datasets and performance becomes critical.

Offset-Based Pagination Problems Link to heading

My initial pagination implementation used offset-based paging:

GET /posts?page=1&limit=20
GET /posts?page=2&limit=20

This approach had serious performance issues with large datasets due to database query costs and data consistency problems when new items were added.

Cursor-Based Pagination Link to heading

Cursor-based pagination solved these issues:

// Better approach - cursor-based pagination
app.get("/posts", async (req, res) => {
  const { cursor, limit = 20 } = req.query

  const posts = await Post.find({
    ...(cursor && { _id: { $lt: cursor } }),
  })
    .sort({ _id: -1 })
    .limit(parseInt(limit) + 1)

  const hasNextPage = posts.length > limit
  if (hasNextPage) posts.pop()

  const nextCursor = hasNextPage ? posts[posts.length - 1]._id : null

  res.json({
    data: posts,
    pagination: {
      next_cursor: nextCursor,
      has_next_page: hasNextPage,
    },
  })
})

Input Validation and Sanitisation Link to heading

Input validation is security 101, but implementing it comprehensively across an API requires systematic thinking.

Schema-Based Validation Link to heading

Using a schema validation library made validation consistent and maintainable:

const Joi = require("joi")

const userSchema = Joi.object({
  name: Joi.string().trim().min(2).max(50).required(),
  email: Joi.string().email().lowercase().required(),
  age: Joi.number().integer().min(13).max(120).optional(),
})

app.post("/users", async (req, res) => {
  try {
    const validatedData = await userSchema.validateAsync(req.body, {
      stripUnknown: true,
      abortEarly: false,
    })

    const user = await User.create(validatedData)
    res.status(201).json({ data: user })
  } catch (error) {
    if (error.isJoi) {
      return res.status(400).json({
        error: {
          type: "validation_error",
          details: error.details.map((detail) => ({
            field: detail.path.join("."),
            message: detail.message,
            value: detail.context.value,
          })),
        },
      })
    }
    throw error
  }
})

Authentication and Authorisation Link to heading

The difference between authentication (who you are) and authorisation (what you can do) became critical when implementing role-based access control.

JWT Implementation Link to heading

JWTs work well for stateless authentication but require careful implementation:

const jwt = require("jsonwebtoken")

// Authentication middleware
const authenticate = (req, res, next) => {
  const token = req.headers.authorization?.replace("Bearer ", "")

  if (!token) {
    return res.status(401).json({
      error: { type: "authentication_required", message: "Token required" },
    })
  }

  try {
    const payload = jwt.verify(token, process.env.JWT_SECRET)
    req.user = payload
    next()
  } catch (error) {
    return res.status(401).json({
      error: { type: "invalid_token", message: "Token is invalid or expired" },
    })
  }
}

// Authorisation middleware
const authorize = (permissions) => (req, res, next) => {
  const hasPermission = permissions.some((permission) => req.user.permissions.includes(permission))

  if (!hasPermission) {
    return res.status(403).json({
      error: {
        type: "insufficient_permissions",
        message: "You do not have permission to access this resource",
        required_permissions: permissions,
      },
    })
  }

  next()
}

Documentation That Actually Helps Link to heading

Good API documentation is code that teaches itself. I learned to write documentation from the user’s perspective, not the implementer’s.

OpenAPI/Swagger Integration Link to heading

Generating documentation from code ensures it stays current:

/**
 * @swagger
 * /users:
 *   post:
 *     summary: Create a new user
 *     tags: [Users]
 *     requestBody:
 *       required: true
 *       content:
 *         application/json:
 *           schema:
 *             type: object
 *             required:
 *               - name
 *               - email
 *             properties:
 *               name:
 *                 type: string
 *                 example: "John Doe"
 *               email:
 *                 type: string
 *                 format: email
 *                 example: "john@example.com"
 *     responses:
 *       201:
 *         description: User created successfully
 */

Rate Limiting and Security Link to heading

Rate limiting protects your API from abuse and ensures fair usage among clients.

Sliding Window Rate Limiting Link to heading

const rateLimit = require("express-rate-limit")

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // Limit each IP to 100 requests per windowMs
  message: {
    error: {
      type: "rate_limit_exceeded",
      message: "Too many requests from this IP",
      retry_after: Math.ceil(windowMs / 1000),
    },
  },
  standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
  legacyHeaders: false, // Disable the `X-RateLimit-*` headers
})

Testing Strategy Link to heading

API testing requires multiple layers: unit tests for business logic, integration tests for endpoints, and contract tests for client compatibility.

Integration Test Example Link to heading

describe("POST /users", () => {
  it("should create a new user with valid data", async () => {
    const userData = {
      name: "John Doe",
      email: "john@example.com",
    }

    const response = await request(app).post("/users").send(userData).expect(201)

    expect(response.body.data).toMatchObject({
      name: userData.name,
      email: userData.email.toLowerCase(),
      id: expect.any(String),
    })

    // Verify user was actually created in database
    const user = await User.findById(response.body.data.id)
    expect(user).toBeTruthy()
  })

  it("should return validation errors for invalid data", async () => {
    const response = await request(app).post("/users").send({ name: "", email: "invalid-email" }).expect(400)

    expect(response.body.error.type).toBe("validation_error")
    expect(response.body.error.details).toHaveLength(2)
  })
})

Key Takeaways Link to heading

Building APIs that are both powerful and pleasant to use requires attention to many details:

  1. Consistency trumps cleverness - Predictable patterns are more valuable than optimal solutions
  2. Error handling is a feature - Good error responses are as important as successful responses
  3. Version from day one - Having a versioning strategy before you need it saves pain later
  4. Performance matters early - Pagination and query optimisation issues compound quickly
  5. Security by design - Authentication, authorisation, and input validation can’t be afterthoughts
  6. Documentation as code - Generate docs from your implementation to keep them accurate

The best APIs feel intuitive to use and handle edge cases gracefully. They’re the result of thinking from the client’s perspective and iterating based on real usage patterns.

What API design lessons have you learned from real-world projects? Which aspects do you find most challenging to get right from the start?