Recommended Testing Methodology

[spencerbn]

Is your feature request related to a problem? Please describe. I am using the express-openapi-validator for various services at my company. Currently the only way we have found to effectively test if our requests will pass the validation is using superagent and then hitting it with real requests.

However this is a very slow way to test since it has to spin up an express app. Ideally there would be a way to write tests that pass in the expected request body and evaluate them against the service's open api schema.

Describe the solution you'd like Does anyone know of a unit-test-ish approach to validating request against the open api schema?

[cdimascio]

You might also consider posting this question to StackOverflow for wider reach.

[msimerson]

the only way we have found to effectively test if our requests will pass the validation is using superagent and then hitting it with real requests.

I've never considered doing it another way, and I don't find it slow. I guess YMMV?

const assert    = require('assert')
const supertest = require('supertest')

const app  = require('../../api/app')
const base = require('../base')
const user = require('../../lib/user')

before(async function () {
  await base.destroyTestObjects(user, [{ id: 7 }])
})

after(async function () {
  await base.destroyTestObjects(user, [{ id: 7 }])
  await user._mongo.disconnect
})

describe('user/routes', function () {
  describe('login as [email protected]', function () {
    const agent = supertest.agent(app)

    it('valid user can authenticate and create session', async function () {
      const res = await agent
        .post('/api/v1/user/session')
        .send({ username: '[email protected]', password: '*********' })
        .expect('Content-Type', /json/)
        .expect(200)

      assert.deepStrictEqual(res.body.user, testUsers[0])
      base.checkError(null, res)
    })

    it('GET /user/0 is same result as login', async function () {
      const res = await agent
        .get('/api/v1/user/0')
        .expect('Content-Type', /json/)
        .expect(200)

      assert.deepStrictEqual(res.body.user, expectedUsers[0])
      base.checkError(null, res)
    })

<dozens more tests...>
})

When run with API validation enabled:

  user/routes
    login as [email protected]
POST /api/v1/user/session 200 76.311 ms - 491
      ✔ valid user can authenticate and create session (95ms)
GET /api/v1/user/0 200 8.684 ms - 553
      ✔ GET /user/0 is same result as login
GET /api/v1/user/1 200 5.323 ms - 553
      ✔ user can retrieve self via id
POST /api/v1/user 200 47.632 ms - 58
      ✔ can create a user (51ms)
DELETE /api/v1/user/7 200 6.186 ms - 26
      ✔ can delete a user

When run without API validation:

  user/routes
    login as [email protected]
POST /api/v1/user/session 200 63.954 ms - 491
      ✔ valid user can authenticate and create session (83ms)
GET /api/v1/user/0 200 6.467 ms - 553
      ✔ GET /user/0 is same result as login
GET /api/v1/user/1 200 4.610 ms - 553
      ✔ user can retrieve self via id
POST /api/v1/user 200 46.381 ms - 58
      ✔ can create a user (49ms)
DELETE /api/v1/user/7 200 4.730 ms - 26
      ✔ can delete a user

We can see that API validation adds a few milliseconds of processing time to each request. The longer time taken for auth is a security feature and that time is spent doing crypto. Most of the rest of each requests time is in the DB layer. IMO, the HTTP bits take inconsequential bits of time.