How we setup our Strapi API test automation using Jest
One early morning of July 2023, I received an SOS call from a colleague. Some of our Strapi based Rest APIs were serving our website users with data that should not be served to them. Say, users were seeing not only their own orders, but also other users' orders! I panicked. But, I was also puzzled. We frequently deployed changes to production but nothing related to the malfunctioning APIs had changed for weeks. In the next few hours, the issue was root-caused to us having upgraded our Strapi setup (some details here). While a fix was immediately applied to address this, the incident highlighted a need for an automated regression test suite for our Strapi based Rest APIs.
1. Which Rest APIs to automate-test
Our setup uses Strapi extensively. With 100+ collections in our Strapi setup, our website consumes ~150+ Strapi based Rest APIs. Setting-up automation testing for such a large number of Rest APIs is a huge ask.
So, we prioritized writing automated API tests for those Rest APIs that:
- Serve any data that shouldn’t be available to all (eg: users' orders, profile)
- Have any customization (via middlewares, policies, controllers) in place
The above criteria enabled us to narrow our focus on ~20 Rest API endpoints. This also meant that we could have an automated regression test suite integrated into our deployment cycle sooner. Once setup, we can gradually cover more API endpoints depending on the priority of the functionality they affect.
2. Why Jest
We started with Jest because Strapi’s documentation detailed the test-setup steps using Jest. But, based on my experience of having implemented this test-setup, any JavaScript based test framework that can access the global strapi object would work:
const Strapi = require("@strapi/strapi");
let instance;
async function setupStrapi() {
if (!instance) {
console.log('Starting Strapi')
await Strapi().load();
instance = strapi;
await instance.server.mount();
}
return instance;
}
module.exports = { setupStrapi };
We need the global strapi object to be able to perform various CRUD tasks like the following during our tests:
const testUser = await strapi.entityService.create('plugin::users-permissions.user', {
data: {
username: testUname,
email: testEmail,
blocked: false,
firstName: testFname,
lastName: testLname,
role: authenticatedRoleId
}
});
3. How did we setup the test data
Our automated tests expected certain data to be already present within our Strapi setup (eg: all the possible order status values that are stored within our order-status collection). As a result, we:
- Created a helper function
setupDatabase.jsthat would execute once at the beginning of our test. - Created a template database that would act as the starting-point for every test run.
The setupDatabase would create a fresh database from the permanently existing template database. This just-created database would be used by the automated tests and be available after test runs (for any manual probing in case of test failures). As a result of this setup, the template database would not be modified via the test runs at any point in time.
//setupDatabase.js
module.exports = async function (globalConfig, projectConfig) {
const pg = knex({
client: 'pg',
connection: {
host : process.env.TEST_DB_HOST,
port : parseInt(process.env.TEST_DB_PORT),
user : process.env.TEST_DB_UNAME,
password : process.env.TEST_DB_PASS,
}
});
try {
await pg.raw(`DROP DATABASE ${process.env.TEST_DB_NAME};`);
}
catch(err) {
console.log(err);
}
await pg.raw(`CREATE DATABASE ${process.env.TEST_DB_NAME} WITH TEMPLATE ${process.env.TEST_SOURCE_DB_TEMPLATE_NAME};`);
}
We enabled execution of our database setup script via Jest globalSetup configuration:
//package.json
{
...,
"jest": {
"testPathIgnorePatterns": [
"/node_modules/",
".tmp",
".cache"
],
"testEnvironment": "node",
"globalSetup": "./tests/helpers/setupDatabase.js",
"coverageDirectory": "./tests/reports/"
}
}
In addition to the above, any test-specific data would be created by the respective tests during their execution via the globally available strapi object.
4. Writing the tests
Once we had all the setup in place, we got down to writing Jest based API tests. Our orders related tests typically look like the following:
///tests/view-my-orders.test.js
//1. code to create user A
const testUserA = await strapi.entityService.create('plugin::users-permissions.user', {
data: {
...
}
});
//2. code to create order X assign to user A
const userAOrder = await strapi.entityService.create('api::user-order.user-order', {
data: {
...
}
});
//3. code to create user B
const testUserB = await strapi.entityService.create('plugin::users-permissions.user', {
data: {
...
}
});
//4. make API request as User A for /my-orders
//and expect order X in the response.
const testUserAToken = await strapi.plugins['users-permissions'].services.jwt.issue({id: testUserA.id});
await request(strapi.server.httpServer)
.get(`/api/user-order/my-orders`)
.set('Accept', 'application/json')
.set("Content-Type", "application/json")
.set('Authorization', `Bearer ${testUserAToken}`)
.expect(200)
.then((data) => {
const orderItems = data.body?.data;
expect(orderItems.length).toBeGreaterThan(0);
expect(orderItems.filter((t) => t.id === userAOrder.id).length).toBeGreaterThan(0);
return data;
});
//5. makle API request as User B for /my-orders
//and expect order X to be not present in the response
const testUserBToken = await strapi.plugins['users-permissions'].services.jwt.issue({id: testUserB.id});
await request(strapi.server.httpServer)
.get(`/api/user-order/my-orders`)
.set('Accept', 'application/json')
.set("Content-Type", "application/json")
.set('Authorization', `Bearer ${testUserAToken}`)
.expect(200)
.then((data) => {
const orderItems = data.body?.data;
expect(orderItems.filter((t) => t.id === userAOrder.id).length).toBe(0);
return data;
});
We also leveraged jest --coverage to ensure adequate test-coverage for our custom middlewares, policies and controllers.
5. Integrating our automated tests into our deployment cycle
We already leverage Jenkins for our Strapi setup deployment. We added our automated tests (by simply running yarn test after yarn build) during our deployment. We leveraged jest-html-reporter along with nodemailer to report the test results.
6. Conclusion
An automated regression API test suite is vital for any backend system (Strapi or otherwise). With Strapi + Jest + Jenkins, we were able to put such a system in place to avoid regression issues while adding features or making changes to our Strapi CMS setup.
Back in mid-2021, one of my clients was having issues with their in-house CMS. Despite me being their frontend architect, they trusted me to build their CMS solution. At this point, evaluation of various frameworks, approaches and products led me to Strapi. And, I ended up implementing Strapi CMS for their data requirements.
Fast-forward to now, I have worked with multiple orgs on implementing, upgrading & customizing Strapi setups based on their requirements.