My experience with upgrading Strapi v3 to v4
Because of the large number of breaking changes introduced with Strapi v4, upgrading a Strapi setup from v3 ⇒ v4 is tricky in general.
It didn’t help that our Strapi v3 setup (in production since 2019) consisted of many collections with dynamic zones, components & relationships that translated into 1000+ database tables (a lot of data to migrate and test!). Our v3 setup also consisted of a few customizations that required a complete rewrite to carry them forward to our v4 setup.
We started our upgrade exercise with the two-fold objective of upgrading in the smallest possible downtime window without any functional regression.
1. The End Goal of our Exercise
The end goal of our exercise was to have our CMS running on Strapi v4 with:
- no data / functionality regressions
- a downtime of 2–4 hours for our Strapi CMS setup
- no frontend website downtime
Achieving the above required us to create a well-tested Strapi upgrade suite that could be run on top of our Strapi v3 setup within an hour (leaving the rest of the CMS downtime for verification).
2. Creating our Strapi upgrade suite
A few weeks prior to our planned CMS upgrade, we started to work on creating an upgrade suite. The objective here was to have a bunch of scripts that could be run on top of our Strapi v3 setup during the downtime to upgrade it to Strapi v4.
2.1 Upgrading our Strapi code
With Strapi, the schema JSON files within the code act as the source of truth with respect to the data schema. The goal of this phase was to create scripts to migrate our schema (residing in the code) from v3 to v4. Running these scripts would enable us to see all our collections, single types, components and dynamic zones in Strapi v4 (minus the data). Also, at this point, we were not focussed on migrating our Strapi customization code (custom middlewares, lifecycle hooks, etc.).
2.1.1 Strapi codemods
We used the Strapi codemods on our v3 repository (via npx @strapi/codemods migrate) to transform our code from v3 to v4. Every time the codemods execution failed, we:
- analyzed & modified the legacy quirks present in our v3 code
- undid all the changes codemods made to our code repo before failing
- reran the codemods
We repeated the above until our codemods execution succeeded. Upon success, we committed our code changes to a new git branch. We also documented our v3 code modifications for future reference.
2.1.2 Starting Strapi v4
At startup, Strapi mirrors the schema present in the code into the underlying database. So, to reflect our just upgraded Strapi v4 code into the database, we pointed it to an empty schema and attempted to start it (yarn develop).
We encountered multiple instances of errors of the kind - error: create index "<super_long_name>" on "public" already exists for some of our components. This required us to manually shorten the collectionName values for these components. We documented each of such renaming for future reference (we’ll see later when & why this is needed).
Once our Strapi setup started, we logged into this new setup to verify if our schema definitions (collections, single types, components, fields) had migrated as expected. We also committed our code changes at this point in time.
All the manual steps in this phase were easy to automate. So, at the end of this phase, we had the Strapi codemods scripts and a bunch of other scripts that we could execute at any point to upgrade our v3 schema to v4 schema.
2.2 Strapi v3 ⇒ v4 data migration
The goal of this phase was to have scripts that could populate our upgraded v4 schema with the data migrated from our v3 setup (both residing in a Postgres database).
Since we could only keep our CMS admin users away from the CMS for 2-4 hours (our downtime window), having data-migration scripts that could migrate our data in a few minutes was an absolute must for us.
2.2.1 Strapi data migration scripts
Originally started as part of the Hacktoberfest 2022, Strapi now provides a bunch of data migration scripts that are inevitable to successfully migrate the data from v3 schema to v4 schema.
We ran these scripts with .env pointing to our v3 and v4 databases (yarn start > migration-log-20230622-run-001.log within v3-sql-v4-sql). While our migration script run succeeded, we observed that a lot of data did not actually migrate. We eventually realized that any mismatch during data-migration caused the migration scripts to simply skip migrating certain tables / columns / rows.
To identify the misses, we analyzed the migration output logs for warnings like:
DESTINATION TABLE components_transaction_transaction_entries DOES NOT EXISTS
We also performed a manual verification of the migrated data (especially relationships, components within components, dynamic zones, relationships within the components).
Some of the misses could be attributed to the component renaming we were forced to do earlier (see 2.1.2). Other misses were due to the data-migration script missing many edge case scenarios (like this and this). After root-causing and addressing every instance of missing-data, we ended up with a dozen+ modifications to the data-migration scripts before we had all of our v3 data appearing on our v4 setup.
But, the good news was that we now had a forked version of the data-migration scripts that we could run at any time to migrate our v3 data to v4 within a few minutes.
2.3 Upgrading our Strapi customizations
From lifecycle hooks to Strapi plugins, there has been a notable change in the way Strapi v4 expects the customizations to be written. As a result, we were required to re-implement most of the customizations present in our v3 setup. Some examples of such re-implementations:
a. With Strapi v3, we leveraged Strapi lifecycle hooks beforeCreate and beforeUpdate for custom input validations. But, with Strapi v4, the component data isn’t available within these lifecycle hooks. This resulted in us re-writing these input validations via middlewares (detailed here).
b. Our Strapi v3 content manager UI was modified by overriding via extensions/content-manager. This approach has been discontinued with Strapi v4. So, we had to rewrite some of these modifications via the Hooks API and others via patch-package.
To make upgrading our Strapi customizations re-runnable at any point, we needed to write codemods scripts. Since this would require considerable time & effort, we chose to rely on one-time manual upgrade of our customizations. The down-side of our choice was that we had to freeze creating any new customizations within our v3 setup for a few weeks. Since this was acceptable to the business, we avoided writing automation scripts to upgrade our Strapi customizations.
2.4 Updating the Rest API calls (Frontend)
Once we had a running Strapi v4 setup available (with data migrated & customizations re-implemented), we needed our frontend to work with our Strapi v4 setup.
With v4, the Rest API parameters have gone through a notable change (for example, the populate parameter). As a result, the frontend team was required to make substantial changes to the Rest APIs invoked to fetch the same data as earlier. The goal here was to modify the data fetching layer of our frontend code so that the rest of the frontend could function as-is.
The frontend team picked each website section and updated the Rest API parameters & shape of the response as needed. This was complemented with a frontend verification exercise to identify & address any regressions. At the end of this phase, we now had a code branch in our frontend repo that could run on top of the Strapi v4 backend.
Like with Strapi customizations upgrade, we decided to not create re-runnable codemods scripts for this phase of the upgrade. Instead, we relied on a one-time manual upgrade of the data-fetching layer. As a result of this choice, major changes to the data-fetching layer of our frontend code were restricted and smaller changes were carefully handled (merge conflicts) for a few weeks.
3. Executing the Strapi upgrade
At the end of creation of our Strapi upgrade suite (and closer to the date of Strapi v4 upgrade in production), we had the following things ready:
- The scripts to upgrade our schema and data at any point.
- The code branch with the upgraded Strapi customizations.
- The code branch with the upgraded data-fetching layer on the frontend.
On the day of the upgrade, we undertook the following during the available downtime for the CMS:
- We executed the codemods and the data-migration scripts within a few minutes.
- The available code for Strapi customizations and data-fetching layer was merged into the respective main branches.
- The rest of the available downtime was used for chores like backups, verification, etc.
For the duration of the CMS downtime:
- The production frontend and the Strapi v3 Rest APIs setup kept running as-is.
- The upgrade was performed & verified on a different / parallel infrastructure.
- Post verification, the new setup was taken live via a DNS switch to this parallel infrastructure.
The above flow ensured zero down-time and no major data / functionality issues on the frontend. It also enabled us to perform the upgrade in a way that, if needed, we could go back to the earlier (Strapi v3) setup at any point prior to the Go-live.
Note : All of the above steps were planned & rehearsed multiple times with all the stakeholders to avoid unforeseen issues.
4. Conclusion
Because of the number of changes between Strapi v3 and v4, the Strapi v3 ⇒ v4 remains a challenging exercise that requires careful planning and tedious data / functionality verification at multiple phases.
Despite the challenges, the availability of the Strapi codemods library & data migration scripts made it possible for us to build an automated upgrade suite for our Strapi setup. This, in-turn, enabled us to achieve an error-free Strapi v3 ⇒ v4 upgrade within an acceptable downtime window.
Back in mid-2021, one of my clients was having issues with their in-house CMS. Despite me being their frontend architect, they trusted me to build their CMS solution. At this point, evaluation of various frameworks, approaches and products led me to Strapi. And, I ended up implementing Strapi CMS for their data requirements.
Fast-forward to now, I have worked with multiple orgs on implementing, upgrading & customizing Strapi setups based on their requirements.