My experience with Gatsby v5 to Next v13 frontend migration

Our Gatsby based statically generated frontend had been serving us well in production for 3+ years. But, after living with some major content publishing pain-points (detailed below) for long, we decided to move to Next.js to permanently resolve them. Also, to remain future-ready, we moved to the Next.js App router instead of the older Next.js Pages router.

1. About our setup

Our website consists of thousands of web pages, most of which are informational articles. The information served on these pages is managed by our content team through our CMS. On a typical day, our content team makes dozens of non-trivial content changes to our website (content update for existing URLs or publishing new web pages). A few of our web pages also serve interactive tools & utilities for our website visitors to use.

Our setup has been built in a way that it allows the content publishing team to:

• Compose any of our site pages with any of the available (90+) section types.
• Create a new slug, category, section or redirection-rule without code changes.
• Change content within any of the global pages or global sections (like menus, headers or footers) without code changes.

2. Our Gatsby pain points

Following were the major pain-points that triggered the move from Gatsby to Next.js for serving our frontend:

2.1 Lack of Incremental Static Rendering

Every time our content team publishes a new page or makes changes to an existing one, they want to be able to take the changes live in a few seconds.

Gatsby provides Deferred Static Generation for a faster go-live of changes. But, even with DSG, every time we published a change, we had to re-run the build step for Gatsby to re-fetch the data for all the pages (and not just the changes). For our setup, not only did this consume 10+ minutes to run, it also resulted in two more issues:

• With the growing amount of data in our setup, the build duration had been increasing.
• With every content publish leading to a fetch of all the content, a lot of needless CPU, memory and disk resource on our CMS side was being consumed.

Gatsby also provides Incremental Builds for faster go-live of changes. But, it expects the source plugin and API to provide the delta-update capability. Since our frontend fetches data from multiple sources, making this feature work for us would have meant considerable custom development effort.

For a content team that frequently publishes changes, lack of an ability to instantly take changes live acted as a major productivity-drag.

2.2 Lack of Instant Previews

Prior to taking the changes live, our content team members seek to be able to preview their changes somewhere.

Gatsby provides plugins to instantly preview the content added into the CMS. But, we could not get the strapi-plugin-gatsby-preview to work on our setup. As a result, we had been living with the following workarounds for our content team to preview the changes:

• try the changes in a non-production CMS and then re-do those in the production CMS
• leverage a custom CMS field along with a non-production frontend to preview the changes

Because both the above workflows required the content team to perform additional steps every time they made changes to be previewed, they were error-prone.

2.3 JavaScript bundle sizes

While this was not a primary pain-point, the introduction of INP (Interaction to Next Paint) as a Google core-web-vital has prompted us to look for ways to reduce our JavaScript bundle sizes in the near-future. For informational article pages on our website, we see React Server Components as a potential approach to solve this.

While Gatsby has released Partial Hydration (in beta), the known limitations of its approach (listed below) are a deal-breaker for us (esp, not working in develop mode):

• Only works with gatsby build/ gatsby serve, and not gatsby develop.
• Bundling of JavaScript is not fully optimized for Partial Hydration, you might still see large JavaScript files be downloaded
• Error handling is missing during static rendering

Also, the fact that Gatsby team anticipates a longer timeline to address these limitations (see here) did not help our cause.

3. Undertaking a Next.js proof-of-concept

Once Next.js was identified as the framework that could address our issues, we followed-it up with a proof-of-concept to be certain of our findings. So, before starting with the actual frontend migration, we identified:

• the most complicated parts of our frontend code
• the non-negotiable aspects of our website features, web dev process & content workflow

We then validated these aspects with Next.js v13 to ascertain our path ahead:

3.1 Handling our fetch logic

Our existing setup allowed our content team to build any URL pattern from any CMS table without requiring dev support / code changes.

To handle such a flexible content policy with Next.js, we built a catch-all dynamic segment within our code. Also, this dynamic catch-all segment needed to find which CMS table to query to obtain the data for the current slug. To resolve this, we built a logic to fetch & cache the slug ⇒ CMS table mapping on the frontend (with adequate cache invalidation & re-fetch mechanism in place).

3.2 Migrating our chakra-ui based UI components

Our frontend UI components were built using chakra-ui. Since chakra-ui is a CSS-in-JS library, our UI components couldn’t be rendered as React Server Components. Knowing this, we validated the following:

• When using the use client; directive, our page content should still be server-side-rendered (for SEO and faster content display purposes).
• By using the Next.js App router, we have a future pathway to leverage React Server Components as we gradually move our UI components from chakra-ui to a non CSS-in-JS styling library.

3.3 Setting up ISR (instantly take changes live)

Our content team wanted to be able to take the changes (publishing a new URL or modifying an existing one) live without re-running the build process.

Next.js dynamic ISR (Incremental Static Re-generation), with the following two constant values within our pages.tsx, solved this for us:

export const dynamicParams = true;
export const revalidate = false;

The above values mean:

• Our pages got served from cache until we invalidate / rebuild.
• We could create new URLs at runtime that did not exist during build time.

In addition to the above, we created a custom route within our Next.js code to invalidate any URL. We set up our CMS to call this route whenever a content change got published.

The above setup ensured that our content team could take changes live within a few seconds after clicking the publish button on our CMS (without all the re-fetching and re-building steps).

3.4 Instant previews

Our content team wanted to preview their edits before taking the change live.

The Draft Mode feature from Next.js, combined with ISR setup detailed above made this work for us. Whenever a request for a certain url contains a Draft Mode cookie, such request is served by fetching data from the CMS instead of serving the existing previously generated page. All this, while rest of the world continues to see the currently-in-production statically generated page.

We set up a Preview button within our CMS to leverage this feature.

4. Undertaking the frontend code migration

Once the most complicated aspects and must-haves for our frontend were validated, we used our proof-of-concept repository as the starting point for migrating our UI code. Here on, we undertook the following:

• We migrated our UI components from Gatsby to Next repository. Since both are React frameworks and, we weren’t changing any UI / styling libraries at this point, this step involved limited changes.
• We migrated our build-step data-fetching code from Gatsby to Next repository. Since both differ in static-site generation (e.g. : Next.js generateStaticParams vs Gatsby gatsby-node)- we had to re-write how our build-time data-fetching code was structured (while not changing the data-fetching calls being made to our CMS & other data sources).
• We also moved code to handle aspects like redirects, sitemaps, etc. Again, with Next.js being very different from Gatsby in these aspects, we had to mostly rewrite these features within our code.

The frontend migration process was complemented with a thorough QA to ensure no regressions from our existing Gatsby website.

5. Going Live

In general, it would be ideal to incrementally migrate a few page-types, take them live & so on. However, we could not leverage incremental migration because:

• Our Gatsby site was hosted on the Gatsby cloud and the Next.js site was to be hosted on Vercel. As a result, building a mechanism that could incrementally serve different pages from different providers could be tricky.
• Since most of our web pages can have any kind of sections (components), an incremental migration would require us to closely synchronize things with the content team.

As a result, we went live with our Next.js site via a single DNS switch once all our code was migrated and changes were validated. However, this approach required us to keep synchronizing the UI code changes being made to our Gatsby repository while our Next.js frontend code migration was in progress.

6. Conclusion

Executing this frontend migration, we experienced the following:

• Since both the frameworks are React based meant we could move a lot of components UI code between the two frameworks with a few changes.
• There were still a few Next.js - Gatsby differences (like using Next.js fetch API, serving sitemaps, etc.) that required rewriting code, so the frontend migration effort was still substantial.
• Next.js appears to be solving our content publishing workflow requirements and offering us a pathway to move to React Server Components. We hope this serves us well in the coming years.

Note: The actual code migration was executed by the client’s in-house frontend team while I performed the initial PoC + setup & resolved the tech issues encountered during the migration (tech-mentoring the migration).