At Causal, documentation really matters:

Our product is a new kind of financial planning tool with a low floor and a high ceiling — it’s easy to get started but there’s a lot of deep functionality that unlocks Causal’s full potential. To help users learn this, our documentation needs to be first-class.

Until recently, we hosted our docs on GitBook. This was a great place to get started, but had some key drawbacks:

• Limited control over look-and-feel: GitBook sites aren’t very customisable, and they all feel like developer documentation (our users aren’t devs!)
• Limited support for video: we’ve invested in professionally produced video content, and we wanted the viewing experience to feel “native” to our docs
• Limited control over page layout: we have lots of different resources, and wanted our docs landing page to be the best starting point to access them

To solve these problems, we spent 1 engineer-month building a custom home for our documentation: https://docs.causal.app.

Our content lives in Sanity CMS, and the site is built in React + Next.js, hosted on Vercel. The site now looks and feels like an extension of our product and we’ve received lots of positive feedback — we wish we’d done it a lot sooner!

In this post, we’ll take you behind the scenes of this project:

• Modelling our content on Sanity and migrating our existing GitBook content over
• Implementing search using Sanity’s built-in query language, GROQ
• Using the latest React and Next.js features in the custom React frontend
• Optimizations in our Next.js and Vercel configuration to make our site load as fast as possible.

Backend: Sanity CMS

Our first decision when we started planning our new docs site was where to host our content. There were a few key considerations:

• It had to be easy to update for non-engineers
• It had to be flexible - we wanted to host different types of content (video series, concepts, and guides alongside traditional documentation pages)
• It had to be fast to query
• We needed to be able to migrate our existing ~100 pages of content from GitBook

We settled on Sanity, in part because we knew Linear (whose excellent documentation site was an inspiration to us!) was using them, and in part because some of our team had positive experiences with it in past projects.

Sanity’s strength is its flexibility. Its content model, along with its custom query language, GROQ, easily supported everything we wanted to do. One of our main concerns was supporting custom content inline, such as callouts, collapsible blocks, embedded videos, and more; Sanity handles custom blocks like this without breaking a sweat. It’s also quite fast: our basic queries always resolve in under 100ms, and even our most complex queries never take more than a few hundred milliseconds.

The default editing experience is not as good as GitBook’s, but it was sufficient. There’s a lot more you can do to build custom editing experiences in Sanity, but we didn’t spend any time on that because it wasn’t worth the investment for us.

However, the migration experience left a lot to be desired.

Migrating our content to Sanity

As discussed above, our existing documentation content was all in GitBook. It’s quite easy to export GitBook’s content into Markdown files. There is some custom syntax to represent callouts and embeds, but it is pretty easy to parse.

However, we were shocked to find that Sanity has almost zero guidance on importing Markdown content. The best I could find was this blog post from 2018! So we spent about a week writing and iterating on a script to migrate our existing content. Some of the challenges we ran into along the way:

• Using images. We had to first upload all the images in our GitBook account to Sanity, then query Sanity to get a mapping from filename to image ID to use when creating an image block in Sanity.
• Extracting the title, description, and other metadata from GitBook files.
• Special block types. We had to support GitBook’s code blocks, embed blocks, callout blocks, collapsible (summary/detail) blocks, and table blocks - parsing GitBook’s custom format (some combination of HTML and their own syntax), and converting it to the format used by our custom Sanity blocks
• Internal links. We wanted links between doc pages to be proper references, as opposed to plain text links that could easily break as pages are moved around or renamed. This isn’t too hard to model in Sanity, but figuring out how to import it into Sanity was another matter! We had to dive deep into the source Sanity source code (which, thankfully, is open source) to figure this one out.

You can check out the full script we used for our migration here. It took a lot of iterations to get that script right, but once we ran it, our content hardly needed any manual clean-up

Search

No good documentation site would be complete without a robust search feature. Once we had all our existing content in Sanity (and we’d added some new content too!), we had to figure out how to build just that.

While third party solutions exist - for example, Linear uses Algolia - we wanted to first try to build search directly in Sanity. We figured the complexity and cost savings made it at least worth a shot. And as it turns out, Sanity is flexible and performant enough to support it! While the fuzzy matches are certainly not as good as they would be for a purpose-built tool like Algolia, we’ve found the search to be more than good enough for what we need. Search queries typically take a couple hundred milliseconds, and with caching optimizations (more on that later!) common searches normally feel instant.

You can see a sample of our GROQ query here.

Frontend: Next.js

Once we had our content migrated to Sanity, we started working on the frontend. Causal’s main app uses React and Next.js, so it was a natural choice to use the same for our documentation site. And we were able to take advantage of some of the latest new features in React and Next.js to help our documentation site be blazing fast!

Aside from familiarity, why use React? One of the biggest reasons we wanted to move to our own docs site was to support custom interactive content. Just as Sanity supported this on the backend, React made it easy to implement on the frontend.

The most prominent example is our new video series pages, such as the Causal 101 series. The videos themselves are hosted on YouTube, but we built a custom component to allow observing and controlling the embedded video externally. We reused the same component for inline embedded videos on content pages. And there’s lots more interactivity throughout the site that was easiest to build with React.

Another huge benefit of using React is the massive ecosystem. For interactive elements like collapsible blocks, we used Radix UI’s excellent primitives. For our search UI, we used the popular cmdk library. And for styling, we used Tailwind CSS (which is not React specific, but does integrate effortlessly with Next.js). Altogether, this combination made it very quick and easy to whip up all the pages and components we wanted.

Server Components

The most recent evolution of React’s component architecture was server components: components with code that never runs on the client, but which can still pass props to child components, even if those child components only run on the client. Server components were offered in a stable form in Next.js v13.4 in May 2023. While there has been some controversy around their adoption and usage, we found them to be very useful for this project.

In particular, React’s server components allowed us to keep 100% of our data fetching on the server side. This has a couple of key benefits:

• Pages are almost entirely rendered on the server side, so the client never renders a blank page while waiting for the client-side React code to hydrate the initial content
• Sanity client code never reaches the browser, keeping bundle sizes down

Server-side rendering is not new, and it’s been available in Next.js for years. What is new with React Server Components is where the data fetching can happen. In the old way of doing things, you would implement this data fetching in getServerSideProps then drill it down to all the components that needed it. But using RSC, each component just fetches the data it needs to render itself, using async/ await. This lets us easily compose different components together, without any top-level data fetching orchestration. For example, the documentation page component and the sidebar component can each fetch their own data, then Next.js handles rendering the content on the server, returning what it can, and streaming everything else as it becomes ready.

Optimization & Hosting: Vercel

Since we were using Next.js, it was a natural choice to host our documentation site on Vercel. And because we were already using Vercel for preview deployments in our main repo, hosting our docs site there had 0 additional cost!

As discussed above, we used React Server Components extensively. And while Sanity’s API already resolves queries reasonably quickly, we were able to get even better by using Vercel’s Data Cache. This cache essentially acts like a CDN for anything you fetch during server-side rendering, which was perfect for our use case. Without too much trouble, we were able to set it up so most requests are served almost instantly from a cache, never even hitting Sanity’s API. And with a bit of additional configuration, we cached the pages themselves in Vercel’s CDN, so the server didn’t even have to do any work to render most requests.

In fact, we were even able to cache search request responses in Vercel’s CDN. This means, for common searches, the response will be served from the edge almost instantly.

Conclusion

Overall, our effort to migrate to a custom documentation site was a big success. Customers coming to the site get a premium, snappy experience; we have an easy way to surface many different types of content; and all told, the project only took about one engineer-month start to finish.

Going forward, we’ll continue to invest in high-quality, custom content to help all our users learn and grow with Causal. And we have all the flexibility we need to host it on our own docs site!

This project was a little different from the type of project we typically work on at Causal. But if you’re interested in performance (frontend or backend!), and you care deeply about crafting beautiful user experiences, we’re hiring! Drop us a line here or reach out to lukas@causal.app to learn more.

‍