initial commit

Author: jescalan

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+node_modules

.prettierrc

@@ -0,0 +1,4 @@
+{
+  "semi": false,
+  "singleQuote": true
+}

README.md

@@ -0,0 +1,90 @@
+# Next MDX Remote
+
+A set of light utilities allowing mdx to be loaded within `getStaticProps` or `getServerSideProps` and hydrated correctly on the client.
+
+### Background & Theory
+
+If you are using mdx within a nextjs app, you are probably using the webpack loader. This means that you have your mdx files locally and are probably using [next-mdx-enhanced](https://github.com/hashicorp/next-mdx-enhanced) in order to be able to render your mdx files into layouts and import their front matter to create index pages. This workflow is fine, but introduces a few limitations that we aim to remove with `next-mdx-remote`:
+
+- The file content must be local. You cannot store mdx files in another repo, a database, etc. For a large enough operation, there will end up being a split between those authoring content and those working on presentation of the content. Overlapping these two concerns in the same repo makes a more difficult workflow for everyone.
+- You are bound to filesystem-based routing. Your pages are generated with urls according to their locations. Or maybe you remap them using `exportPathMap`, which creates confusion for authors. Regardless, moving pages around in any way breaks things -- either the page's url or your `exportPathMap` configuration.
+- You will end up running into performance issues. Webpack is a javascript bundler, forcing it to load hundreds/thousands of pages of text content will blow out your memory requirements - webpack stores each page as a distinct object with a large amount of metadata. One of our implementations with a couple hundred pages hit more than 8gb of memory required to compile the site. Builds took more than 25 minutes.
+- You will be limited in the ways you are able to structure relational data. Organizing content into dynamic, related categories is difficult when your entire data structure is front matter parsed into javascript objects and held in memory.
+
+So, `next-mdx-remote` changes the entire pattern so that you load your mdx content not through an import, but rather through `getStaticProps` or `getServerProps` -- you know, the same way you would load any other data. The library provides the tools to serialize and hydrate the mdx content in a manner that is performant. This removed all of the limitations listed above, and does so at a significantly lower cost -- `next-mdx-enhanced` is a very heavy library with a lot of custom logic and [some annoying limitations](https://github.com/hashicorp/next-mdx-enhanced/issues/17). Early testing has shown build times reduced by 50% or more.
+
+### Installation
+
+```
+npm i next-mdx-remote
+```
+
+### Usage
+
+This library exposes two functions, `renderToString` and `hydrate`, much like `react-dom`. These two are purposefully isolated into their own files -- `renderToString` is intended to be run **server-side**, so within `getStaticProps`, which runs on the server/at build time. `hydrate` on the other hand is intended to be run on the client side, in the browser.
+
+```jsx
+import renderToString from 'next-mdx-enhanced/render-to-string'
+import hydrate from 'next-mdx-enhanced/hydrate'
+import Test from '../components/test'
+
+const components = { Test }
+
+export default function TestPage({ mdxSource }) {
+  const content = hydrate(mdxSource, components)
+  return <div className="wrapper">{content}</div>
+}
+
+export async function getStaticProps() {
+  // mdx text - can be from a local file, database, anywhere
+  const source = 'Some **mdx** text, with a component <Test />'
+  const mdxSource = await renderToString(source, components)
+  return { props: { mdxSource } }
+}
+```
+
+While it may seem strange to see these two in the same file, this is one of the cool things about next.js -- `getStaticProps` and `TestPage`, while appearing in the same file, run in two different places. Ultimately your browser bundle will not include `getStaticProps` at all, or any of the functions only it uses, so `renderToString` will be removed from the browser bundle entirely.
+
+Let's break down each function:
+
+- `renderToString(source: string, components: object)` - This function consumes a string of mdx along with any components it utilizes in the format `{ ComponentName: ActualComponent }`. The function returns an object that is intended to be passed into `hydrate` directly.
+- `hydrate(source: object, components: object)` - This function consumes the output of `renderToString` as well as the same components argument as `renderToString`. Its result can be rendered directly into your component. This function will initially render static content, and hydrate it when the browser isn't busy with higher priority tasks.
+
+### Frontmatter & Custom Processing
+
+Markdown in general is often paired with frontmatter, and normally this means adding some extra custom processing to the way markdown is handled. Luckily, this can be done entirely independently of `next-mdx-remote`, along with any extra custom processing necessary. Let's walk through an example of how we could process frontmatter out of our mdx source.
+
+```jsx
+import renderToString from 'next-mdx-enhanced/render-to-string'
+import hydrate from 'next-mdx-enhanced/hydrate'
+import Test from '../components/test'
+import matter from 'gray-matter'
+
+const components = { Test }
+
+export default function TestPage({ mdxSource, frontMatter }) {
+  const content = hydrate(mdxSource, components)
+  return (
+    <div className="wrapper">
+      <h1>{frontMatter.title}</h1>
+      {content}
+    </div>
+  )
+}
+
+export async function getStaticProps() {
+  // mdx text - can be from a local file, database, anywhere
+  const source = 'Some **mdx** text, with a component <Test />'
+  const { content, data } = matter(content)
+  const mdxSource = await renderToString(content, components)
+  return { props: { mdxSource, frontMatter: data } }
+}
+```
+
+Nice and easy - since we get the content as a string originally and have full control, we can run any extra custom processing needed before passing it into `renderToString`, and easily append extra data to the return value from `getStaticProps` without issue.
+
+### Caveats
+
+There's only one caveat here, which is that `import` cannot be used **inside** an mdx file. If you need to use components in your mdx files, they should be provided through the second argument to the `hydrate` and `renderToString` functions.
+
+Hopefully this makes sense, since in order to work, imports must be relative to a file path, and this library allows content to be loaded from anywhere, rather than only loading local content from a set file path.

babel-plugin-mdx-browser.js

@@ -0,0 +1,34 @@
+module.exports = function BabelPluginMdxBrowser() {
+  return {
+    visitor: {
+      // remove all imports, we will add these to scope manually
+      ImportDeclaration(path) {
+        path.remove()
+      },
+      // the `makeShortcode` template is nice for error handling but we
+      // don't need it here as we are manually injecting dependencies
+      VariableDeclaration(path) {
+        // this removes the `makeShortcode` function
+        if (path.node.declarations[0].id.name === 'makeShortcode') {
+          path.remove()
+        }
+
+        // this removes any variable that is set using the `makeShortcode` function
+        if (
+          path.node &&
+          path.node.declarations &&
+          path.node.declarations[0] &&
+          path.node.declarations[0].init &&
+          path.node.declarations[0].init.callee &&
+          path.node.declarations[0].init.callee.name === 'makeShortcode'
+        ) {
+          path.remove()
+        }
+      },
+      // remove the default export
+      ExportDefaultDeclaration(path) {
+        path.replaceWith(path.node.declaration)
+      },
+    },
+  }
+}

hydrate.js

@@ -0,0 +1,48 @@
+import React, { useMemo, useState } from 'react'
+import { mdx } from '@mdx-js/react'
+
+export default function hydrate({ source, renderedOutput }, components) {
+  const [hydrated, setHydrated] = useState(false)
+
+  // our default result is the server-rendered output
+  // we get this in front of users as quickly as possible
+  const [result, setResult] = useState(
+    <span dangerouslySetInnerHTML={{ __html: renderedOutput }} />
+  )
+
+  // if we're on the client side and have not yet hydrated, we hydrate
+  // the mdx content inside requestIdleCallback, since we can be fairly
+  // confident that markdown-embedded components are not a high priority
+  // to get to interactive compared to... anything else on the page.
+  //
+  // once the hydration is complete, we update the state/memo value and
+  // react re-renders for us
+  typeof window !== 'undefined' &&
+    !hydrated &&
+    window.requestIdleCallback(() => {
+      // first we set up the scope which has to include the mdx custom
+      // create element function as well as any components we're using
+      const scope = { mdx, ...components }
+      const keys = Object.keys(scope)
+      const values = Object.values(scope)
+
+      // now we eval the source code using a function constructor
+      // in order for this to work we need to have React, the mdx createElement,
+      // and all our components in scope for the function, which is the case here
+      // we pass the names (via keys) in as the function's args, and execute the
+      // function with the actual values.
+      const hydratedFn = new Function(
+        'React',
+        ...keys,
+        `${source}
+        return React.createElement(MDXContent, {});`
+      )(React, ...values)
+
+      // finally, we flip the hydrated status so this doesn't run again, and set
+      // the output as the new result so that
+      setHydrated(true)
+      setResult(hydratedFn)
+    })
+
+  return useMemo(() => result, [source, result])
+}