Richer frontmatter support in gatsby-plugin-mdx

[ChristopherBiscardi]

Summary

We use gray-matter for gatsby-plugin-mdx frontmatter support. We should passthrough the excerpt_separator option.

Basic example

We would likely just enable passthrough in the options as such:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-mdx`,
      options: {
        grayMatter: {
          excerpt_separator: "<<<my-separator>>>"
        }
      }
    }
  ]
};

Would set this mdx content:

---
title: Blog
---
My awesome blog.
<<<my-separator>>>
<h1>Hello world</h1>

to this result:

{ 
  data: { title: 'Blog'}, 
  excerpt: 'My awesome blog.', 
  content: 'My awesome blog.\n<<<my-separator>>>\n<h1>Hello world</h1>' 
}

Open Questions

• Does the excerpt stay in the content afterwards?
• gray-matter supports a lot of options, some of which are literal functions. Is there a better way to support those since gatsby-config is serialized?
• How does the excerpt interact with the gatsby-plugin-mdx excerpt resolver. Currently if there is an excerpt we'd prefer it over the resolver.
• What happens to react components in excerpts? currently we don't render them.
• We should probably test what already works (like frontmatter language inference) before attempting to add this functionality.

Motivation

When porting from other ecosystems it can be easier if we support their frontmatter delimiters and excerpt formats. That was the use case in the original gatsby-mdx issues

ported from ChristopherBiscardi/gatsby-mdx#386

[varya]

Is this still actual?

[jsg2021]

Yeah, its still an issue.

It would be nice to pass options to grayMatter from the mdx plugin.

[MrSaints]

I'd love to see this supported.

I had a go at trying to implement it, and it's not easy. I attempted:

1. To pass just the excerpt (extracted using gray-matter to genMDX, but I realised that this does not account for cases where there are markdown links defined later on in the markdown / MDX file (in the raw node data)
2. To compile the HTML so that I can simply "split" by the separator, no luck here
3. To process body returned from genMDX, but it is Babel transformed code
4. To process the AST, and compile it to the Babel transformed code, but no luck here. Relatively convoluted code

EDIT: I have come up with a simple, and non-intrusive solution in the meantime.

Instead of using a separator like <!--separator-->, I have wrapped the excerpt with <div data-excerpt> ... </div>, ensuring I leave enough space between my paragraph so that it retains the markdown formatting.

Now, comes the hacks. I then use body with MDXRenderer like so:

<MDXRenderer onlyExcerpt={true}>{node.body}</MDXRenderer>

You can configure what gets rendered through changing the wrapper:

<MDXProvider
    components={{
        wrapper: ({ onlyExcerpt = false, children }) => {
            let updatedChildren = [ ...children ];

            if (onlyExcerpt) {
                updatedChildren = children.filter((child, _) => {
                  return child.props && child.props["data-excerpt"];
                });
            }

            return (
                <>{updatedChildren}</>
            );
        },
    })
/>

This also guarantees MDX processing on your content, with your options / remark plugins too.

I'm sure you can configure this for even finer grain content filtering.

[mathieudutour]

We should take the same approach as gatsby-transformer-remark, and unify the options and behaviors.

[varya]

@MrSaints Thank you for this tip. I managed to implement much better excerpts for my blog than it used to be with remark.

[joyfulelement]

Blocker from adopting gatsby-plugin-mdx

Without the support of excerpt_separator that gatsby-transformer-remark had, this was my main blocker from migrating and adopting gatsby-plugin-mdx.

Keen to find out if this will be supported anytime soon?

Alternatives

Hi @MrSaints @varya would you be able to share how it was implemented for your use case without the need of excerpt_separator? With my limited understanding with <MDXProvider>, I put it together with <MDXRenderer> as follow base on @MrSaints' earlier example:

<MDXProvider
  components={{
    wrapper: ({ onlyExcerpt = false, children }) => {
      let updatedChildren = [...children];

      if (onlyExcerpt) {
        updatedChildren = children.filter((child, _) => {
          return child.props && child.props['data-excerpt'];
        });
      }

      return <>{updatedChildren}</>;
    }
  }}
>
  <MDXRenderer frontmatter={frontmatter} onlyExcerpt={true}>
    {props.post.body}
  </MDXRenderer>
</MDXProvider>

In index.md, I have converted:

---
frontmatters: xyz
---
post excerpts...<!-- excerpt-end -->

main post content...

to index.mdx with:

---
frontmatters: xyz
---
<div data-excerpt>post excerpts...</div>

main post content...

However, nothing gets rendered including both excerpts or main post content from this attempt.

Would be very much appreciated if you could provide me the pointers for how this alternative solution could be implemented? I may have misunderstood the usage from the earlier example. Thank you~!

[MrSaints]

@joyfulelement I'm not exactly sure why it's not working for you, but there are two things I'd suggest from what I can see (lacking additional context):

1. Using props.mdx.body instead

In your code:

{props.post.body}

I'm not sure how you obtained post (as it may be an alias), but in my case, I used data.mdx.body where data comes from my props, and it is populated with page query using mdx in the GraphQL.

2. Leaving some space around the element

Instead of:

<div data-excerpt>post excerpts...</div>

Try:

<div data-excerpt>

Content

</div>

Otherwise, it's implemented exactly as how I shared it. And it's currently powering my blog.

[joyfulelement]

1. Using props.mdx.body instead

In your code:

{props.post.body}

I'm not sure how you obtained post (as it may be an alias), but in my case, I used data.mdx.body where data comes from my props, and it is populated with page query using mdx in the GraphQL.

Thanks @MrSaints for the guidance~
I was curious if the data can be base on the excerpt value queried from mdx GraphQL, instead of the body field? i.e.

mdx {
      id
      body
      excerpt <--- can the solution be base on the excerpt terminator <div data-excerpt>excerpts...</div>?
      ...
}

The reason I am asking is because I had GraphQL queries deliberately not querying for the full body field since most of time, excerpt field account for <5% of the entire post content (maybe even less now compare with mdx body) in the hope to improve the query/build performance.

I will give it a try with the mdx.body to verify the expected behaviour.

2. Leaving some space around the element

Instead of:

<div data-excerpt>post excerpts...</div>

Try:

<div data-excerpt>

Content

</div>

Thanks for the pointer about spacing. Didn't know the spacing could have contributed the issue. I'll try to verify this behaviour as well.

Otherwise, it's implemented exactly as how I shared it. And it's currently powering my blog.

I am grateful to have your validation on the implementation details~ It's a beautiful website you got there 👍 I did notice the link Handcrafted with love by Ian at the footer points to 404 page on Github, probably intentional for private repo, but if not, thought it's worth pointing out.

[MrSaints]

Apologies for the late response @joyfulelement.

I was curious if the data can be base on the excerpt value queried from mdx GraphQL, instead of the body field? i.e.

AFAIK, no. You have to use the body. Because the excerpt would have filtered out any non-text elements, plus, it is truncated.

The reason I am asking is because I had GraphQL queries deliberately not querying for the full body field since most of time, excerpt field account for <5% of the entire post content (maybe even less now compare with mdx body) in the hope to improve the query/build performance.

I'd see how well it performs before optimising. Personally, didn't have too much of an issue on this front.

I did notice the link Handcrafted with love by Ian at the footer points to 404 page on Github

Thanks for your kind words. And yes, it is intentional, for now. I'm in the process of cleaning up before I make it public.

[theowenyoung]

Is there any update here?

[suhasdara]

To those who are still looking for a solution until something is implemented in the library itself, you can read my short blog where I gave a fix (temporary) for this:
https://suhasdara.me/blogs/gatsby-mdx-excerpts

[darylwright]

If the problem is determining a way to customize content excerpts, why not just add an excerpt field to frontmatter and display it if it's not null or empty (otherwise display the default excerpt)?

// This is located in the render function of the excerpt displaying component
const excerpt = post.frontmatter.excerpt && post.frontmatter.excerpt.length > 0
    ? post.frontmatter.excerpt
    : post.excerpt;

I'm sure there's good reasons for solving this problem at the resolver level, I just don't know what those are yet. The above approach gets me the desired end result with the least amount of complexity. This may not be the case for everyone with this same issue.

I've only tested with text, but it should be easy to use HTML or Markdown in the frontmatter excerpt and render them accordingly.