[RFC] Semantic data models in Gatsby

[freiksenet]

Semantic data models in Gatsby

Currenty Gatsby data and schema models locks data sources and types, especially in "classic" plugins like gatsby-source-filesystem or gatsby-transformer-remark. Usually they add one data type, that can mean many different things. JSON file can be an author, can be a blog post, can be any other arbitrary data. A similar problem arises when some node is created from different sources. MarkdownRemark in Gatsby can come both from .md files, but also from bodies of a CMS posts. It makes types multi-purpose and occasionally convoluted. Not only it'd be convinient to query things by purpose, it also would be convinient to query things coming from different sources, that represent same thing, with the same query.

This proposes a solution - data interfaces. Instead of representing a particular data source, they would represent a particular data purpose. BlogPost would be one, or Author or Comment. Each can be backed by one or multiple data sources.

I see two primary use cases for data interfaces. Themes would use that to solve a particular use cases while not being tied to a data source. Thus there would be a gatsby-theme-blog that can be powered by markdown, mdx or WordPress (or by them all at the same time). When the theme won't be available or aren't suitable for the app/website purpose, the developer could create those interfaces directly to define their data model.

Concise example

Let's use a blog post example. We'll define node interface BlogPost. We will be using a forthcoming feature for GraphQL Interfaces, where they can implement other interfaces. Currently we shim it with a @nodeInterface directive.

interface BlogPost implements Node {
# interface BlogPost @nodeInterface
  id: ID!
  title: String!
  date: Date! @dateformat
  html: String
}

As we are using md we can have it process by gatsby-transformer-remark. In addition we'll be sourcing data from Contentful using gatsby-source-contentful Here are the (abridged) types for them.

type MarkdownRemark implements Node {
  id: ID!
  frontmatter: MarkdownRemarkFrontmatter
  rawMarkdownBody: String
  fileAbsolutePath: String
  html: String
  headings: [String]
}

type MarkdownRemarkFrontmatter {
  title: String!
  date: Date! @dateformat
}

type ContentfulBlogPost implements Node {
  id: ID!
  title: String!
  date: Date! @dateformal
  body: String
  shortSummary: String
}

They are not directly compatible - markdown remark has frontmatter type, contentful has a more compatible structure. Therefore they need adapter types to fit into the BlogPost interface.

type BlogPostMarkdownRemark implements Node & BlogPost
  @childOf(type: "MarkdownRemark")
  @autoCreate {
  id: ID!
  title: String! @proxy(from: "parent.frontmatter.title")
  date: Date! @dateformat @proxy(from: "parent.frontmatter.date")
  html: String @proxy(from: "parent.html")
  headings: [String] @proxy(from: "parent.headings") # It's okay to add extra fields
}

type BlogPostContentful implements Node & BlogPost {
  id: ID!
  title: String! @proxy(from: "parent.title")
  date: Date! @dateformat @proxy(from: "parent.date")
  html: String @proxy(from: "parent.body")
  shortSummary: String @proxy(from: "parent.shortSummary")
}

@autoCreate is a helper directive that creates a node for each parent node. @proxy proxies data from the parent.

Inside the components it's possible to query BlogPost. For types implementing additional functionality, one can use fragment spreads to get that info.

query {
  allBlogPost {
    nodes {
      title
      date
      html
      ... on BlogPostMarkdownRemark {
        headings
      }
    }
  }
}

By having a hierarchy of interfaces (eg BlogPost, BlogPostWithImage, BlogPostWithComments) it's possible to implement different levels of support based on what a given source can do. This way it's easier to implement gradual adoption of a more capable sources.

Considerations

• Proxying between adapter types and parent types might be an overhead. Maybe there is a value to somehow optimize or shortcut that.

• Querying interfaces is slow with Loki

• It might be worth defining a taxonomy of Gatsby supported interface types

• Should sources define adapters for all those types or should adapter be a separate plugin "class"?

API and Status

• @nodeIntefaces / interface implements Node - @nodeInterface is supported.
• Renderable - a type representing something that returns content as a React component, not as html string. To generalize support for things like Mdx. Issue #18976
• @autoCreate Issue #18977
• @proxy - proxy is in, but there are issue with mapping from fields that itself are resolvable. This might need an additional directive like @projection
• page data processors: Page data processors #20092