The path to backlinks

[boehs]

Enter your suggestions in details:

Hi, I'm futzing with Lume for my site, which already uses Vento and Eleventy. Reading the docs I had some philosophical reservations, but I suspect many of them will be alleviated once I actually get to using Lume as its a bit of a paradigm shift in many regards (and, to be clear, it already looks like there's a lot of decisions I think are huge improvements)

The one thing I can't conceivably envision a workaround for is the backlinks feature, which displays on pages the other pages that link to that page (holy word salad). In Eleventy, I do this with a disgusting computedData hack. As far as I can tell, Lume doesn't have computed data which was a big shocker, but quite honestly maybe is an improvement.

At its most basic, the backlink feature requires the following things

1. The ability to export data generated during markdown template rendering

If I wanted I could implement my own loader to do this. This would be a mild hack but tolerable. It is possible that, as objects are reference types, I could modify the value of data?

  async function render(
    content: string,
    data?: Record<string, unknown>,
    filename?: string,
  )

is this officially supported? can it be? What would happen is my wikilinks plugin would, for each wikilink, do data.linksTo.push('/path/to/page')

2. Use this data in a layout

The layout would do something like search.pages("linksTo.includes('{{thisPageUrl}}')") (yes this syntax is invalid but you get the idea).

A different page could specify renderOrder so that all data is available. This is not possible here because backlinks are displayed in the page itself, so the layout would need to set renderOrder. According to order of operations, "For each group of pages with the same renderOrder .... Render the page using the assigned template engine and layout," so this is not possible.

A really easy path to this that occurs to me would be to internally model layouts as pages. How this would work is when lume encounters the layout specifier, the page queue is amended as if the layout was a page itself, and is just passed {{ content, ...data }} for what the value of data would be. That way, layouts would by default be rendered fairly close to directly after the content page, but if renderOrder was set higher this would be deferred for free.

---

If you see any other options for how backlinks could be done today, or other paths to lume supporting this, please let me know! I'd be happy to discuss.

[oscarotero]

Hi! It's a really interesting topic and, honesty, I never had thought about it before.

My first idea was, as you have suggested, implement this as a markdown plugin. Lume has some specific markdown plugins that you can install like toc that extract the table of contents and save it in a property in the page data. This allows to render it in the layout using the html you want.

But that doesn't work in this case because backlinks need to render all pages before rendering the layouts, and renderOrder can break this (as you noticed). The proposal of rendering the layouts after all pages content are rendered looks good. Interestingly enough, I'm working on a new major version (Lume v3) that I was planing to release the next week. I can test this solution to make sure it doesn't break anything.

Another option is to use preprocessors (the Lume approach for computedData). Preprocessors are functions that you can run before the pages are rendered. You can use them to change the page data. For example:

site.preprocess([".md"], (pages) => {
    for (const page of pages) {
      const links = searchLinksInMarkdown(page.data.url, page.data.content);

      for (const link of links) {
        const linkedPage = pages.find((page) => page.data.url === link);
        linkedPage.data.backlinks ??= [];
        linkedPage.data.backlinks.push(page);
      }
    }
});

I didn't test it but you get an idea. Now all pages have a backlinks property which is an array of other pages linked to it.
The drawback of this solution is the markdown must be rendered twice, first to search the links in the preprocessor, and then to render the page. Markdown rendering is a fast operation so maybe it's not a problem.

And another option is to use processors (like preprocessors but when all pages are already rendered). Processors allow to use the DOM API so it's easy to search for links:

site.process([".html"], (pages) => {
  for (const page of pages) {
    const links = page.document.querySelectorAll(".content a");
    for (const link of links) {
      const resolved = resolve(page.url, link.getAttribute("href"));
      const linkedPage = pages.find((page) => page.data.url === resolved);
      linkedPage.data.backlinks ??= [];
      linkedPage.data.backlinks.push(page);
    }
  }

  // Now, render the backlinks of evey page
  for (const page of pages) {
    const { backlinks } = page.data;
    if (backlinks) {
      const container = page.document.querySelector(".backlinks");
      container.innerHTML = renderBacklinks(backlinks);
    }
  }
});

The drawback of this option is you cannot render the backlinks in the layouts. The advantage is that it's more universal because it doesn't depend on a specific template engine like markdown. It's compatible with mdx, jsx, etc.

[oscarotero]

Update: I just made a change to render the layouts once all pages of all renderOrder are rendered and looks like it works fine! (I could even detect a bug in a plugin).

This allows to use a markdown-it plugin to collect all backlinks pages, no matter the renderOrder.

I want to test this with real projects to make sure it doesn't break anything, but it seems a good decision. If you want to test it, just upgrade Lume to the latest development version with deno task lume upgrade --dev.

[RickCogley]

Happy to test this. How can I do so? @oscarotero

[oscarotero]

Thanks @RickCogley!

Just upgrade Lume to the latest version deno task lume upgrade --dev and make sure it doesn't break any of your sites using Lume 3.

[RickCogley]

Ok I can try tomorrow. Do I add the above processor?

[oscarotero]

@RickCogley No. They are just different options to solve the same problem (calculate backlinks).
The change I made aims to make it possible to implement it as a markdown plugin (a plugin that doesn't exist yet), but I just want to make sure this change doesn't break anything.

[boehs]

Woah, you’re an actual 10x developer. I did consider the processor route but I dismissed it due to the reasons you outlined, and especially given how close the codebase already was to a really elegant one. Super happy you agree!

I have a large site with thousands of lines worth of configuration so I can’t test immediately but I hope I’ll be able to start and finish a first pass tonight. Will keep you posted on how it goes!

[RickCogley]

@oscarotero hi, I upgraded on my blog site that uses tailwind 4 (only about 25 pages), and after installing your plugin order deno lint plugin (on the new deno 2.3.1) and clearing all the order errors, I don't see any other errors related to this one. FYI.

[oscarotero]

Thank you @RickCogley !

[RickCogley]

I am not sure, but I noticied something off with field aliases and mentioned it in the "help" section on the discord. It may or may not be related to this, or maybe to the order things are being loaded...