RFC 81: StreamField-based rich text

[gasman]

This RFC proposes a new implementation of rich text that leverages the StreamField data model for managing content as a sequence of blocks, while preserving the familiar Word-like user interface as closely as possible.

[lb-]

A few additional questions, I'll try to do a full review later.

• Confirming - this RFC's goal is to replace Draftail - or provide a second richtext interface
• How will this impact or work with the RFC Outline RFC 46: Single paragraph rich text fields/blocks #46 & RFC 60: Draftail Usage for General Text Entry #60
• Does the Wagtail core team / Torchbox really want to take on the responsibility of a full JS text editor build. It could be quite a complex endeavour - with even Facebook (Meta) giving up and starting from scratch recently with Draft.js to Lexical
• Should we consider building on a library like ProseMirror or even TipTap.js that provide the be building blocks for a JS rich text editor without the UI baggage. https://tiptap.dev/ (their data model is quite sound and has support for block level / inline level concepts and plugins).
• Will this RFC aim to resolve many of the outstanding bugs with Draftail?
• What about projects that don't want to buy in / go all in too much with StreamField?

[gasman]

• Confirming - this RFC's goal is to replace Draftail - or provide a second richtext interface
• What about projects that don't want to buy in / go all in too much with StreamField?

(These two are best tackled together I think...)

I definitely want to position this as the single path forward for rich text, rather than an "opt-in" alternative to Draftail. The goals of this RFC are twofold:

• to preserve the MS-Word-like UX of rich text closely enough that editors do not feel a need to stick with Draftail out of familiarity;
• to make the programming model transparent enough that developers can use it without having to explicitly understand or buy in to StreamField as a concept.

To expand on the latter point: RichTextField (or some direct replacement for it) will still exist, and happens to be implemented as a StreamField with a certain pre-baked configuration, but that's just an implementation detail. The editor will have a look and feel that's as close as possible to "classic" rich text, and writing {{ page.body }} on a template will render the expected HTML markup. If they then want to go beyond the stock rich text functionality - such as inserting CTA snippets into the flow of text, say - then that's the point where they'll learn about StreamField blocks and swap out the RichTextField for a formal StreamField definition.

I'm hopeful that we can systematically address any remaining technical and UX hurdles that would give any reason to stick with Draftail. Inevitably there'll still be a certain amount of organisational inertia, and so I think it's important to the success of the exercise that we pitch it as "this is how rich text will work from now on" rather than "here's an alternative to rich text that you might want to try".

I don't have a firm idea of how the transition will work, and so there could still be some period of Draftail existing alongside the new implementation - perhaps something like the use_json_field flag where there's a certain window of time to perform the migration but developers have to do it sooner or later.

(I'm also not totally ruling out the possibility that Draftail could still be the basis of the new implementation, if it proves to be capable of the kind of custom integrations we need.)

• How will this impact or work with the RFC Outline RFC 46: Single paragraph rich text fields/blocks #46 & RFC 60: Draftail Usage for General Text Entry #60

That's a good question! Clearly the ideas in RFC 46 are a key building block for this bit of development, but I don't know whether that's an argument in favour of building a Draftail-based implementation of single-paragraph rich text as a stepping stone towards this, or for starting right away with a new rich text implementation. There's probably a certain amount of overlap with RFC 60 too, although the new data model and details of interactions between adjacent blocks don't apply there.

• Does the Wagtail core team / Torchbox really want to take on the responsibility of a full JS text editor build. It could be quite a complex endeavour - with even Facebook (Meta) giving up and starting from scratch recently with Draft.js to Lexical
• Should we consider building on a library like ProseMirror or even TipTap.js that provide the be building blocks for a JS rich text editor without the UI baggage. https://tiptap.dev/ (their data model is quite sound and has support for block level / inline level concepts and plugins).

We should absolutely consider those, yes. Very keen to get your input on this - I haven't done any proper evaluation of the existing rich text solutions that are out there, and I realise there's a danger of falling victim to Not Invented Here syndrome and taking a "how hard can it be, really" stance that then comes back to bite us :-) Right now my inclination towards writing our own editor is based on these thoughts:

• Given that we're tightly scoping the rich text widget itself to just dealing with the inline entities within a block-level element, and all of the 'fancier' behaviours like headings and toolbars are handled outside of that, in StreamField territory - I feel that there's probably not a huge amount of added value that off-the-shelf libraries can provide beyond what browsers give us as standard through the contentEditable API.
• Whichever route we take, the hardest parts are probably going to be UI behaviour at paragraph boundaries, undo/redo, and handling pasted content - all of which demand the deepest integration with the StreamField mechanism and the most extensive custom code. It's arguably going to be easier to approach that from a clean slate, rather than grapple with someone else's API with baked-in assumptions about how those should be handled.

[lb-]

Thanks for the epic and thoughtful feedback here - nothing too much to add beyond us trying to leverage some of the building blocks in the JS land for text editing. However, I agree that the underlying library here can be an implementation detail.

I also recommend you look at https://grapesjs.com/ - not to use it but to get some inspiration of another approach to the 'block' building concept and how the data can be stored/worked with. Note: This very much heads into the WYSIWYG direction which I think should be avoided, but it's a good approach for it's use case nonetheless.

[chosak]

Suggested change

Retrieving the value of a RichTextField or RichTextBlock in user code will now yield a StreamValue. For simple template rendering cases, this should work with no code changes - a tag such as `{{ page.body }}` will return the StreamValue's `__str__` representation, which will be the HTML rendering as desired. The `|richtext` filter will become redundant, and we would encourage authors of new code to use `{% include_block page.body %}` instead. The native value type of ParagraphBlock will also be a custom Python type with a `__str__` method returning an HTML rendering.

Retrieving the value of a RichTextField or RichTextBlock in user code will now yield a StreamValue. For simple template rendering cases, this should work with no code changes - a tag such as `{{ page.body }}` will return the StreamValue's `__str__` representation, which will be the HTML rendering as desired. The `|richtext` filter will become redundant, and we would encourage authors of new code to use `{% include_block page.body %}` instead. Template rendering of rich text using `{% include_block %}` will pass context variables from the calling template. The native value type of ParagraphBlock will also be a custom Python type with a `__str__` method returning an HTML rendering.

Supporting {% include_block %} for rich text opens up the promising possibility of passing template context to rich text rendering, which is not possible with the current | richtext filter. This would be very useful to, for example:

• Render page links as relative, not absolute (Rich text page links always rendered as absolute URLs wagtail#5876 (comment))
• Add hreflang attributes to page links, but only if the target page's language differs from the current page

It sounds like this proposal would work similarly to how StreamField block rendering does: writing {{ my_block }} does not pass context variables, but {% include_block my_block %} does. Do you think it'd be worth explicitly calling that out here?

[nmorduch]

This is my dream! I am very much in favor.