Option to preserve comments in the AST #1967
Comments
|
I'm happy to maintain a patched lexer/parser in an external package if you don't feel comfortable implementing this option in core. I'd greatly appreciate some guidance on how to change the lexer/parser to preserve comments 🙂 I suppose I must somehow stop ignoring comments in the lexer, and also create a new |
|
OK I managed to get something working 🙂 Basically:
ignored_tokens = frozenset(
[
# TOKEN_COMMENT_BEGIN,
# TOKEN_COMMENT,
# TOKEN_COMMENT_END,
TOKEN_WHITESPACE,
TOKEN_LINECOMMENT_BEGIN,
TOKEN_LINECOMMENT_END,
TOKEN_LINECOMMENT,
]
)class Comment(Stmt):
"""A template comment."""
fields = ("data",)
data: str# in Parser.subparse
elif token.type == "comment_begin":
flush_data()
next(self.stream)
body.append(nodes.Comment(next(self.stream).value))
self.stream.expect("comment_end")Printing the AST gives something like: |
|
Bump 🙂 Would maintainers be interested in such functionality? I think it would allow nice developments in the Jinja ecosystem 🙂 |
|
Yeah, I'm open for reviewing a PR for this, but I just have very limited time to focus on new features for Jinja right now. |
|
Fantastic |
Hello, thanks for this wonderful piece of software 🙂
I'm maintaining mkdocstrings, which uses Jinja templates to render API documentation for different languages. Recently I had this idea of doing the same thing (collecting and rendering API docs) for... Jinja templates themselves 😛 It's just an idea for now, but I'll be tracking my progress here: mkdocstrings/mkdocstrings#661.
I see that Jinja provides the
Environment.parsemethod, which builds an AST of the Jinja source: this is fantastic. I'll be able to visit such trees to build useful information and store it in structures that make it easy to render them in other Jinja templates.However... Jinja comments are not preserved 😢 This #719 (comment) says that they're probably discarded by the lexer.
I can already extract lots of useful information from templates, but I had in mind to allow template writers to use Jinja comments to document variables, filters, blocks, macros, whatever, directly in the template, just like we document attributes, functions, classes, etc., directly within Python code.
For this I would need an option to tell
Environment.parseto preserve the comments in the generated AST. Do you think that is feasible? I can definitely free up some of my time to work on this.In any case, what do you think of this idea of auto-documentation for Jinja templates? I think that would be a wonderful tool in the Jinja ecosystem 😄 I personally have lots of Jinja templates that I would document this way 🙂
The text was updated successfully, but these errors were encountered: