Add LuaCATS as alternative tag handling scheme

[alerque]

Unfortunately documentation tagging systems have proliferated and fragmented. The current pretty de facto Lua Language Server (Sumneko) uses the LuaCATS annotation system which are derived from EmmyLua annotations which are derived from LDoc which is derived from LuaDoc. Of course LDoc eventually dropped cross-compatibility with LuaDoc, and EmmyLua eventually dropped cross-compatibility with LDoc, and LuaCATS dropped cross-compatibility with EmmyLua.

So 🤦

Having the LSP type hints working is rather handy in some editors, but it doesn't play nice with LDoc style comments. Unfortunately the docs generation features of Lua_LS or EmmyLua are pretty rudimentary compared to how LDoc can put things together. I'd rather be able to keep using ldoc instead of lua-language-server --doc while also getting LSP features.

I'm wondering it it wouldn't be possible to support all or most of the LuaCATS tag syntax without too much trouble. Already with custom tags I can get part of the way there, but it's obnoxious to do everything with custom parsers and try to disable the default features for overlapping tags like @param.

For example:

--- LDoc
-- @tparam string var A string type parameter.

--- LuaCATS
-- @param var string A string type parameter.

The issue is LuaCATS doesn't know and can't easily be taught how to handle @tparam and LDoc can't easily override @param to handle typing using a different option order than the default.

[chdoc]

For the DFHack project, we are currently working on transitioning from largely untyped and hand-documented Lua libraries to library definitions for lua-language-server. In this context, we are also evaluating LDoc for generating documentation.

We have more than a dozen modules exported from C++ (for which we would be generating definition files (i.e., using ---@meta) and 30+ modules written in Lua. In addition to the ubiquitous use of @param and @return, we are also making extensive use of @class and @field. Moreover, we will use the full language of type specifications (e.g., tables, unions, functions, etc.)

So, I guess it goes without saying, that this is a major blocker for us. How realistic is it that LuaCATS support can be implemented in the not too distant future (i.e., weeks or moths, not years). I suppose we would be willing to help, at the very least as it comes to testing and providing feedback.

CC: @vallode

[Elv13]

For AwesomeWM. My plan is (... eventually...) to just create a second LDoc LTP template to "transpile" the ldoc tags to whatever those LSP servers eat. This doesn't require any change to ldoc and is easy to "just do". The main issue is that project who define their own tags (we have ~25) will need to also fork that LTP template. But that's already the case for the current HTML LTP template too, so 🤷 . It's also less important than it was a couple years ago since LLMs like our human readable doc much more than they like synthetic non standard javadoc derived tag formats.

@alerque, all that's needed to kickstart this effort if you want it in ldoc itself is to copy https://github.com/lunarmodules/ldoc/blob/master/ldoc/html/ldoc_ltp.lua into ldoc_cats_ltp.lua and make it output fake --- blocks instead of HTML. It's really not that much work, a couple hour at most. It's just nowhere near my priority right now (and also fixing the MIT license text so it's detected properly by GitHub would avoid me having to request an exception from my employer if I want to ever contribute).

[vallode]

@Elv13 those insights are great, if I understand what you suggested correctly the main issue with that approach (for us) is that we make pretty heavy use of features that LDoc lacks but LuaCATS supports. For us simply transpiling from LDoc to a LuaCATS-compatible library would remove a lot of benefits. Hmm, unless it is as simple as defining some custom tags for LuaCATS and shuffling that forward.

No solutions from me yet, I've had a cursory look through the logic of LDoc and agree it could be finagled into covering a lot more of LuaCATS but have no idea how long it would take and how that would work within the structure of the project.

[CppCXY]

I can offer another approach. EmmyLuaAnalyzer might make this easier to implement. It is a compatible analysis library and language server with luals syntax, with easier-to-use APIs and higher performance.
I wrote a CLI project for it and implemented the most basic document generation (based on docfx) inside it. Perhaps we can improve this tool to meet these requirements.