Templated/binary logging

[Nady93]

Is is possible you will implement templated/binary logging?
General idea is to have log template, which include message format with its metadata like log level, filename, line number etc, only once in file.
Each log entry, only refer to metadata by ID and adds dynamic parameters.
Everything is saved to binary file and is formated on file read.

Pros of this feature:
Log files can be smaller, due to removed redundancy & binary save
Increased log throughput, because log formating can be done offline (on file read),
Floating point numbers are logged more precisely,
Can decide about log format on file read instead of log save

There is already library that implements it, but it miss other features that Quill offers:
https://github.com/morganstanley/binlog

[odygrd]

When I started developing the library, one of the main motivations was to steer away from using nanolog due to the constant hassle of decoding binary logs.

Quill functions much like a binary log, but with a twist—it operates within the application's memory rather than relying on a binary log file.

Essentially, all metadata and related data reside as static variables in memory during compile time. A pointer to this metadata, doubling as an ID, gets handed off to the backend logging thread. Together with dynamic parameters, this information is formatted and inscribed into a human-readable log file. This approach carries a significant advantage: logging any user-defined element merely requires providing an 'fmt formatter' or an overload for 'operator <<', rendering the library more versatile. In contrast, in binary logging scenarios, explicitly furnishing a serialize method becomes necessary.

While binary logging fits many use cases, it does pose certain inconveniences.

If I were to reimagine the library, I'd narrow support to built-in language types. I've observed that accommodating user-defined types adds complexity. My vision involves employing a shared memory producer-consumer queue per thread for dynamic parameters and another shared memory file per application for the metadata, created at application startup.
This setup would empower the backend worker thread to function separately, detached from the main application. Despite the design will still be leaning towards human-readable logs, it could easily be adapted for binary recording, although it's crucial to note that Quill's backend worker thread operates on pointers, rendering binary log reading outside the application impossible—it's strictly read and decoded within the application.

Presently, I'm stretched for time to embark on a new project, but a new logger complete rewrite might be possible with C++23/26 targets in the future.