How should we configure Log4j across services?

[leopaul36]

This discussion is about how we should configure Log4j across services.

Right now, all Log4j are configured by a templated log4j.properties.j2 with a few properties manageable as tdp_vars (log location mainly, sometimes log level).

If a TDP cluster admin wants to change these configurations, he has no other choices than modifiying the templates in tdp-collection which is not great for configuration versionning.

We have 3 options for improvement:

1. Make log4j.properties a dict in tdp_vars 😄

Pros

• Configuration is totally customizable.

Cons

• Additional work if the default Log4j file of a component is changed through a component major upgrade.
• Lots of additionnal properties keys to manage (can already be "hidden" in TDP UI).
• We lose having the "default" log4j file of the component

2. Make log4j.properties a big multiline string in tdp_vars (same approach as Apache Ambari) 👀

Pros

• Configuration is totally customizable.
• Just one additionnal property in tdp_vars

Cons

• More prone to errors

3. Version the entire log4j.properties.j2 template as a tdp_vars 🚀

Pros

• More transparent to the users in case of a major change in the log4j properties file
• Users can add their own configuration keys to the template

Cons

• Extra work needed in tdp-lib since it's not supported yet.

Let's vote on this by reacting with 1. 😄 2. 👀 3. 🚀

Please comment if I forgot somethings from our previous discussions.

[rpignolet]

We have the same problem with all templates. For example if someone wants to override a systemd service template, he can't. If we choose the first or second option we will add all the systemd service template in a big string in tdp_vars ? Same for all templates ? Writing Jinja templates in a multiline string inside YAML is ugly, we will not have the IDE help with colors, auto-complete, etc.
IMO, a generic system to override all templates is the better option.