[FLINK-37099][docs] Document memory/performance considerations and config possibilities for Netty4

[ferenc-csaky]

What is the purpose of the change

Add some guidance in the Flink docs about Netty4 memory and performance configs.

Verifying this change

Doc change

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[tomncooper]

LGTM. I wonder if the netty 4 reflection setting should be enabled by default?

[ferenc-csaky]

LGTM. I wonder if the netty 4 reflection setting should be enabled by default?

IMO it would still be safe top append it to env.java.opts.all.

[afedulov]

Thanks @ferenc-csaky , LGTM.

[flinkbot]

CI report:

• 3bea726 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[asardaes]

I asked about this in the mailing list but I think it got lost. Isn't io.netty.maxDirectMemory also important? Although that seems to only be documented in the source code.

[ferenc-csaky]

I asked about this in the mailing list but I think it got lost. Isn't io.netty.maxDirectMemory also important? Although that seems to only be documented in the source code.

I'm not sure if it would be a good idea to bring Netty-specific memory mgmt to the Flink docs. Personally I'd rather keep this doc part as light as it can be, to make sure we do not add any noise and create confusion where users might shoot themselves in the foot. Mixing that with JVM memory handling props from Flink configs could result in undesired setups pretty easily IMO.

[afedulov]

@asardaes I agree with @ferenc-csaky . People who are running in such memory-constrained scenarios are welcome to investigate the details of Netty configuration. The documentation section is there primarily to indicate what is the reason for the change in behavior inside of Flink. The path for a "normal" Flink user would be to simply allocate slightly more overall memory to TMs/JM.

[asardaes]

In my experience with Netty outside of Flink, running with the default (io.netty.maxDirectMemory = -1) means that the value of -XX:MaxDirectMemorySize will not be respected by Netty, so I think taskmanager.memory.framework.off-heap.size will no longer account for everything.