Breaks site-to-site test, when upgrade app version from 1.16.3 to 1.19.0

[dtrdnk]

Describe the bug
Current app version 1.16.3 works fine, but if you upgrade to 1.19.0, then the automatic rotation of certificates breaks.
This breaks also breaks the test-site-to-site

Version of Helm, Kubernetes and the Nifi chart:
Helm : v3.10.2
Kubernetes: v1.25.3
NiFi chart: 1.1.3

What happened:
NiFi server is still working with the old certificate, even if new cert exist in the store

What you expected to happen:
NiFi server must use a new certificate on the fly without restart

How to reproduce it (as minimally and precisely as possible):
Just upgrade image tag from 1.16.3 to 1.19.0 or 1.18.0

Anything else we need to know:

Here are some information that help troubleshooting:
NiFi has autoreload function, but I don't know how to invoke it. By default autoreload set to false. Even if I set manualy autoreload to true, this is doesn't help.

• if relevant, provide your values.yaml or the changes made to the default one (after removing sensitive information)
• the output of the folowing commands:

Check if a pod is in error:
There is no a pod error

[jdesroch]

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

[dtrdnk]

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

Maybe you are right. What we can to do with this problem?

[wknickless]

This is most likely due to NiFi 1.19.0 docker base image changed to eclipse-temurin:jre-11 from openjdk:8-jre. This changed the cacerts location from /usr/local/openjdk-8/lib/security/cacerts to /opt/java/openjdk/lib/security/cacerts.

Yes, but with cert-manager enabled the chart overrides that default by setting the nifi.security.truststore configuration property. And it also sets the nifi.security.autoreload.enabled and nifi.security.autoreload.interval properties per the documentation (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security_configuration).

We are also interested in upgrading to NiFi 1.18 or 1.19 for other reasons, so I'll get to work replicating the problem.

[wknickless]

Confirmed that the certificate rotation broke between 1.16.3 and 1.17.0. Suspicious of this commit: apache/nifi@4b655ec

[banzo]

same for 1.20, see #294

[banzo]

@wknickless @dtrdnk #294 fixes the issue and bumps NiFi to 1.20.

I'm happy to merge the PR but I would be more at ease with more eyes on the changes.

[nathluu]

Hi @banzo, in my PR #294, I just reatart nifi pod to make it reload new keystore, it seems the autoreload feature is broken and has not been fixed yet

[dtrdnk]

@wknickless @dtrdnk #294 fixes the issue and bumps NiFi to 1.20.

I'm happy to merge the PR but I would be more at ease with more eyes on the changes.

Good news! I wiil wait merge of PR 294. And then I will merge all commits from master into my PR

[dtrdnk]

Hi @banzo, in my PR #294, I just reatart nifi pod to make it reload new keystore, it seems the autoreload feature is broken and has not been fixed yet

You are right. Autoreload function is broken in new versions of Docker image, and I don't know how to fix this problem.

[banzo]

@dtrdnk is one of those issues describing the problem or do we need to create a new one?

https://issues.apache.org/jira/browse/NIFI-10425?jql=project%20%3D%20NIFI%20AND%20text%20~%20autoreload

[dtrdnk]

@dtrdnk is one of those issues describing the problem or do we need to create a new one?

https://issues.apache.org/jira/browse/NIFI-10425?jql=project%20%3D%20NIFI%20AND%20text%20~%20autoreload

Thank you! This issue NIFI-10425 looks like our problem. I make research by the Jetty problem, and get some facts:

• Option reportRealPaths available in Jetty 10 and 11
• Currently NiFi use jetty 9, which reached EOF at May 2022
• There is another task NIFI-11518 to upgrade Jetty in NiFi

I think we should just wait for the NIFI-11518 task to be released.

[nathluu]

Hi @dtrdnk,
I tested locally on windows (no symlink) machine and the autoreload feature has also broken since v1.17.0.
I created NIFI-11536 for tracking this issue. The feature works for 1.16.3 so it clearly is not the issue with jetty