/etc/resolv.conf search domains

[BonzTM]

What is the most appropriate way to permanently modify the /etc/resolv.conf on a node in an OKD 4.13 cluster since systemd-resolved is worked-around?

The search domain on each node is propagated to my pods and resulting in hundreds of thousands of invalid forwarded lookups.

I've currently applied a machineConfig that overwrites /run/systemd/resolve/resolv.conf with a custom file, but I want to make sure I follow appropriate procedures and don't circumvent any systems in-place

[vrutkovs]

Set nameserver / search domains in NetworkManager - it would update /var/run/NetworkManager/resolv.conf and subsequently /run/systemd/resolve/resolv.conf

[goern]

is that to be done via Kubernetes NMState Operator's NodeNetworkConfigurationPolicy ?

[LennertMertens]

How did you guys solve this? @goern do you have a solution?

[BonzTM]

is that to be done via Kubernetes NMState Operator's NodeNetworkConfigurationPolicy ?

I attempted to apply a NodeNetworkConfigurationPolicy config to the nmstate.io/v1 api first, only to realize that those CRDs don't exist in OKD (even though OpenShift 4.13 documentation tells you to do it this way).

Ultimately I made the modifications via nmcli to the interface directly on each node.
Something like: nmcli con mod ens18 ipv4.dns-search "foo.bar.com"

Edit: It looks like you can install it after the fact: https://docs.okd.io/latest/networking/k8s_nmstate/k8s-nmstate-about-the-k8s-nmstate-operator.html

[goern]

Edit: It looks like you can install it after the fact: https://docs.okd.io/latest/networking/k8s_nmstate/k8s-nmstate-about-the-k8s-nmstate-operator.html

you can, that's what I did. But I think I did not fully understand the NNCP. Do you still have the .yaml you created around?

[BonzTM]

Edit: It looks like you can install it after the fact: https://docs.okd.io/latest/networking/k8s_nmstate/k8s-nmstate-about-the-k8s-nmstate-operator.html

you can, that's what I did. But I think I did not fully understand the NNCP. Do you still have the .yaml you created around?

Here is the OpenShift documentation:
https://docs.openshift.com/container-platform/4.13/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.html

And the very simple yaml I was going to apply before I ran nmcli

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: 99-okd-worker-resolvconf
spec:
  nodeSelector: 
    node-role.kubernetes.io/worker: "" 
  desiredState:
    dns-resolver: 
      config:
        server:
        - 10.0.10.1
        - 10.0.10.2
        search:
        - domain.lan

[LennertMertens]

I'm running into some issues configuring the search domains. I tried applying the followin NNCP but when doing that, the cluster fails to update, service routes and API becomse unavailable and starts throwing 503 service errors.

NNCP:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: okd-nodes-resolvconf

spec:
  maxUnavailable: 1
  desiredState:
    dns-resolver: 
      config:
        server:
          - 10.24.12.17
          - 10.242.5.17
        search:
          - cluster.local
          - k8s.cluster.be # basedomain
          - rmp.be
          - xyz.be
          - servers.xyz.be
          - servers.dmf.be

Anybody that was able to solve this issue? Thanks

[goern]

any updates on this?!

[LennertMertens]

Hi @goern, seems like you were unable to set the search domains using the operator either? What did you try yet?

[goern]

actually, I wanted to completely unset it, b4mad/op1st-emea-b4mad@b8d19c0#diff-c8d79ed8662a41394a65288ed72d9ad31302677dcb4df514a1de5c5f293033c6 is the latest code. it 'felt' like the cluster is not always reconfigured correctly (after reboots??) so that /etc/resolv.conf is overwritten by NetworkManager. let's see...

[LennertMertens]

I was unable to revert my changes using this either. My cluster seems "broken" after trying to update the search domains.

[LorbusChris]

There were some changes made recently to this in OKD/FCOS 4.15 - it might need to be backported? openshift/okd-machine-os#693

[LennertMertens]

@LorbusChris so these are required in order for the NMstate operator along with my config to work?