[Karpenter-AddOn] Remove AmazonEKS_CNI_Policy if VPC-CNI AddOn enabled with serviceAccountPolicies defined

[vumdao]

Describe the feature

In order for the Amazon VPC CNI to manage IP addresses, it requires AWS Identity and Access Management (AWS IAM) permissions. Amazon EKS recommends creating a separate AWS IAM role with the permissions defined in AmazonEKS_CNI_Policy managed policy then associate that role with the VPC CNI using IRSA (i.e., AWS IAM roles for service account).

When specified, the node role for the cluster provider can be configured without the CNI policy (Ref: https://github.com/aws-quickstart/cdk-eks-blueprints/blob/main/lib/addons/vpc-cni/index.ts#L268). This help the node role more least privilege

Use Case

My addons

    new VpcCniAddOn({
      enableNetworkPolicy: true,
      serviceAccountPolicies: [ ManagedPolicy.fromAwsManagedPolicyName('AmazonEKS_CNI_Policy')]
    }),
    new KarpenterAddOn()

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request

CDK version used

2.101.1

EKS Blueprints Version

1.12.0

Node.js Version

v20.8.1

Environment details (OS name and version, etc.)

macOS Sonoma 14.0

[shapirov103]

@youngjeong46 please take a look. Since we are hardcoding the policies for the node role in the Karpenter addon, let's 1/ create a mechanism to override the behavior with custom policies and 2/ consider adding a flag to exclude CNI policy.

We should not make the Karpenter addon dependent on the VPC CNI but we can make it configurable to satisfy the customer use case.

[github-actions]

This issue has been automatically marked as stale because it has been open 60 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[youngjeong46]

@shapirov103 @vumdao A couple comments here:

1. There's another issue asking for custom policy to be configurable on the karpenter node role ([Karpenter] Cannot configure node role policy in KarpenterAddOn #893). On the next PR I can add that feature (add required policies to the node role, then be able to add any additional policies as required).
2. I can add a flag to not add the EKS CNI policy to the node role if there is a VPC CNI addon. However, there is no assurance that the user has configured the EKS CNI policy to the VPC CNI addon. Here it states the requirement but is not actually enforced programmatically. If a user does not provide one to the VPC CNI addon, there could be problems. Thoughts?

[shapirov103]

Seems reasonable to go with approach 1/ and see what customer experience if the use case is to just exclude a specific policy.

For pipeline build policies for example, I used DEFAULT BUILD policies here https://github.com/aws-quickstart/cdk-eks-blueprints/blob/main/lib/pipelines/code-pipeline.ts#L200 There can be several sets of policies defined that customers can pick and choose.

Option 2 could be just a boolean flag, e.g. addCniPolicy (true by default). that makes it explicit. This option would make sense if that turns out to be the most common use case for the customer. Let's hear the customer voice.

[github-actions]

This issue has been automatically marked as stale because it has been open 60 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

Issue closed due to inactivity.