[Karpenter] Cannot configure node role policy in KarpenterAddOn

[Shellmode]

Describe the feature

Since the policies of node role and service account are hardcode, there is no way to configure the policy of nodes created by Karpenter in cdk-eks-blueprints.

I can only modify the policy in another way which is not so IaC.

Use Case

Define policy of nodes created by Karpenter as needed.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request

CDK version used

2.105.0

EKS Blueprints Version

No response

Node.js Version

v20.9.0

Environment details (OS name and version, etc.)

MacOS 14.1.2 (23B92)

[shapirov103]

@youngjeong46 please take a look when you get a chance and let me know on the LOE.

[youngjeong46]

@Shellmode I'm trying to understand this better.

1. There are two roles in question - Node Role that gets attached to the Karpenter nodes, and the role for the controller that gets attached to the service account, and which the karpenter controller uses to provision instances. Are you trying to replace both with your custom roles? What would be the reasoning?
2. Would your ideal feature here be to 1/ provide additional policy on top of what is being generated, or 2/ provide a new role all together (which you as a user would manage to make sure it has the minimal required policy to operate Karpenter functionally)?

[Shellmode]

@Shellmode I'm trying to understand this better.

1. There are two roles in question - Node Role that gets attached to the Karpenter nodes, and the role for the controller that gets attached to the service account, and which the karpenter controller uses to provision instances. Are you trying to replace both with your custom roles? What would be the reasoning?

2. Would your ideal feature here be to 1/ provide additional policy on top of what is being generated, or 2/ provide a new role all together (which you as a user would manage to make sure it has the minimal required policy to operate Karpenter functionally)?

First question, the role of the nodes which are scaled in or scaled out by karpenter.

It's a common situation that worker nodes(karpenter scale them in or out) need various of permissions to finish tasks.

Second question, I think additional policy will be better, because developers always focus on the permissions and don't want to define another role resource.

[jsamuel1]

+1. Hitting this now, when trying to configure CloudwatchInsightsAddon with Karpenter. No easy way to configure the extra permissions needed.
Ideally, I just want to either pass in the extra policies, or expose the role out from the addon, so that I can add to it after creation.

[yubingjiaocn]

cdk-eks-blueprints/lib/addons/karpenter/index.ts

Lines 700 to 711 in 7869d39

	private setUpNodeRole(cluster: Cluster, stackName: string, region: string): [iam.Role, iam.CfnInstanceProfile] {
	// Set up Node Role
	const karpenterNodeRole = new iam.Role(cluster, 'karpenter-node-role', {
	assumedBy: new iam.ServicePrincipal(`ec2.${cluster.stack.urlSuffix}`),
	managedPolicies: [
	iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEKSWorkerNodePolicy"),
	iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEKS_CNI_Policy"),
	iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEC2ContainerRegistryReadOnly"),
	iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore"),
	],
	//roleName: `KarpenterNodeRole-${name}` // let role name to be generated as unique
	});

Karpenter add-on creates node role with a fixed logical name karpenter-node-role. You can use clusterInfo.cluster.node.findChild('karpenter-node-role') as iam.IRole to refer to node IAM role created by Karpenter. After fetched the role you can add policy by attachInlinePolicy("your-policy") or add managed policy.

[jsamuel1]

Karpenter add-on creates node role with a fixed logical name karpenter-node-role. You can use clusterInfo.cluster.node.findChild('karpenter-node-role') as iam.IRole to refer to node IAM role created by Karpenter. After fetched the role you can add policy by attachInlinePolicy("your-policy") or add managed policy.

If this logical name is fixed, can we document the recommended way in the KarpenterAddon docs?

[Shellmode]

@yubingjiaocn thanks for providing the walk around, I'll try that to make the system more IaC.

[github-actions]

This issue has been automatically marked as stale because it has been open 60 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

Issue closed due to inactivity.