diff --git a/01-EKS-Create-Cluster-using-eksctl/01-01-Install-CLIs/README.md b/01-EKS-Create-Cluster-using-eksctl/01-01-Install-CLIs/README.md deleted file mode 100644 index 7a13fff9..00000000 --- a/01-EKS-Create-Cluster-using-eksctl/01-01-Install-CLIs/README.md +++ /dev/null @@ -1,126 +0,0 @@ -# Install AWS, kubectl & eksctl CLI's - -## Step-00: Introduction -- Install AWS CLI -- Install kubectl CLI -- Install eksctl CLI - -## Step-01: Install AWS CLI -- Reference-1: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html -- Reference-2: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html -### Step-01-01: MAC - Install and configure AWS CLI -- Download the binary and install via command line using below two commands. -``` -# Download Binary -curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg" - -# Install the binary -sudo installer -pkg ./AWSCLIV2.pkg -target / -``` -- Verify the installation -``` -aws --version -aws-cli/2.0.7 Python/3.7.4 Darwin/19.4.0 botocore/2.0.0dev11 - -which aws -``` -- Reference: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-mac.html - -### Step-01-02: Windows 10 - Install and configure AWS CLI -- The AWS CLI version 2 is supported on Windows XP or later. -- The AWS CLI version 2 supports only 64-bit versions of Windows. -- Download Binary: https://awscli.amazonaws.com/AWSCLIV2.msi -- Install the downloaded binary (standard windows install) -``` -aws --version -aws-cli/2.0.8 Python/3.7.5 Windows/10 botocore/2.0.0dev12 -``` -- Reference: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-windows.html - -### Step-01-03: Configure AWS Command Line using Security Credentials -- Go to AWS Management Console --> Services --> IAM -- Select the IAM User: kalyan -- **Important Note:** Use only IAM user to generate **Security Credentials**. Never ever use Root User. (Highly not recommended) -- Click on **Security credentials** tab -- Click on **Create access key** -- Copy Access ID and Secret access key -- Go to command line and provide the required details -``` -aws configure -AWS Access Key ID [None]: ABCDEFGHIAZBERTUCNGG (Replace your creds when prompted) -AWS Secret Access Key [None]: uMe7fumK1IdDB094q2sGFhM5Bqt3HQRw3IHZzBDTm (Replace your creds when prompted) -Default region name [None]: us-east-1 -Default output format [None]: json -``` -- Test if AWS CLI is working after configuring the above -``` -aws ec2 describe-vpcs -``` - -## Step-02: Install kubectl CLI -- **IMPORTANT NOTE:** Kubectl binaries for EKS please prefer to use from Amazon (**Amazon EKS-vended kubectl binary**) -- This will help us to get the exact Kubectl client version based on our EKS Cluster version. You can use the below documentation link to download the binary. -- Reference: https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html - -### Step-02-01: MAC - Install and configure kubectl -- Kubectl version we are using here is 1.16.8 (It may vary based on Cluster version you are planning use in AWS EKS) - -``` -# Download the Package -mkdir kubectlbinary -cd kubectlbinary -curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.16.8/2020-04-16/bin/darwin/amd64/kubectl - -# Provide execute permissions -chmod +x ./kubectl - -# Set the Path by copying to user Home Directory -mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin -echo 'export PATH=$PATH:$HOME/bin' >> ~/.bash_profile - -# Verify the kubectl version -kubectl version --short --client -Output: Client Version: v1.16.8-eks-e16311 -``` - - -### Step-02-02: Windows 10 - Install and configure kubectl -- Install kubectl on Windows 10 -``` -mkdir kubectlbinary -cd kubectlbinary -curl -o kubectl.exe https://amazon-eks.s3.us-west-2.amazonaws.com/1.16.8/2020-04-16/bin/windows/amd64/kubectl.exe -``` -- Update the system **Path** environment variable -``` -C:\Users\KALYAN\Documents\kubectlbinary -``` -- Verify the kubectl client version -``` -kubectl version --short --client -kubectl version --client -``` - -## Step-03: Install eksctl CLI -### Step-03-01: eksctl on Mac -``` -# Install Homebrew on MacOs -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" - -# Install the Weaveworks Homebrew tap. -brew tap weaveworks/tap - -# Install the Weaveworks Homebrew tap. -brew install weaveworks/tap/eksctl - -# Verify eksctl version -eksctl version -``` - -### Step-03-02: eksctl on windows or linux -- For windows and linux OS, you can refer below documentation link. -- **Reference:** https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html#installing-eksctl - - -## References: -- https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html \ No newline at end of file diff --git a/01-EKS-Create-Cluster-using-eksctl/01-02-Create-EKSCluster-and-NodeGroups/README.md b/01-EKS-Create-Cluster-using-eksctl/01-02-Create-EKSCluster-and-NodeGroups/README.md deleted file mode 100644 index 6487c5cc..00000000 --- a/01-EKS-Create-Cluster-using-eksctl/01-02-Create-EKSCluster-and-NodeGroups/README.md +++ /dev/null @@ -1,130 +0,0 @@ -# Create EKS Cluster & Node Groups - -## Step-00: Introduction -- Understand about EKS Core Objects - - Control Plane - - Worker Nodes & Node Groups - - Fargate Profiles - - VPC -- Create EKS Cluster -- Associate EKS Cluster to IAM OIDC Provider -- Create EKS Node Groups -- Verify Cluster, Node Groups, EC2 Instances, IAM Policies and Node Groups - - -## Step-01: Create EKS Cluster using eksctl -- It will take 15 to 20 minutes to create the Cluster Control Plane -``` -# Create Cluster -eksctl create cluster --name=eksdemo1 \ - --region=us-east-1 \ - --zones=us-east-1a,us-east-1b \ - --without-nodegroup - -# Get List of clusters -eksctl get cluster -``` - - -## Step-02: Create & Associate IAM OIDC Provider for our EKS Cluster -- To enable and use AWS IAM roles for Kubernetes service accounts on our EKS cluster, we must create & associate OIDC identity provider. -- To do so using `eksctl` we can use the below command. -- Use latest eksctl version (as on today the latest version is `0.21.0`) -``` -# Template -eksctl utils associate-iam-oidc-provider \ - --region region-code \ - --cluster \ - --approve - -# Replace with region & cluster name -eksctl utils associate-iam-oidc-provider \ - --region us-east-1 \ - --cluster eksdemo1 \ - --approve -``` - - - -## Step-03: Create EC2 Keypair -- Create a new EC2 Keypair with name as `kube-demo` -- This keypair we will use it when creating the EKS NodeGroup. -- This will help us to login to the EKS Worker Nodes using Terminal. - -## Step-04: Create Node Group with additional Add-Ons in Public Subnets -- These add-ons will create the respective IAM policies for us automatically within our Node Group role. - ``` -# Create Public Node Group -eksctl create nodegroup --cluster=eksdemo1 \ - --region=us-east-1 \ - --name=eksdemo1-ng-public1 \ - --node-type=t3.medium \ - --nodes=2 \ - --nodes-min=2 \ - --nodes-max=4 \ - --node-volume-size=20 \ - --ssh-access \ - --ssh-public-key=kube-demo \ - --managed \ - --asg-access \ - --external-dns-access \ - --full-ecr-access \ - --appmesh-access \ - --alb-ingress-access -``` - -## Step-05: Verify Cluster & Nodes - -### Verify NodeGroup subnets to confirm EC2 Instances are in Public Subnet -- Verify the node group subnet to ensure it created in public subnets - - Go to Services -> EKS -> eksdemo -> eksdemo1-ng1-public - - Click on Associated subnet in **Details** tab - - Click on **Route Table** Tab. - - We should see that internet route via Internet Gateway (0.0.0.0/0 -> igw-xxxxxxxx) - -### Verify Cluster, NodeGroup in EKS Management Console -- Go to Services -> Elastic Kubernetes Service -> eksdemo1 - -### List Worker Nodes -``` -# List EKS clusters -eksctl get cluster - -# List NodeGroups in a cluster -eksctl get nodegroup --cluster= - -# List Nodes in current kubernetes cluster -kubectl get nodes -o wide - -# Our kubectl context should be automatically changed to new cluster -kubectl config view --minify -``` - -### Verify Worker Node IAM Role and list of Policies -- Go to Services -> EC2 -> Worker Nodes -- Click on **IAM Role associated to EC2 Worker Nodes** - -### Verify Security Group Associated to Worker Nodes -- Go to Services -> EC2 -> Worker Nodes -- Click on **Security Group** associated to EC2 Instance which contains `remote` in the name. - -### Verify CloudFormation Stacks -- Verify Control Plane Stack & Events -- Verify NodeGroup Stack & Events - -### Login to Worker Node using Keypai kube-demo -- Login to worker node -``` -# For MAC or Linux or Windows10 -ssh -i kube-demo.pem ec2-user@ - -# For Windows 7 -Use putty -``` - -## Step-06: Update Worker Nodes Security Group to allow all traffic -- We need to allow `All Traffic` on worker node security group - -## Additional References -- https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html -- https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html diff --git a/01-EKS-Create-Cluster-using-eksctl/01-03-EKSCluster-Pricing/README.md b/01-EKS-Create-Cluster-using-eksctl/01-03-EKSCluster-Pricing/README.md deleted file mode 100644 index b5969f2f..00000000 --- a/01-EKS-Create-Cluster-using-eksctl/01-03-EKSCluster-Pricing/README.md +++ /dev/null @@ -1,27 +0,0 @@ -# EKS Cluster Pricing - -## Steo-01: Very Important EKS Pricing Note -- EKS is not free (Unlike other AWS Services) -- In short, no free-tier for EKS. -### EKS Cluster Pricing - - We pay $0.10 per hour for each Amazon EKS cluster - - Per Day: $2.4 - - For 30 days: $72 -### EKS Worker Nodes Pricing - EC2 - - You pay for AWS resources (e.g. EC2 instances or EBS volumes) - - T3 Medium Server in N.Virginia - - $0.0416 per Hour - - Per Day: $0.9984 - Approximately $1 - - Per Month: $30 per 1 t3.medium server - - Reference: https://aws.amazon.com/ec2/pricing/on-demand/ - - In short, if we run 1 EKS Cluster and 1 t3.medium worker node **continuously** for 1 month, our bill is going to be around $102 to $110 - - If we take 5 days to complete this course, and if we run 1 EKS Cluster and 2 t3.medium Worker nodes continuosly for 5 days it will cost us approximately around $25. -### EKS Fargate Profile - - AWS Fargate pricing is calculated based on the **vCPU and memory** resources used from the time you start to download your container image until the EKS Pod terminates. - - **Reference:** https://aws.amazon.com/fargate/pricing/ - - Amazon EKS support for AWS Fargate is available in us-east-1, us-east-2, eu-west-1, and ap-northeast-1. - -### Important Notes -- **Important Note-1:** If you are using your personal AWS Account, then ensure you delete and recreate cluster and worker nodes as and when needed. -- **Important Note-2:** We cant stop our EC2 Instances which are in Kubernetes cluster unlike regular EC2 Instances. So we need to delete the worker nodes (Node Group) if we are not using it during our learning process. - \ No newline at end of file diff --git a/01-EKS-Create-Cluster-using-eksctl/01-04-Delete-EKSCluster-and-NodeGroups/README.md b/01-EKS-Create-Cluster-using-eksctl/01-04-Delete-EKSCluster-and-NodeGroups/README.md deleted file mode 100644 index 95b974bb..00000000 --- a/01-EKS-Create-Cluster-using-eksctl/01-04-Delete-EKSCluster-and-NodeGroups/README.md +++ /dev/null @@ -1,38 +0,0 @@ -# Delete EKS Cluster & Node Groups - -## Step-01: Delete Node Group -- We can delete a nodegroup separately using below `eksctl delete nodegroup` -``` -# List EKS Clusters -eksctl get clusters - -# Capture Node Group name -eksctl get nodegroup --cluster= -eksctl get nodegroup --cluster=eksdemo1 - -# Delete Node Group -eksctl delete nodegroup --cluster= --name= -eksctl delete nodegroup --cluster=eksdemo1 --name=eksdemo1-ng-public1 -``` - -## Step-02: Delete Cluster -- We can delete cluster using `eksctl delete cluster` -``` -# Delete Cluster -eksctl delete cluster -eksctl delete cluster eksdemo1 -``` - -## Important Notes - -### Note-1: Rollback any Security Group Changes -- When we create a EKS cluster using `eksctl` it creates the worker node security group with only port 22 access. -- When we progress through the course, we will be creating many **NodePort Services** to access and test our applications via browser. -- During this process, we need to add an additional rule to this automatically created security group, allowing access to our applications we have deployed. -- So the point we need to understand here is when we are deleting the cluster using `eksctl`, its core components should be in same state which means roll back the change we have done to security group before deleting the cluster. -- In this way, cluster will get deleted without any issues, else we might have issues and we need to refer cloudformation events and manually delete few things. In short, we need to go to many places for deletions. - -### Note-2: Rollback any EC2 Worker Node Instance Role - Policy changes -- When we are doing `EBS Storage Section with EBS CSI Driver` we will add a custom policy to worker node IAM role. -- When you are deleting the cluster, first roll back that change and delete it. -- This way we don't face any issues during cluster deletion. diff --git a/01-EKS-Create-Cluster-using-eksctl/README.md b/01-EKS-Create-Cluster-using-eksctl/README.md deleted file mode 100644 index 7c9ebb48..00000000 --- a/01-EKS-Create-Cluster-using-eksctl/README.md +++ /dev/null @@ -1,15 +0,0 @@ -# EKS - Create Cluster - -## List of Topics -- Install CLIs - - AWS CLI - - kubectl - - eksctl -- Create EKS Cluster -- Create EKS Node Groups -- Understand EKS Cluster Pricing - - EKS Control Plane - - EKS Worker Nodes - - EKS Fargate Profile -- Delete EKS Clusters - diff --git a/11-DevOps-with-AWS-Developer-Tools/Application-Manifests/buildspec.yml b/11-DevOps-with-AWS-Developer-Tools/Application-Manifests/buildspec.yml index 1e5a945d..e4214695 100644 --- a/11-DevOps-with-AWS-Developer-Tools/Application-Manifests/buildspec.yml +++ b/11-DevOps-with-AWS-Developer-Tools/Application-Manifests/buildspec.yml @@ -1,60 +1,48 @@ version: 0.2 +run-as: root + phases: + install: commands: - - echo "Install Phase - Nothing to do using latest Amazon Linux Docker Image for CodeBuild which has all AWS Tools - https://github.com/aws/aws-codebuild-docker-images/blob/master/al2/x86_64/standard/3.0/Dockerfile" + - echo Installing app dependencies... + - curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.18.9/2020-11-02/bin/linux/amd64/kubectl + - chmod +x ./kubectl + - mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin + - echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc + - source ~/.bashrc + - echo 'Check kubectl version' + - kubectl version --short --client + - chmod +x eks_cicd/prereqs.sh + - sh eks_cicd/prereqs.sh + pre_build: - commands: - # Docker Image Tag with Date Time & Code Buiild Resolved Source Version - - TAG="$(date +%Y-%m-%d.%H.%M.%S).$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)" - # Update Image tag in our Kubernetes Deployment Manifest - - echo "Update Image tag in kube-manifest..." - - sed -i 's@CONTAINER_IMAGE@'"$REPOSITORY_URI:$TAG"'@' kube-manifests/01-DEVOPS-Nginx-Deployment.yml - # Verify AWS CLI Version - - echo "Verify AWS CLI Version..." - - aws --version - # Login to ECR Registry for docker to push the image to ECR Repository - - echo "Login in to Amazon ECR..." - - $(aws ecr get-login --no-include-email) - # Update Kube config Home Directory - - export KUBECONFIG=$HOME/.kube/config + commands: + - echo Logging in to Amazon EKS... + - aws eks --region $AWS_DEFAULT_REGION update-kubeconfig --name $AWS_CLUSTER_NAME + - echo check config + - kubectl config view --minify + - echo check kubectl access + - kubectl get svc + - echo Logging in to Amazon ECR... + - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com + - REPOSITORY_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME + - docker pull $REPOSITORY_URI:$IMAGE_TAG + + build: commands: - # Build Docker Image - - echo "Build started on `date`" - - echo "Building the Docker image..." - - docker build --tag $REPOSITORY_URI:$TAG . + - echo Build started on `date` + - echo Building the Docker image... + - docker build --cache-from $REPOSITORY_URI:$IMAGE_TAG -t $IMAGE_REPO_NAME:$IMAGE_TAG . + - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG + post_build: commands: - # Push Docker Image to ECR Repository - - echo "Build completed on `date`" - - echo "Pushing the Docker image to ECR Repository" - - docker push $REPOSITORY_URI:$TAG - - echo "Docker Image Push to ECR Completed - $REPOSITORY_URI:$TAG" - # Extracting AWS Credential Information using STS Assume Role for kubectl - - echo "Setting Environment Variables related to AWS CLI for Kube Config Setup" - - CREDENTIALS=$(aws sts assume-role --role-arn $EKS_KUBECTL_ROLE_ARN --role-session-name codebuild-kubectl --duration-seconds 900) - - export AWS_ACCESS_KEY_ID="$(echo ${CREDENTIALS} | jq -r '.Credentials.AccessKeyId')" - - export AWS_SECRET_ACCESS_KEY="$(echo ${CREDENTIALS} | jq -r '.Credentials.SecretAccessKey')" - - export AWS_SESSION_TOKEN="$(echo ${CREDENTIALS} | jq -r '.Credentials.SessionToken')" - - export AWS_EXPIRATION=$(echo ${CREDENTIALS} | jq -r '.Credentials.Expiration') - # Setup kubectl with our EKS Cluster - - echo "Update Kube Config" - - aws eks update-kubeconfig --name $EKS_CLUSTER_NAME - # Apply changes to our Application using kubectl - - echo "Apply changes to kube manifests" - - kubectl apply -f kube-manifests/ - - echo "Completed applying changes to Kubernetes Objects" - # Create Artifacts which we can use if we want to continue our pipeline for other stages - - printf '[{"name":"01-DEVOPS-Nginx-Deployment.yml","imageUri":"%s"}]' $REPOSITORY_URI:$TAG > build.json - # Additional Commands to view your credentials - #- echo "Credentials Value is.. ${CREDENTIALS}" - #- echo "AWS_ACCESS_KEY_ID... ${AWS_ACCESS_KEY_ID}" - #- echo "AWS_SECRET_ACCESS_KEY... ${AWS_SECRET_ACCESS_KEY}" - #- echo "AWS_SESSION_TOKEN... ${AWS_SESSION_TOKEN}" - #- echo "AWS_EXPIRATION... $AWS_EXPIRATION" - #- echo "EKS_CLUSTER_NAME... $EKS_CLUSTER_NAME" -artifacts: - files: - - build.json - - kube-manifests/* \ No newline at end of file + - echo Build completed on `date` + - echo Pushing the Docker image... + - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG + - echo Push the latest image to cluster + - kubectl apply -f kube-manifests/deployment.yaml + - kubectl apply -f kube-manifests/service.yaml + - kubectl rollout restart -f kube-manifests/deployment.yaml