Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(migrate-template-gen): add readme generator #13948

Merged
merged 8 commits into from
Oct 2, 2024

Conversation

abhi7cr
Copy link
Contributor

@abhi7cr abhi7cr commented Oct 1, 2024

Description of changes

Add README generator for using generated CFN stack templates to aid customers for stack refactoring Gen1 resources into Gen2 stack.

There will be one README per category and will be stored along with templates under .amplify/migration/templates/<category>. The reason for not have a single README is it will become too long a file for customers to sift through. Verified this structure with @josefaidt.

TODO: Add rollback step for Refactor (step 3)

Note: The actual generation of templates and the command to invoke it will be in different PR(s) to keep all PRs small.

Sample README output for storage category

Stack refactor steps for storage category

STEP 1: UPDATE GEN-1 STORAGE STACK

It is a non-disruptive update since the template only replaces resource references with their resolved values. This is a required step to execute cloudformation stack refactor later.

aws cloudformation update-stack \
 --stack-name amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27 \
 --template-body file://.amplify/migration/templates/storage/step1-gen1PreProcessUpdateStackTemplate.json \
 --parameters '[{"ParameterKey":"bucketName","ParameterValue":"testauth6a8797560abd4a15a06bd063a98b7979"},{"ParameterKey":"s3PublicPolicy","ParameterValue":"Public_policy_3e30a1be"},{"ParameterKey":"AuthenticatedAllowList","ParameterValue":"ALLOW"},{"ParameterKey":"unauthRoleName","ParameterValue":"amplify-testauth-dev-36113-unauthRole"},{"ParameterKey":"s3PrivatePolicy","ParameterValue":"Private_policy_3e30a1be"},{"ParameterKey":"selectedGuestPermissions","ParameterValue":""},{"ParameterKey":"s3PermissionsAuthenticatedPublic","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"s3PermissionsAuthenticatedUploads","ParameterValue":"s3:PutObject"},{"ParameterKey":"s3UploadsPolicy","ParameterValue":"Uploads_policy_3e30a1be"},{"ParameterKey":"functionS3Trigger45371987Arn","ParameterValue":"arn:aws:lambda:us-east-1:517770102601:function:S3Trigger45371987-dev"},{"ParameterKey":"functionS3Trigger45371987LambdaExecutionRole","ParameterValue":"S3Trigger45371987LambdaRole45371987-dev"},{"ParameterKey":"s3PermissionsAuthenticatedProtected","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"s3PermissionsGuestUploads","ParameterValue":"DISALLOW"},{"ParameterKey":"s3ReadPolicy","ParameterValue":"read_policy_3e30a1be"},{"ParameterKey":"selectedAuthenticatedPermissions","ParameterValue":"s3:PutObject,s3:GetObject,s3:ListBucket,s3:DeleteObject"},{"ParameterKey":"s3PermissionsGuestPublic","ParameterValue":"DISALLOW"},{"ParameterKey":"s3PermissionsAuthenticatedPrivate","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"env","ParameterValue":"dev"},{"ParameterKey":"unauthPolicyName","ParameterValue":"s3_amplify_3e30a1be"},{"ParameterKey":"authRoleName","ParameterValue":"amplify-testauth-dev-36113-authRole"},{"ParameterKey":"GuestAllowList","ParameterValue":"DISALLOW"},{"ParameterKey":"triggerFunction","ParameterValue":"S3Trigger45371987"},{"ParameterKey":"authPolicyName","ParameterValue":"s3_amplify_3e30a1be"},{"ParameterKey":"s3ProtectedPolicy","ParameterValue":"Protected_policy_3e30a1be"},{"ParameterKey":"functionS3Trigger45371987Name","ParameterValue":"S3Trigger45371987-dev"}]' \
 --capabilities CAPABILITY_NAMED_IAM
aws cloudformation describe-stacks \
--stack-name amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27

Rollback step:

aws cloudformation update-stack \
--stack-name amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27 \
--template-body file://.amplify/migration/templates/storage/step1-gen1PreProcessUpdateStackTemplate-rollback.json
--parameters '[{"ParameterKey":"bucketName","ParameterValue":"testauth6a8797560abd4a15a06bd063a98b7979"},{"ParameterKey":"s3PublicPolicy","ParameterValue":"Public_policy_3e30a1be"},{"ParameterKey":"AuthenticatedAllowList","ParameterValue":"ALLOW"},{"ParameterKey":"unauthRoleName","ParameterValue":"amplify-testauth-dev-36113-unauthRole"},{"ParameterKey":"s3PrivatePolicy","ParameterValue":"Private_policy_3e30a1be"},{"ParameterKey":"selectedGuestPermissions","ParameterValue":""},{"ParameterKey":"s3PermissionsAuthenticatedPublic","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"s3PermissionsAuthenticatedUploads","ParameterValue":"s3:PutObject"},{"ParameterKey":"s3UploadsPolicy","ParameterValue":"Uploads_policy_3e30a1be"},{"ParameterKey":"functionS3Trigger45371987Arn","ParameterValue":"arn:aws:lambda:us-east-1:517770102601:function:S3Trigger45371987-dev"},{"ParameterKey":"functionS3Trigger45371987LambdaExecutionRole","ParameterValue":"S3Trigger45371987LambdaRole45371987-dev"},{"ParameterKey":"s3PermissionsAuthenticatedProtected","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"s3PermissionsGuestUploads","ParameterValue":"DISALLOW"},{"ParameterKey":"s3ReadPolicy","ParameterValue":"read_policy_3e30a1be"},{"ParameterKey":"selectedAuthenticatedPermissions","ParameterValue":"s3:PutObject,s3:GetObject,s3:ListBucket,s3:DeleteObject"},{"ParameterKey":"s3PermissionsGuestPublic","ParameterValue":"DISALLOW"},{"ParameterKey":"s3PermissionsAuthenticatedPrivate","ParameterValue":"s3:PutObject,s3:GetObject,s3:DeleteObject"},{"ParameterKey":"env","ParameterValue":"dev"},{"ParameterKey":"unauthPolicyName","ParameterValue":"s3_amplify_3e30a1be"},{"ParameterKey":"authRoleName","ParameterValue":"amplify-testauth-dev-36113-authRole"},{"ParameterKey":"GuestAllowList","ParameterValue":"DISALLOW"},{"ParameterKey":"triggerFunction","ParameterValue":"S3Trigger45371987"},{"ParameterKey":"authPolicyName","ParameterValue":"s3_amplify_3e30a1be"},{"ParameterKey":"s3ProtectedPolicy","ParameterValue":"Protected_policy_3e30a1be"},{"ParameterKey":"functionS3Trigger45371987Name","ParameterValue":"S3Trigger45371987-dev"}]' \
--capabilities CAPABILITY_NAMED_IAM
aws cloudformation describe-stacks \
--stack-name amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27

STEP 2: REMOVE GEN-2 STORAGE STACK RESOURCES

This step is required since we will eventually replace gen-2 resources with gen-1 resources as part of Step 3 (refactor) .

aws cloudformation update-stack \
 --stack-name amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V \
 --template-body file://.amplify/migration/templates/storage/step2-gen2ResourcesRemovalStackTemplate.json \
  --parameters '[{"ParameterKey":"referencetoamplifymygen2apprjabhisandboxa7ef9235a4authNestedStackauthNestedStackResource80F6C892Outputsamplifymygen2apprjabhisandboxa7ef9235a4authamplifyAuthauthenticatedUserRole25B64256Ref","ParameterValue":"amplify-mygen2app-rjabhi--amplifyAuthauthenticatedU-knLb1dx0g6Lk"},{"ParameterKey":"referencetoamplifymygen2apprjabhisandboxa7ef9235a4functionNestedStackfunctionNestedStackResourceD7D9CE79Outputsamplifymygen2apprjabhisandboxa7ef9235a4functionS3Trigger45371987devlambda35AE298DArn","ParameterValue":"arn:aws:lambda:us-east-1:517770102601:function:amplify-mygen2app-rjabhi--S3Trigger45371987devlamb-M4XpHyN7bJwb"}]' \
 --capabilities CAPABILITY_NAMED_IAM
aws cloudformation describe-stacks \
--stack-name amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V

Rollback step:

aws cloudformation update-stack \ 
--stack-name amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V \
--template-body file://.amplify/migration/templates/storage/step2-gen2ResourcesRemovalStackTemplate-rollback.json \
--parameters '[{"ParameterKey":"referencetoamplifymygen2apprjabhisandboxa7ef9235a4authNestedStackauthNestedStackResource80F6C892Outputsamplifymygen2apprjabhisandboxa7ef9235a4authamplifyAuthauthenticatedUserRole25B64256Ref","ParameterValue":"amplify-mygen2app-rjabhi--amplifyAuthauthenticatedU-knLb1dx0g6Lk"},{"ParameterKey":"referencetoamplifymygen2apprjabhisandboxa7ef9235a4functionNestedStackfunctionNestedStackResourceD7D9CE79Outputsamplifymygen2apprjabhisandboxa7ef9235a4functionS3Trigger45371987devlambda35AE298DArn","ParameterValue":"arn:aws:lambda:us-east-1:517770102601:function:amplify-mygen2app-rjabhi--S3Trigger45371987devlamb-M4XpHyN7bJwb"}]' \
--capabilities CAPABILITY_NAMED_IAM
aws cloudformation describe-stacks \
--stack-name amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V

STEP 3: CREATE AND EXECUTE CLOUDFORMATION STACK REFACTOR FOR storage CATEGORY

This step will move the Gen1 storage resources to Gen2 stack.

3.a) Upload the source and destination templates to S3

export BUCKET_NAME=<<YOUR_BUCKET_NAME>>
aws s3 cp .amplify/migration/templates/storage/step3-sourceTemplate.json s3://$BUCKET_NAME
aws s3 cp .amplify/migration/templates/storage/step3-destinationTemplate.json s3://$BUCKET_NAME

3.b) Create stack refactor

aws cloudformation create-stack-refactor  --stack-definitions StackName=amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27,TemplateURL=s3://$BUCKET_NAME/step3-sourceTemplate.json  StackName=amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V,TemplateURL=s3://$BUCKET_NAME/step3-destinationTemplate.json  --resource-mappings  '[{"Source":{"StackName":"amplify-testauth-dev-36113-storages3abhi-3QDW0WGSZW27","LogicalResourceId":"S3Bucket"},"Destination":{"StackName":"amplify-mygen2app-rjabhi-sandbox-a7ef9235a4-storage0EC3F24A-1NEC6Q2KHRH1V","LogicalResourceId":"testauth6a8797560abd4a15a06bd063a98b797936113devBucketA4CDFF97"}}]'
export STACK_REFACTOR_ID=<<REFACTOR-ID-FROM-CREATE-STACK-REFACTOR_CALL>>

3.c) Describe stack refactor to check for creation status

aws cloudformation describe-stack-refactor --stack-refactor-id $STACK_REFACTOR_ID

3.d) Execute stack refactor

aws cloudformation execute-stack-refactor --stack-refactor-id $STACK_REFACTOR_ID

3.e) Describe stack refactor to check for execution status

aws cloudformation describe-stack-refactor --stack-refactor-id $STACK_REFACTOR_ID

Description of how you validated changes

test locally following the README. There is an existing bug with refactor API related to Tags and Properties that is preventing it from successfully executing.

Checklist

  • PR description included
  • yarn test passes
  • Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)
  • New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
  • Pull request labels are added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@abhi7cr abhi7cr requested a review from a team as a code owner October 1, 2024 17:30
@abhi7cr abhi7cr merged commit fe6f6c2 into migrations Oct 2, 2024
5 checks passed
@abhi7cr abhi7cr deleted the migration-template-gen branch October 2, 2024 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants