Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1067: Refactored mozart.tf so that deploy_pges.py and deploy_pcm_and… #1068

Merged
merged 1 commit into from
Jan 30, 2025

Conversation

philipjyoon
Copy link
Contributor

@philipjyoon philipjyoon commented Jan 28, 2025

…_pges run in parallel. This saves 20 mins of deploymnt time on dev cluster: 1 hour and 15 mins to 54 mins

Purpose and Proposed Changes

  • In mozart.tf pull out remote_execs that installs bach_ui and runs deploy_pges.py into its own null_resource: bach_and_deploy_pges to speed up deployment

Explanation

  • The end result is that these two will run in parallel to other mozart terraform resources running, in particular deploy_pcm_and_pges This and deploy_pges.py are the two longest running resource when deploying Mozart and by allowing them to run in parallel, 20 mins is saves in deployment time: 1 hour and 15 mins to 54 mins on average.

Issues

Testing

  • Destroyed and deployed in my personal dev cluster twice. Both were successful and both took 54 mins to deploy (as opposed to 1:15 previously)
  • Ran all major query, download, SCIFLO, and CNM-S jobs and they all succeeded.
  • You can see lines like this now:
    module.common.null_resource.install_pcm_and_pges: Still creating... [16m30s elapsed]
    module.common.null_resource.bach_and_deploy_pges: Still creating... [16m30s elapsed]
  • And this is the final line output for deploying pges confirming that it was running in parallel and that it took 17 minutes
    module.common.null_resource.bach_and_deploy_pges: Creation complete after 17m14s [id=2647260127362677226]
    module.common.null_resource.install_pcm_and_pges: Still creating... [17m20s elapsed]

…_pges run in parallel. This saves 20 mins of deploymnt time on dev cluster: 1 hour and 15 mins to 54 mins
@hhlee445 hhlee445 merged commit 8d65e1a into develop Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants