Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️👷 Refactor CICD Pipeline #1311

Merged
merged 97 commits into from
Feb 7, 2025
Merged

♻️👷 Refactor CICD Pipeline #1311

merged 97 commits into from
Feb 7, 2025

Conversation

rblaine95
Copy link
Collaborator

@rblaine95 rblaine95 commented Feb 4, 2025

  • Merge style-check.yml into CICD
  • Major refactor of CICD pipeline
    • Use composite actions to make CICD pipeline more modular and maintainable
  • Currently the OSS deployment is 1 replica of everything
    • Except Ledger Nodes as that requires 4

Deploy and Test EKS is now a single job consisting of Composite actions.
This makes maintaining the CICD pipeline easier as it modularizes the workflow a bit.
This also means that there global concurrency group isn't on the entire pipeline, but instead just on the Deploy and Test EKS.
This means that, other than Deploy and Test EKS, multiple workflows can run simultaneously.

Previously, for running the local tests, we used a matrix to create 5 Github hosted runners to run 5 copies of the local dev stack, and then run subsets of the tests. We would then combine the test coverage reports.
Now, due to optimizations made with parallelizing tests and to the local dev infrastructure, we can run everything on a single Github hosted runner, greatly simplifying the local testing and coverage reporting.

Closes DIDC-105 and DIDC-305


FYI, @ff137 and @cl0ete, due to the removal of uv you will need to reinitialize the .venv (rm -rf .venv && mise run poetry:install)

@rblaine95 rblaine95 force-pushed the refactor/cicd-pipeline branch from 3af644e to 2138ece Compare February 4, 2025 12:55
@rblaine95 rblaine95 force-pushed the refactor/cicd-pipeline branch from 2138ece to 1f12d57 Compare February 4, 2025 12:57
@rblaine95 rblaine95 force-pushed the refactor/cicd-pipeline branch from ebec554 to 247bdd6 Compare February 4, 2025 15:04
Copy link

github-actions bot commented Feb 6, 2025

Coverage

K8s Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631281%111–112, 125–141, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1663579%118–160, 201–205, 360–364, 403–408, 441–445, 473–478, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py18856%17–33
   credentials.py14286%13, 18
   retry_method.py361558%20–35, 76–77, 91–103
   save_exchange_record.py13192%29
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14286%34–35
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496054789% 

Tests Skipped Failures Errors Time
895 6 💤 0 ❌ 0 🔥 4m 27s ⏱️

Copy link

github-actions bot commented Feb 6, 2025

Coverage

K8s Regression Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631970%44–50, 80, 111–112, 125–141, 147–150, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1664672%118–160, 201–205, 360–364, 403–408, 441–445, 463–478, 484–493, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py181233%14–34
   check_endorser_connection.py10460%14–24
   credentials.py14286%13, 18
   retry_method.py363017%15–36, 68–114
   save_exchange_record.py13192%29
   string.py15473%15–16, 20, 24
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14750%21–39
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496059788% 

Tests Skipped Failures Errors Time
895 46 💤 0 ❌ 0 🔥 4m 0s ⏱️

Copy link

github-actions bot commented Feb 7, 2025

Coverage

K8s Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631281%111–112, 125–141, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1663579%118–160, 201–205, 360–364, 403–408, 441–445, 473–478, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py18856%17–33
   credentials.py14286%13, 18
   retry_method.py361558%20–35, 76–77, 91–103
   save_exchange_record.py13192%29
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14286%34–35
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496054789% 

Tests Skipped Failures Errors Time
895 6 💤 0 ❌ 0 🔥 4m 43s ⏱️

Copy link

github-actions bot commented Feb 7, 2025

Coverage

K8s Regression Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631970%44–50, 80, 111–112, 125–141, 147–150, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1664672%118–160, 201–205, 360–364, 403–408, 441–445, 463–478, 484–493, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py181233%14–34
   check_endorser_connection.py10460%14–24
   credentials.py14286%13, 18
   retry_method.py363017%15–36, 68–114
   save_exchange_record.py13192%29
   string.py15473%15–16, 20, 24
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14750%21–39
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496059788% 

Tests Skipped Failures Errors Time
895 46 💤 0 ❌ 0 🔥 4m 3s ⏱️

* To be moved out of this repo Soon™
Copy link

sonarqubecloud bot commented Feb 7, 2025

@rblaine95 rblaine95 enabled auto-merge (squash) February 7, 2025 08:43
Copy link

github-actions bot commented Feb 7, 2025

Coverage

K8s Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631281%111–112, 125–141, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1663579%118–160, 201–205, 360–364, 403–408, 441–445, 473–478, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py18856%17–33
   credentials.py14286%13, 18
   retry_method.py361558%20–35, 76–77, 91–103
   save_exchange_record.py13192%29
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14286%34–35
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496054789% 

Tests Skipped Failures Errors Time
895 6 💤 0 ❌ 0 🔥 4m 30s ⏱️

Copy link

github-actions bot commented Feb 7, 2025

Coverage

K8s Regression Test Coverage
FileStmtsMissCoverMissing
app
   main.py80298%121–122
app/models
   wallet.py36489%100–105
app/routes
   jsonld.py604722%23–93, 104–137
   messaging.py241154%43–52, 85–95
   oob.py462839%58–99, 131–144, 170–181
   trust_registry.py62297%52–53
   verifier.py1393376%88–90, 95, 152–154, 159, 231, 239–241, 246, 288–293, 299–301, 362–364, 369, 406–408, 413, 449–451, 505–507
app/routes/wallet
   credentials.py805334%57–70, 96–108, 129–140, 166–180, 216–232, 266–283, 309–321, 342–353
app/services
   acapy_ledger.py631970%44–50, 80, 111–112, 125–141, 147–150, 200–201
   acapy_wallet.py40490%60–61, 98–99
   revocation_registry.py1664672%118–160, 201–205, 360–364, 403–408, 441–445, 463–478, 484–493, 527–528
app/services/issuer
   acapy_issuer_v2.py971288%63, 97–106, 110, 126–127, 172, 225–226, 252
app/services/onboarding
   tenants.py634922%30–103, 113–143
   verifier.py37392%67, 83–87
app/services/onboarding/util
   register_issuer_did.py95595%100–102, 276–281
   set_endorser_metadata.py724044%32–34, 58–60, 85–87, 118–137, 149–160, 172–185, 198–212
app/services/trust_registry
   actors.py107992%95–100, 110, 209–210, 212–217, 282–285
   schemas.py54983%57–63, 89–99
app/services/trust_registry/util
   actor.py31294%26–27
   issuer.py23483%39–40, 48–49
app/util
   acapy_verifier_utils.py123596%81–84, 164–168, 245
   assert_public_did.py181233%14–34
   check_endorser_connection.py10460%14–24
   credentials.py14286%13, 18
   retry_method.py363017%15–36, 68–114
   save_exchange_record.py13192%29
   string.py15473%15–16, 20, 24
   tenants.py451762%24–29, 44–45, 52–59, 90–91, 97, 124
   transaction_acked.py14750%21–39
shared
   log_config.py532062%41–94, 103–110, 152, 161–173
shared/models
   connection_record.py24196%47
   credential_exchange.py43784%85–88, 101–104
   endorsement.py866920%36–70, 92–122, 132–181
   presentation_exchange.py40978%54–56, 62–64, 82, 87, 91
   protocol.py10190%18
   trustregistry.py63789%22, 44, 75–83
shared/services
   nats_jetstream.py36489%56, 60, 64, 68
shared/util
   api_router.py14193%17
   mock_agent_controller.py55689%23, 74–78
trustregistry
   main.py84495%49–51, 113
waypoint
   main.py50198%67
waypoint/services
   nats_service.py112397%73–75
TOTAL496059788% 

Tests Skipped Failures Errors Time
895 46 💤 0 ❌ 0 🔥 4m 5s ⏱️

@rblaine95 rblaine95 merged commit 4a2d3bc into master Feb 7, 2025
31 checks passed
@rblaine95 rblaine95 deleted the refactor/cicd-pipeline branch February 7, 2025 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants