Integrates GitLab Runner into DTaaS

[aryanpingle]

Integrates GitLab Runner into DTaaS

Addresses #892 by integrating a GitLab Runner setup into the DTaaS software.

Type of Change

✔ New feature
✔ Documentation update

Description

Adds a compose.runner.yml file that spins up a Docker container running the gitlab/gitlab-runner image, and automatically starts a gitlab runner based on the settings in config.toml.

Testing

Tested using a localhost and a server installation, both using a server-hosted gitlab instance.

Impact

The process of registering a GitLab Runner will be greatly simplified. Developers will be able to use the digital twins preview page with a self-hosted runner.

Additional Information

• On localhost, the hello-world digital twin fails during execution as the runner executor cannot access the host machine's localhost.
• Currently, network_mode is set to host for the gitlab-runner container. This should instead use or create a docker network.
• Added a guide in the services/gitlab directory.
• Need to update/create the admin guide for using the Digital Twins Preview Page.

Checklist

• My code adheres to the coding and style guidelines of the project.
• I have added tests for all the new code and any changes made to
  existing code.
• I have made corresponding changes to the documentation.

[prasadtalasila]

@aryanpingle Thanks for the PR. Please see the comments.

[prasadtalasila]

Does this solve the problem of self-generated certificates?

[aryanpingle]

The gitlab-runner container is built on the gitlab/gitlab-runner image using alpine and contains the gitlab-runner executable. Starting the container executes gitlab-runner register and gitlab-runner start automatically, so we only need to deal with the runner-config.toml file to set things up.

The gitlab/gitlab-runner image uses a custom image of its own, from the "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper" repository. This uses the docker engine to spawn a sibling container to gitlab-runner, which actually executes the CI/CD job.

Now, if you try using docker ps you won't see the sibling executor container/s, for a reason I don't fully understand. Here's a link to the SO answer: https://stackoverflow.com/a/41994792/8089674.

[prasadtalasila]

@aryanpingle A related correction in the docs

line-69 of deploy/services/gitlab/INTEGRATION.md file should be:

If the DTaaS application is hosted at https://foo.com/, then configure the following files:

[prasadtalasila]

The gitlab-runner container is built on the gitlab/gitlab-runner image using alpine and contains the gitlab-runner executable. Starting the container executes gitlab-runner register and gitlab-runner start automatically, so we only need to deal with the runner-config.toml file to set things up.

The gitlab/gitlab-runner image uses a custom image of its own, from the "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper" repository. This uses the docker engine to spawn a sibling container to gitlab-runner, which actually executes the CI/CD job.

Now, if you try using docker ps you won't see the sibling executor container/s, for a reason I don't fully understand. Here's a link to the SO answer: https://stackoverflow.com/a/41994792/8089674.

please add this information in a separate section at the end of GITLAB-RUNNER.md

[aryanpingle]

The latest commits add content from Chapter 3, Section 3 and Chapter 4 from "Design and implementation of an integrated DevOps framework for Digital Twins as a Service software platform" (Scherma V.) as a new section "DevOps Framework" in the developer documentation. I've reordered a few sections and edited some parts to make it more developer friendly.

[prasadtalasila]

@aryanpingle thanks for the updates. please see the comments.

[prasadtalasila]

@aryanpingle please add the following citation at the end of pages taken from Vanessa's thesis:

Ref: Vanessa Scherma, Design and implementation of an integrated DevOps framework for Digital Twins as a Service software platform, Master’s Degree Thesis, Politecnico Di Torino, 2024.

[prasadtalasila]

Thanks @aryanpingle. The changes have been merged via PR #1115.