Good suggestion! Are you willing to contribute this?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

This issue has been automatically closed due to inactivity.

Hi, folks! C:

I'm willing to contribute to this one, but I'm not sure the implementation details are completely clear to me.

Do I understand it correctly that gitlab api shall be polled at this endpoint https://docs.gitlab.com/ee/api/runners.html#list-owned-runners for the count of the runners equivalent to https://github.com/kedacore/keda/blob/main/pkg/scalers/github_runner_scaler.go?

Thanks in advance!

hey!
Thanks! 🚀
I think that the correct endpoint is the pipeline one -> https://docs.gitlab.com/ee/api/pipelines.html
But yeah, the idea is that KEDA will poll the queue length calling the API using a token

Hi, @JorTurFer!
I've created a poc of this solution, could you please take a glance https://github.com/kedacore/keda/pull/6087/files?
It's not ready (in terms of tests, helm charts, docs etc), but it does have some logic in there.

If it looks fine, I'll continue with the rest of the required parts.

Thanks in advance!

I've taken a look and it looks nice. It's exactly how a scaler works :)

Hi! I've fixed the previous issues mentioned, and added some tests. Please take a look at your convenience <3

I had to add a new parser for *url.URL, but it wasn't very clear to me as to why there is this argument "params" in the function

func setConfigValueHelper(params Params, valFromConfig string, field reflect.Value) error {

in pkg/scalers/scalersconfig/typed_config.go

Seemingly it's passed through the recursion, but actually is not really used. Maybe I'm mistaken, I'm new to this part of the codebase.

Hi! I've fixed the previous issues mentioned, and added some tests. Please take a look at your convenience <3

I had to add a new parser for *url.URL, but it wasn't very clear to me as to why there is this argument "params" in the function

func setConfigValueHelper(params Params, valFromConfig string, field reflect.Value) error {

in pkg/scalers/scalersconfig/typed_config.go

Seemingly it's passed through the recursion, but actually is not really used. Maybe I'm mistaken, I'm new to this part of the codebase.

@wozniakjan FYI

hi @fira42073, great job with #6087!

Regarding params in setConfigValueHelper(), you don't have to pass it to your setConfigValueURL(), the value setter function can have an arbitrary signature, it just happened to be the case that most of these need params so they take it :)

@fira42073 , what do you need for the e2e tests? I'm a total newbie with Gitlab so IDK if we have to create a Gitlab account or if it runs locally and you can deploy an instance directly into the cluster for the e2e test or how we can include e2e tests

I think we can totally run it in the cluster. I'm using https://github.com/sameersbn/docker-gitlab?tab=readme-ov-file#quick-start for local testing, maybe it's a good option to use it for e2e tests as well.

The only part I'm not sure about is creating the actual pipelines and checking how they scale. I'm not sure if shared runners are available in self-hosted gitlab. Probably they are not, so we'd need to connect gitlab's runner system to cluster and let it provision new pods.

After that it should be easy to programmatically create a repo, push a file with .gitlab-ci.yml that will contain an instruction like sleep 300, so the runner is occupied with some pseudowork.

I'm not sure what would be the best option to increase the number of runners tbh. Probably just continuously committing some garbage data, so that it can trigger more pipelines every time would work.

p.s.
sorry for late replies, I have my fulltime job during the weekdays, so I usually can contribute only on the weekends

Hey! I've taken a look at the existing e2e tests and I'm a bit puzzled.

As far I understood, I'll need to add some setup code to tests/utils/setup_test.go and tests/utils/cleanup_test.go.
Here I should be able to initialize gitlab itself. So it probably makes more sense to use a helmchart to do that.
I've discovered this one, and it looks quite good.

As far as I understand, I can use ExecuteCommand to do helm install for this gitlab chart. I'm a bit confused about how the decision of running the test or not is being concluded.

I there there are some values taken from the env:

	AzureRunWorkloadIdentityTests = os.Getenv("AZURE_RUN_WORKLOAD_IDENTITY_TESTS")
	AwsIdentityTests              = os.Getenv("AWS_RUN_IDENTITY_TESTS")
	GcpIdentityTests              = os.Getenv("GCP_RUN_IDENTITY_TESTS")

From which I can conclude that these values determine if the test will be run or not. But I couldn't figure out where they are actually being set.

The only other occurrence is in Makefile:deploy. But as far as I understand that's unrelated, because there are some other actions happening that are related somehow to kustomize.

I've figured that actually creating the instance using the chart is pretty easy:

Deploy

# # Instructions per https://docs.gitlab.com/charts/development/kind/#nginx-ingress-nodeport-with-ssl
helm repo add gitlab https://charts.gitlab.io/
helm repo update

# with ssl
kind create cluster --config examples/kind/kind-ssl.yaml
helm upgrade --install gitlab gitlab/gitlab \
  --set global.hosts.domain=local.gd \
  --set global.edition=ce \
  -f examples/kind/values-base.yaml \
  -f examples/kind/values-ssl.yaml

firefox http://gitlab.local.gd/

Get secret

username is root
password is:

kubectl get secret <name>-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

# in case of previous commands
kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

I figured out that the runners are already being installed as part of the chart and there is no need to install them separately, so the next instructions are strikedthrogh, because they are not relevant

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml

curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.28.0/install.sh | bash -s v0.28.0

kubectl create -f https://operatorhub.io/install/gitlab-runner-operator.yaml

# generate PAT
echo $RANDOM | shasum | head -c 30

# replace token
kubectl exec -it -c toolbox deploy/gitlab-toolbox -- gitlab-rails runner "token = User.find_by_username('root').personal_access_tokens.create(scopes: ['create_runner'], name: 'create_runner_pat', expires_at: 1.days.from_now); token.set_token('REPLACE_ME_WITH_PREVIOUS_STEP_VALUE'); token.save!"

# so it would result in something like 
kubectl exec -it -c toolbox deploy/gitlab-toolbox -- gitlab-rails runner "token = User.find_by_username('root').personal_access_tokens.create(scopes: ['create_runner'], name: 'create_runner_pat', expires_at: 1.days.from_now); token.set_token('b2c6552d23ed47548549ceceed07e2'); token.save!"

---
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-runner-secret
type: Opaque
# Only one of the following fields can be set. The Operator fails to register the runner if both are provided.
# NOTE: runner-registration-token is deprecated and will be removed in GitLab 18.0. You should use runner-token instead.
stringData:
  runner-token: b2c6552d23ed47548549ceceed07e2  # your project runner token from previous step
  # runner-registration-token: ""  # your project runner secret
---
apiVersion: apps.gitlab.com/v1beta2
kind: Runner
metadata:
  name: gitlab-runner
spec:
  gitlabUrl: https://gitlab.local.gd
  buildImage: alpine
  token: gitlab-runner-secret

curl -k --request POST --header "PRIVATE-TOKEN: b2c6552d23ed47548549ceceed07e2" --data "runner_type=instance_type"  "https://gitlab.local.gd/api/v4/user/runners"

As per adding the runners I haven't still figured out how to do that properly, because I'm facing some issues.

This fails to connect runners to the gitlab instance, but that's probably related to dns, because gitlab-runner pod crashes with ERROR: Registering runner... failed runner=JD6wIp1U status=couldn't execute POST against https://gitlab.local.gd/api/v4/runners: Post "https://gitlab.local.gd/api/v4/runners": dial tcp 127.0.0.1:443: connect: connection refused

I've stumbled upon this project, but it looks dead. Also it's probably better to use real runners instead of mocking them. After all, it's e2e C:

But I can imagine after filling that gap, I can already create a ScaledObject that will reference runners deployment.

Then I somehow need to trigger several pipelines, that won't stop immediately, but will hang in there for some time, while tests are being run.

Mock pipeline:

.gitlab-ci.yml

# This file is a template, and might need editing before it works on your project.
# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/index.html#stages
#
# You can copy and paste this template into a new `.gitlab-ci.yml` file.
# You should not add this template to an existing `.gitlab-ci.yml` file by using the `include:` keyword.
#
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Getting-Started.gitlab-ci.yml
stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy
build-job:       # This job runs in the build stage, which runs first.
  stage: build
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."
unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - sleep 60
    - echo "Code coverage is 90%"
lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  script:
    - echo "Linting code... This will take about 10 seconds."
    - sleep 10
    - echo "No lint issues found."
deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  environment: production
  script:
    - echo "Deploying application..."
    - echo "Application successfully deployed."

I'll keep you posted on the progress.

Okay, my first clue about the dns was right, the pod was trying to reach its own localhost, so that was failing. Fixing the issue was as easy as substituting local.gd with nip.io and my own local ip address. It's not clear to me how this is going to work in CI for the e2e test.

The rest of steps look roughly like this:
Substitute https://gitlab.192.168.1.165.nip.io with your local IP address if running this all locally. Again, not sure how this is going to look in the CI for e2e.

Deploy

# # Instructions per https://docs.gitlab.com/charts/development/kind/#nginx-ingress-nodeport-with-ssl
helm repo add gitlab https://charts.gitlab.io/
helm repo update

# with ssl
kind create cluster --config examples/kind/kind-ssl.yaml
helm upgrade --install gitlab gitlab/gitlab \
  --set global.hosts.domain=192.168.1.165.nip.io \
  --set global.edition=ce \
  -f examples/kind/values-base.yaml \
  -f examples/kind/values-ssl.yaml

firefox https://gitlab.192.168.1.165.nip.io/

Here we also need a mechanism to check that gitlab is up and the api is not returning 502 anymore. Regular polling every 10 seconds or so might work well.

(Optional) Get secret to access ui on https://gitlab.192.168.1.165.nip.io/

username is root
password is:

kubectl get secret <name>-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

# in case of previous commands
kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

Create repo and add ci file

# create personal access token
kubectl exec -it -c toolbox deploy/gitlab-toolbox -- gitlab-rails runner "token = User.find_by_username('root').personal_access_tokens.create(scopes: ['read_repository', 'write_repository', 'api', 'read_api'], name: 'create_runner_pat', expires_at: 1.days.from_now); token.set_token('static_pat_123'); token.save!"

# new repo
curl -k --request POST --header "PRIVATE-TOKEN: static_pat_123" \
     --header "Content-Type: application/json" --data '{
        "name": "new_project", "description": "New Project", "path": "new_project", "initialize_with_readme": "false"}' \
     --url "https://gitlab.192.168.1.165.nip.io/api/v4/projects/"

# add .gitlab-ci.yml file
curl -k --request POST \
--header 'PRIVATE-TOKEN: static_pat_123' \
--header "Content-Type: application/json" \
--data "{\"branch\": \"main\", \"author_email\": \"[email protected]\", \"author_name\": \"Firstname Lastname\", \"content\": \"stages: [deploy]\\ndeploy-job:\\n  stage: deploy\\n  script: [\\\"sleep 600\\\"]\", \"commit_message\": \"create a new file\"}" \
--url "https://gitlab.192.168.1.165.nip.io/api/v4/projects/1/repository/files/.gitlab-ci.yml"

I guess currently there are a couple of questions to answer:

• how to actually perform all of these setup actions for the cases when gitlab-runner is tested in the e2e tests.
• how to determine which local ip to use in the instance, since localhost would not work
• figure out if this setup is going to work on the e2e test runners that are currently being used, because this was tested using KinD. there may be additional challenges with the real ci runner.

These are the challenging points I see for now. Please let me know if you have any ideas about overcoming these.

Thanks in advance! <3

Hello
Sorry for the slow response :(

• how to actually perform all of these setup actions for the cases when gitlab-runner is tested in the e2e tests.

I think that gitlab is something scoped within the test context, so I'd spin up the gitlab env during in the beginning of the test and just delete it at the end rather than doing that with global setup/cleanup.

• how to determine which local ip to use in the instance, since localhost would not work

Isn't there a kubernetes service that you can use?

• figure out if this setup is going to work on the e2e test runners that are currently being used, because this was tested using KinD. there may be additional challenges with the real ci runner.

In theory based on docs, we should be able to spin up a real GitLab runner and connect it to the GitLab instance -> https://docs.gitlab.com/runner/. In any case, if the changes are uploaded, I can trigger the e2e test in our testing cluster and check how it goes

Hey!
No rush C:

I think that gitlab is something scoped within the test context, so I'd spin up the gitlab env during in the beginning of the test and just delete it at the end rather than doing that with global setup/cleanup.

If some other scaler already does such setup, could you please refer me to where I can see that?

Isn't there a kubernetes service that you can use?

That's the thing C:
It creates its own signed certificate, and I need to use some kind of fixed host. I'm using nip.io, which is a dns server that returns localhost for localhost.nip.io or 192.168.0.1 for 192.168.0.1.nip.io. It creates a fqdn that can be signed using self-issued certificate.

In theory based on docs, we should be able to spin up a real GitLab runner and connect it to the GitLab instance -> https://docs.gitlab.com/runner/. In any case, if the changes are uploaded, I can trigger the e2e test in our testing cluster and check how it goes

Yeah, checking it on the end cluster where it will be running will definitely be helpful!

If some other scaler already does such setup, could you please refer me to where I can see that?

Sure! This example creates the selenium hub from scratch using raw manifests and this other example deploys IBMMQ server using helm

It creates its own signed certificate, and I need to use some kind of fixed host. I'm using nip.io, which is a dns server that returns localhost for localhost.nip.io or 192.168.0.1 for 192.168.0.1.nip.io. It creates a fqdn that can be signed using self-issued certificate

Yeah, but you can use the service as fixed host if the backend exposes TLS. I mean, you can access to the workload via service calling to https://service_name.namespace. If you set the host as service_name.namespace, the caller will accept the served cert for service_name.namespace

Thanks for the examples, I'll look into how I can replicate a similar setup for gitlab as well. Looks easy.

Yeah, but you can use the service as fixed host if the backend exposes TLS. I mean, you can access to the workload via service calling to https://service_name.namespace. If you set the host as service_name.namespace, the caller will accept the served cert for service_name.namespace

This is probably the best solution in this context, for some reason I didn't even think of it. Using k8s service fqdn totally works.

I'll do my best to carve out some time for it this weekend. Thanks!

I'll do my best to carve out some time for it this weekend. Thanks!

No rush at all. We really appreciate contributions, but OSS is OSS. Take your time and don't be presured about any expected date :)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

Hey! I had quite a busy couple of months, but I'm planning to get back at this issue in the upcoming weeks. Could you unmark it as stale, please? Thanks in advance!

Hey! I've gotten back to this issue and I cannot quite understand what we are scaling. Afaik gitlab asks k8s to provision a new pod per pipeline anyway.

https://docs.gitlab.com/runner/fleet_scaling/#kubernetes-executor
https://docs.gitlab.com/runner/executors/kubernetes/

Interesting... as Gitlab already has an scaling system, should we have our own? @kedacore/keda-contributors
I'm wondering if Gitlab exposes an API that we can use to get the current Jobs in case of continuing with this scaler

@JorTurFer

We can totally get the current jobs, in fact the scaler does precisely that. But we don't really need to scale the workers ourselves since it's done by gitlab as far as I understand. Please correct me if I'm wrong.

Interesting... as Gitlab already has an scaling system, should we have our own? @kedacore/keda-contributors
I'm wondering if Gitlab exposes an API that we can use to get the current Jobs in case of continuing with this scaler

One thing to say yes, is to use it with Azure Container Apps, as the GitLab scaling system only works only with a Kubenetes Cluster. But Azure Container Apps allows to use KEDA.

Interesting... as Gitlab already has an scaling system, should we have our own? @kedacore/keda-contributors
I'm wondering if Gitlab exposes an API that we can use to get the current Jobs in case of continuing with this scaler

One thing to say yes, is to use it with Azure Container Apps, as the GitLab scaling system only works only with a Kubenetes Cluster. But Azure Container Apps allows to use KEDA.

Yeah, my thoughts were related with 3rd parties offering services on top of Kubernetes but hiding Kubernetes

That makes sense, but for which kind of workload shall I write the e2e tests then? Is there an idiomatic way to test scaler without a specific workload? Maybe some dummy pods?

To add the e2e tests, you can use ScaledJobs with just a dummy Gitlab runner that process a dummy job -> https://docs.gitlab.com/runner/install/docker.html

My question would be if we can spin up and configure a gitlab suite in kubernetes to connect the runner to, or if we need to create a gitlab account (which is totally doable if they offer a free tier)

Hey! I've been trying to get gitlab up and running in the kind, but I'm failing.

Using k8s fqdn doesn't work, because we cannot set the host ourselves
https://docs.gitlab.com/charts/charts/globals.html#hostsuffix

The only thing we can set is global.hosts.domain, which will be prepended with gitlab. subdomain, so if global.hosts.domain=local.gd, the address will be gitlab.local.gd.

So there is no way to use something like gitlab-webservice-default.default.svc.cluster.local.

I propose to create a test account in gitlab.com, because even though this setup is possible, there are problems involved with ssl, and I'm not sure I have enough expertise of how this is handled in keda e2e tests and what kind of infra is already out there.

Perhaps I could write a new chart that would leverage existing e2e test infra (like traefik or some other ingress controller or something), but it feels like a lot of work that doesn't make a lot of sense either.

I can use my own account for development, I'll leave a mechanism of substitution like env var, so that it can be ran properly with the existing keda account you'd create.

I'll manage cleanup on the e2e test level as well in order not to exceed gitlab free tier quotas.

Please let me know C:

Hi! Merry xmas and/or new year if you celebrate!

I've added some e2e tests, but they keep failing. When I described the scaledobject it says this

Status:
  Authentications Types:  gitlab-trigger-auth
  Conditions:
    Message:               failed to ensure HPA is correctly created for ScaledObject
    Reason:                ScaledObjectCheckFailed
    Status:                False
    Type:                  Ready
    Message:               ScaledObject check failed
    Reason:                UnknownState
    Status:                Unknown
    Type:                  Active
    Message:               No fallbacks are active on this scaled object
    Reason:                NoFallbackFound
    Status:                False
    Type:                  Fallback
    Status:                Unknown
    Type:                  Paused
  Original Replica Count:  0
  Scale Target GVKR:
    Group:            apps
    Kind:             Deployment
    Resource:         deployments
    Version:          v1
  Scale Target Kind:  apps/v1.Deployment
  Triggers Types:     gitlab-runner
Events:
  Type     Reason                   Age              From           Message
  ----     ------                   ----             ----           -------
  Warning  KEDAScalerFailed         0s (x9 over 2s)  keda-operator  no scaler found for type: gitlab-runner
  Warning  ScaledObjectCheckFailed  0s (x9 over 2s)  keda-operator  failed to ensure HPA is correctly created for ScaledObject

Which makes no sense since I've already registered the scaler in pkg/scaling/scalers_builder.go.
I'm using kind.

kind create cluster
go test -v -tags e2e ./utils/setup_test.go
env GITLAB_PAT=xxx go test -v -tags e2e ./scalers/gitlab_runner/...

I'm kind of out of ideas of what I might've been doing wrong at this point. What do you think?

You are running the test against a version of KEDA that doesn't contain your code. See the setup_test.go file where KEDA installation is handled, it is using make deploy target, which uses defaults set in the Makefile (ie main). Just build your KEDA and use it: https://github.com/kedacore/keda/blob/main/BUILD.md#custom-keda-as-an-image