Loki gateway cannot be scraped by prometheus when basicAuth is enabled

[adippl]

Describe the bug
Prometheus cannot collect logs from loki-gateway when authentication is enabled.

To Reproduce
Steps to reproduce the behavior:

1. Install loki 6.10.0 from helm
2. Enable monitoring and basicAuth

gateway:
  enabled: true
  basicAuth:
    enabled: true
monitoring:
  dashboards:
    enabled: true
  rules:
    enabled: true
    labels:
      prometheus: main    
  serviceMonitor:
    enabled: true
    labels:
      prometheus: main

3. Prometheus servicemonitor is created but prometheus cannot scrape gateway because of authorization. Error 401
   
   (All other targets created by this servicemonitor are scraped correctly)

Expected behavior
Servicemonitor should configure prometheus to use some authorization or gateway should be scraped via some non authenticated port.

Environment:

• Infrastructure: Kubernetes, kvm virtual machines
• Deployment tool: argocd

[dominikdeichsel]

Quick Workaround:

1. Add nginx-prometheus-exporter as described here:
   Loki gateway metrics (Nginx) #9522 (comment)

2. Deactivate basic_auth for the /metrics URL via

gateway:
  nginxConfig:
    serverSnippet: |
      location = /stub_status {
        stub_status on;
        auth_basic off;
        allow 127.0.0.1;
        deny all;
      }
      location = /metrics {
        proxy_pass       http://127.0.0.1:9113/metrics;
        auth_basic off;
      }

Loki Helm Chart does not implement basic auth configuration for the ServiceMonitor (tested with Helm Chart 6.12.0)