Certificate retrieval failed! - Wait for certificate

[bitsmyth]

Description of problem

Certificate cannot be issued for app with FQDN as the app name when it has multiple domains added to it using Docker as a build strategy.

I have tested my app and it is available on non https protocol. My Dockerfile exposes :5000 by default.

How to reproduce

dokku domains:clear-global (serving multiple client projects on a single server)
dokku apps:create example.com
dokku domains:add example.com example.com www.example.com
dokku letsencrypt:set example.com server staging
dokku letsencrypt:enable example.com

Actual Results

dokku letsencrypt:enable example.io -v

user@host:/var/log/nginx$ dokku letsencrypt:enable example.io -v
=====> Enabling letsencrypt for example.io
-----> Enabling ACME proxy for example.io...
-----> Getting letsencrypt certificate for example.io via HTTP-01
        - Domain 'www.example.io'
        - Domain 'example.io'
2025/01/02 13:51:35 No key found for account [email protected]. Generating a P256 key.
2025/01/02 13:51:35 Saved key to /certs/accounts/acme-staging-v02.api.letsencrypt.org/[email protected]/keys/[email protected]
2025/01/02 13:51:36 [INFO] acme: Registering account for [email protected]
2025/01/02 13:51:36 [INFO] [www.example.io, example.io] acme: Obtaining bundled SAN certificate
       !!!! HEADS UP !!!!

       Your account credentials have been saved in your Let's Encrypt
       configuration directory at "/certs/accounts".

       You should make a secure backup of this folder now. This
       configuration directory will also contain certificates and
       private keys obtained from Let's Encrypt so making regular
       backups of this folder is ideal.
2025/01/02 13:51:37 [INFO] [example.io] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178497124/15573219194
2025/01/02 13:51:37 [INFO] [www.example.io] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178497124/15573219204
2025/01/02 13:51:37 [INFO] [example.io] acme: Could not find solver for: tls-alpn-01
2025/01/02 13:51:37 [INFO] [example.io] acme: use http-01 solver
2025/01/02 13:51:37 [INFO] [www.example.io] acme: Could not find solver for: tls-alpn-01
2025/01/02 13:51:37 [INFO] [www.example.io] acme: use http-01 solver
2025/01/02 13:51:37 [INFO] [example.io] acme: Trying to solve HTTP-01
2025/01/02 13:51:42 [INFO] [www.example.io] acme: Trying to solve HTTP-01
2025/01/02 13:51:49 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178497124/15573219194
2025/01/02 13:51:50 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178497124/15573219204
2025/01/02 13:51:50 Could not obtain certificates:
        error: one or more domains had a problem:
[example.io] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 188.245.184.22: Invalid response from http://example.io/.well-known/acme-challenge/BOcppMr0rpfWZTENvvNwxtgbVRpRPa-9oclzBd7DlQo: 404
[www.example.io] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 188.245.184.22: Invalid response from http://www.example.io/.well-known/acme-challenge/vSRoLkMMyWsD415y7PUoGosqBcYk_dSPAjItK1-MleE: 404
-----> Certificate retrieval failed!
-----> Disabling ACME proxy for example.io...
 !     Failed to setup letsencrypt
 !     Check log output for further information on failure

Expected Results

Issue SSL certificate for www and non-www domain names.

Environment Information

dokku report example.io

[sudo] password for user: 
-----> uname: Linux host 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec  5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
-----> memory: 
                      total        used        free      shared  buff/cache   available
       Mem:            3820         788         431          24        2918        3031
       Swap:           2047           0        2047
-----> disk utilization: 
       Filesystem      Size  Used Avail Use% Mounted on
       /dev/sda1        38G  6.7G   30G  19% /
-----> disk inode utilization: 
       Filesystem     Inodes IUsed IFree IUse% Mounted on
       /dev/sda1        2.4M  136K  2.2M    6% /
-----> docker version: 
       Client: Docker Engine - Community
        Version:           27.4.1
        API version:       1.47
        Go version:        go1.22.10
        Git commit:        b9d17ea
        Built:             Tue Dec 17 15:45:46 2024
        OS/Arch:           linux/amd64
        Context:           default
       
       Server: Docker Engine - Community
        Engine:
         Version:          27.4.1
         API version:      1.47 (minimum version 1.24)
         Go version:       go1.22.10
         Git commit:       c710b88
         Built:            Tue Dec 17 15:45:46 2024
         OS/Arch:          linux/amd64
         Experimental:     false
        containerd:
         Version:          1.7.24
         GitCommit:        88bf19b2105c8b17560993bee28a01ddc2f97182
        runc:
         Version:          1.2.2
         GitCommit:        v1.2.2-0-g7cb3632
        docker-init:
         Version:          0.19.0
         GitCommit:        de40ad0
-----> docker daemon info: 
       Client: Docker Engine - Community
        Version:    27.4.1
        Context:    default
        Debug Mode: true
        Plugins:
         buildx: Docker Buildx (Docker Inc.)
           Version:  v0.19.3
           Path:     /usr/libexec/docker/cli-plugins/docker-buildx
         compose: Docker Compose (Docker Inc.)
           Version:  v2.32.1
           Path:     /usr/libexec/docker/cli-plugins/docker-compose
       
       Server:
        Containers: 2
         Running: 2
         Paused: 0
         Stopped: 0
        Images: 7
        Server Version: 27.4.1
        Storage Driver: overlay2
         Backing Filesystem: extfs
         Supports d_type: true
         Using metacopy: false
         Native Overlay Diff: true
         userxattr: false
        Logging Driver: json-file
        Cgroup Driver: systemd
        Cgroup Version: 2
        Plugins:
         Volume: local
         Network: bridge host ipvlan macvlan null overlay
         Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
        Swarm: inactive
        Runtimes: runc io.containerd.runc.v2
        Default Runtime: runc
        Init Binary: docker-init
        containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
        runc version: v1.2.2-0-g7cb3632
        init version: de40ad0
        Security Options:
         apparmor
         seccomp
          Profile: builtin
         cgroupns
        Kernel Version: 6.8.0-51-generic
        Operating System: Ubuntu 24.04.1 LTS
        OSType: linux
        Architecture: x86_64
        CPUs: 2
        Total Memory: 3.73GiB
        Name: host
        ID: 1a353ed3-0f1b-41e0-b89f-68dc6678b75f
        Docker Root Dir: /var/lib/docker
        Debug Mode: false
         File Descriptors: 35
         Goroutines: 51
         System Time: 2025-01-02T14:09:35.97199872Z
         EventsListeners: 0
        Experimental: false
        Insecure Registries:
         127.0.0.0/8
        Live Restore Enabled: false
       
-----> herokuish version: 
 !     Herokuish image gliderlabs/herokuish:latest-24 is not available
-----> dokku version: dokku version 0.35.13
 !     dokku-event-listener binary is not available
 !     dokku-update binary is not available
-----> docker-container-healthchecker version: v0.11.3
-----> docker-image-labeler version: v0.8.0
-----> git version: git version 2.43.0
-----> lambda-builder version:        v0.8.0                                                                          
-----> netrc version: v0.10.0
 !     pack binary is not available
-----> plugn version: plugn: v0.16.0
-----> sigil version: v0.11.0
-----> sshcommand version: sshcommand v0.18.1
-----> dokku plugins: 
         00_dokku-standard    0.35.13 enabled    dokku core standard plugin
         20_events            0.35.13 enabled    dokku core events logging plugin
         app-json             0.35.13 enabled    dokku core app-json plugin
         apps                 0.35.13 enabled    dokku core apps plugin
         builder              0.35.13 enabled    dokku core builder plugin
         builder-dockerfile   0.35.13 enabled    dokku core builder-dockerfile plugin
         builder-herokuish    0.35.13 enabled    dokku core builder-herokuish plugin
         builder-lambda       0.35.13 enabled    dokku core builder-lambda plugin
         builder-nixpacks     0.35.13 enabled    dokku core builder-nixpacks plugin
         builder-null         0.35.13 enabled    dokku core builder-null plugin
         builder-pack         0.35.13 enabled    dokku core builder-pack plugin
         buildpacks           0.35.13 enabled    dokku core buildpacks plugin
         caddy-vhosts         0.35.13 enabled    dokku core caddy-vhosts plugin
         certs                0.35.13 enabled    dokku core certificate management plugin
         checks               0.35.13 enabled    dokku core checks plugin
         common               0.35.13 enabled    dokku core common plugin
         config               0.35.13 enabled    dokku core config plugin
         cron                 0.35.13 enabled    dokku core cron plugin
         docker-options       0.35.13 enabled    dokku core docker-options plugin
         domains              0.35.13 enabled    dokku core domains plugin
         enter                0.35.13 enabled    dokku core enter plugin
         git                  0.35.13 enabled    dokku core git plugin
         haproxy-vhosts       0.35.13 enabled    dokku core haproxy-vhosts plugin
         letsencrypt          0.20.4 enabled    Automated installation of let's encrypt TLS certificates
         logs                 0.35.13 enabled    dokku core logs plugin
         mysql                1.41.0 enabled    dokku mysql service plugin
         network              0.35.13 enabled    dokku core network plugin
         nginx-vhosts         0.35.13 enabled    dokku core nginx-vhosts plugin
         openresty-vhosts     0.35.13 enabled    dokku core openresty-vhosts plugin
         plugin               0.35.13 enabled    dokku core plugin plugin
         ports                0.35.13 enabled    dokku core ports plugin
         proxy                0.35.13 enabled    dokku core proxy plugin
         ps                   0.35.13 enabled    dokku core ps plugin
         redirect             0.9.1 enabled    Plugin for managing application redirects
         registry             0.35.13 enabled    dokku core registry plugin
         repo                 0.35.13 enabled    dokku core repo plugin
         resource             0.35.13 enabled    dokku core resource plugin
         run                  0.35.13 enabled    dokku core run plugin
         scheduler            0.35.13 enabled    dokku core scheduler plugin
         scheduler-docker-local 0.35.13 enabled    dokku core scheduler-docker-local plugin
         scheduler-k3s        0.35.13 enabled    dokku core scheduler-k3s plugin
         scheduler-null       0.35.13 enabled    dokku core scheduler-null plugin
         shell                0.35.13 enabled    dokku core shell plugin
         ssh-keys             0.35.13 enabled    dokku core ssh-keys plugin
         storage              0.35.13 enabled    dokku core storage plugin
         trace                0.35.13 enabled    dokku core trace plugin
         traefik-vhosts       0.35.13 enabled    dokku core traefik-vhosts plugin
=====> example.io app-json information
       App json computed selected:    app.json
       App json global selected:      app.json
       App json selected:             
=====> example.io app information
       App created at:                1735768135
       App deploy source:             git-push
       App deploy source metadata:    8f2d945c1d767e056069b88952a9c77bb935d763
       App dir:                       /home/dokku/example.io
       App locked:                    false
=====> example.io builder information
       Builder build dir:             laravel
       Builder computed build dir:    laravel
       Builder computed selected:     
       Builder global build dir:      
       Builder global selected:       
       Builder selected:              
=====> example.io builder-dockerfile information
       Builder dockerfile computed dockerfile path: Dockerfile               
       Builder dockerfile global dockerfile path: Dockerfile               
       Builder dockerfile dockerfile path:                          
=====> example.io builder-herokuish information
       Builder herokuish computed allowed: true                     
       Builder herokuish global allowed: true                     
       Builder herokuish allowed:                              
=====> example.io builder-lambda information
       Builder lambda computed lambdayml path: lambda.yml               
       Builder lambda global lambdayml path: lambda.yml               
       Builder lambda lambdayml path:                          
=====> example.io builder-nixpacks information
       Builder nixpacks computed nixpackstoml path: nixpacks.toml            
       Builder nixpacks global nixpackstoml path: nixpacks.toml            
       Builder nixpacks nixpackstoml path:                          
       Builder nixpacks computed no cache: false                    
       Builder nixpacks global no cache: false                    
       Builder nixpacks no cache:                              
=====> example.io builder-pack information
       Builder pack computed projecttoml path: project.toml             
       Builder pack global projecttoml path: project.toml             
       Builder pack projecttoml path:                          
=====> example.io buildpacks information
       Buildpacks computed stack:     gliderlabs/herokuish:latest-24
       Buildpacks global stack:       
       Buildpacks list:               
       Buildpacks stack:              
=====> example.io caddy information
       Caddy image:                   lucaslorentz/caddy-docker-proxy:2.9
       Caddy letsencrypt email:                                
       Caddy letsencrypt server:      https://acme-v02.api.letsencrypt.org/directory
       Caddy log level:               ERROR                    
       Caddy polling interval:        5s                       
       Caddy tls internal:            false                    
=====> example.io ssl information
       Ssl dir:                       /home/dokku/example.io/tls
       Ssl enabled:                   false                    
       Ssl hostnames:                                          
       Ssl expires at:                                         
       Ssl issuer:                                             
       Ssl starts at:                                          
       Ssl subject:                                            
       Ssl verified:                                           
=====> example.io checks information
       Checks disabled list:          none                     
       Checks skipped list:           none                     
       Checks computed wait to retire: 60                       
       Checks global wait to retire:  60                       
       Checks wait to retire:                                  
=====> example.io docker options information
       Docker options build:          --link dokku.mysql.example:dokku-mysql-example --target=production 
       Docker options deploy:         --link dokku.mysql.example:dokku-mysql-example --restart=on-failure:10 
       Docker options run:            --link dokku.mysql.example:dokku-mysql-example 
=====> example.io domains information
       Domains app enabled:           true                     
       Domains app vhosts:            www.example.io example.io  
       Domains global enabled:        false                    
       Domains global vhosts:                                  
=====> example.io git information
       Git deploy branch:             laravel/setup            
       Git global deploy branch:      master                   
       Git keep git dir:              false                    
       Git rev env var:               GIT_REV                  
       Git sha:                       HEAD                     
       Git source image:                                       
       Git last updated at:           1735768487               
=====> example.io haproxy information
       Haproxy image:                 byjg/easy-haproxy:4.4.0  
       Haproxy letsencrypt email:                              
       Haproxy letsencrypt server:    https://acme-v02.api.letsencrypt.org/directory
       Haproxy log level:             ERROR                    
Could not open file or uri for loading certificate from /home/dokku/example.io/tls/server.crt
4087F446A67C0000:error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file
4087F446A67C0000:error:80000002:system library:file_open:No such file or directory:../providers/implementations/storemgmt/file_store.c:267:calling stat(/home/dokku/example.io/tls/server.crt)
Unable to load certificate
=====> example.io letsencrypt information
       Letsencrypt active:            false                    
       Letsencrypt autorenew:         true                     
       Letsencrypt computed dns provider:                          
       Letsencrypt global dns provider:                          
       Letsencrypt dns provider:                               
       Letsencrypt computed email:    [email protected]  
       Letsencrypt global email:      [email protected]  
       Letsencrypt email:                                      
       Letsencrypt expiration:        1735776000               
       Letsencrypt computed graceperiod: 2592000                  
       Letsencrypt global graceperiod:                          
       Letsencrypt graceperiod:                                
       Letsencrypt computed lego docker args:                          
       Letsencrypt global lego docker args:                          
       Letsencrypt lego docker args:                           
       Letsencrypt computed server:   https://acme-staging-v02.api.letsencrypt.org/directory
       Letsencrypt global server:                              
       Letsencrypt server:            staging                  
=====> example.io logs information
       Logs computed max size:        10m
       Logs global max size:          10m
       Logs global vector sink:       
       Logs max size:                 
       Logs vector global image:      timberio/vector:0.43.1-debian
       Logs vector sink:              
=====> example.io network information
       Network attach post create:           
       Network attach post deploy:           
       Network bind all interfaces:          false
       Network computed attach post create:  
       Network computed attach post deploy:  
       Network computed bind all interfaces: false
       Network computed initial network:     
       Network computed tld:                 
       Network global attach post create:    
       Network global attach post deploy:    
       Network global bind all interfaces:   false
       Network global initial network:       
       Network global tld:                   
       Network initial network:              
       Network static web listener:          
       Network tld:                          
       Network web listeners:                172.17.0.4:5000
=====> example.io nginx information
       Nginx access log format:                                
       Nginx computed access log format:                          
       Nginx global access log format:                          
       Nginx access log path:                                  
       Nginx computed access log path: /var/log/nginx/example.io-access.log
       Nginx global access log path:  /var/log/nginx/example.io-access.log
       Nginx bind address ipv4:                                
       Nginx computed bind address ipv4:                          
       Nginx global bind address ipv4:                          
       Nginx bind address ipv6:                                
       Nginx computed bind address ipv6: ::                       
       Nginx global bind address ipv6: ::                       
       Nginx client body timeout:                              
       Nginx computed client body timeout: 60s                      
       Nginx global client body timeout: 60s                      
       Nginx client header timeout:                            
       Nginx computed client header timeout: 60s                      
       Nginx global client header timeout: 60s                      
       Nginx client max body size:                             
       Nginx computed client max body size: 1m                       
       Nginx global client max body size: 1m                       
       Nginx disable custom config:                            
       Nginx computed disable custom config: false                    
       Nginx global disable custom config: false                    
       Nginx error log path:                                   
       Nginx computed error log path: /var/log/nginx/example.io-error.log
       Nginx global error log path:   /var/log/nginx/example.io-error.log
       Nginx hsts include subdomains:                          
       Nginx computed hsts include subdomains: true                     
       Nginx global hsts include subdomains: true                     
       Nginx hsts max age:                                     
       Nginx computed hsts max age:   15724800                 
       Nginx global hsts max age:     15724800                 
       Nginx hsts preload:                                     
       Nginx computed hsts preload:   false                    
       Nginx global hsts preload:     false                    
       Nginx hsts:                                             
       Nginx computed hsts:           true                     
       Nginx global hsts:             true                     
       Nginx last visited at:                                  
       Nginx keepalive timeout:                                
       Nginx computed keepalive timeout: 75s                      
       Nginx global keepalive timeout: 75s                      
       Nginx lingering timeout:                                
       Nginx computed lingering timeout: 5s                       
       Nginx global lingering timeout: 5s                       
       Nginx nginx conf sigil path:                            
       Nginx computed nginx conf sigil path: nginx.conf.sigil         
       Nginx global nginx conf sigil path: nginx.conf.sigil         
       Nginx proxy buffer size:                                
       Nginx computed proxy buffer size: 4k                       
       Nginx global proxy buffer size: 4k                       
       Nginx proxy buffering:                                  
       Nginx computed proxy buffering: on                       
       Nginx global proxy buffering:  on                       
       Nginx proxy buffers:                                    
       Nginx computed proxy buffers:  8 4k                     
       Nginx global proxy buffers:    8 4k                     
       Nginx proxy busy buffers size:                          
       Nginx computed proxy busy buffers size: 8k                       
       Nginx global proxy busy buffers size: 8k                       
       Nginx proxy connect timeout:                            
       Nginx computed proxy connect timeout: 60s                      
       Nginx global proxy connect timeout: 60s                      
       Nginx proxy read timeout:                               
       Nginx computed proxy read timeout: 60s                      
       Nginx global proxy read timeout: 60s                      
       Nginx proxy send timeout:                               
       Nginx computed proxy send timeout: 60s                      
       Nginx global proxy send timeout: 60s                      
       Nginx send timeout:                                     
       Nginx computed send timeout:   60s                      
       Nginx global send timeout:     60s                      
       Nginx underscore in headers:                            
       Nginx computed underscore in headers: off                      
       Nginx global underscore in headers: off                      
       Nginx x forwarded for value:                            
       Nginx computed x forwarded for value: $remote_addr             
       Nginx global x forwarded for value: $remote_addr             
       Nginx x forwarded port value:                           
       Nginx computed x forwarded port value: $server_port             
       Nginx global x forwarded port value: $server_port             
       Nginx x forwarded proto value:                          
       Nginx computed x forwarded proto value: $scheme                  
       Nginx global x forwarded proto value: $scheme                  
       Nginx x forwarded ssl:                                  
       Nginx computed x forwarded ssl:                          
       Nginx global x forwarded ssl:                           
=====> example.io openresty information
       Openresty access log format:                            
       Openresty access log path:     /var/log/nginx/example.io-access.log
       Openresty allowed letsencrypt domains func base64: cmV0dXJuIHRydWUK         
       Openresty bind address ipv4:                            
       Openresty bind address ipv6:   ::                       
       Openresty client body timeout: 60s                      
       Openresty client header timeout: 60s                      
       Openresty client max body size:                          
       Openresty error log path:      /var/log/nginx/example.io-error.log
       Openresty global hsts:         true                     
       Openresty computed hsts:       true                     
       Openresty hsts:                                         
       Openresty hsts include subdomains: true                     
       Openresty hsts max age:        15724800                 
       Openresty hsts preload:        false                    
       Openresty image:               dokku/openresty-docker-proxy:0.9.1
       Openresty keepalive timeout:   75s                      
       Openresty letsencrypt email:                            
       Openresty letsencrypt server:  https://acme-v02.api.letsencrypt.org/directory
       Openresty lingering timeout:   5s                       
       Openresty proxy buffer size:   4k                       
       Openresty proxy buffering:     on                       
       Openresty proxy buffers:       8 4k                     
       Openresty proxy busy buffers size: 8k                       
       Openresty proxy connect timeout: 60s                      
       Openresty proxy read timeout:  60s                      
       Openresty proxy send timeout:  60s                      
       Openresty send timeout:        60s                      
       Openresty underscore in headers: off                      
       Openresty x forwarded for value: $remote_addr             
       Openresty x forwarded port value: $server_port             
       Openresty x forwarded proto value: $scheme                  
       Openresty x forwarded ssl:                              
=====> example.io ports information
       Ports map:                     ttp:80:5000
       Ports map detected:            http:80:5000
=====> example.io proxy information
       Proxy computed type:           nginx
       Proxy enabled:                 true
       Proxy global type:             nginx
       Proxy type:                    
=====> example.io ps information
       Deployed:                      true
       Processes:                     1
       Ps can scale:                  true
       Ps computed procfile path:     Procfile
       Ps global procfile path:       Procfile
       Ps procfile path:              
       Ps restart policy:             on-failure:10
       Restore:                       true
       Running:                       true
       Status web 1:                  running (CID: a826ae0f29e)
=====> example.io registry information
       Registry computed image repo:        dokku/example.io
       Registry computed push on release:   false
       Registry computed server:            
       Registry global image repo template: 
       Registry global push on release:     
       Registry global server:              
       Registry image repo:                 
       Registry push extra tags:            
       Registry push on release:            
       Registry server:                     
       Registry tag version:                
=====> example.io resource information
=====> example.io scheduler information
       Scheduler computed selected:   docker-local
       Scheduler global selected:     docker-local
       Scheduler selected:            
=====> example.io scheduler-docker-local information
       Scheduler docker local init process: true                     
       Scheduler docker local parallel schedule count:                          
=====> example.io scheduler-k3s information
       Scheduler k3s computed deploy timeout:       300s
       Scheduler k3s computed image pull secrets:   
       Scheduler k3s computed letsencrypt server:   prod
       Scheduler k3s computed namespace:            default
       Scheduler k3s computed rollback on failure:  false
       Scheduler k3s deploy timeout:                
       Scheduler k3s global deploy timeout:         300s
       Scheduler k3s global image pull secrets:     
       Scheduler k3s global ingress class:          nginx
       Scheduler k3s global kube context:           
       Scheduler k3s global kubeconfig path:        /etc/rancher/k3s/k3s.yaml
       Scheduler k3s global letsencrypt email prod: 
       Scheduler k3s global letsencrypt email stag: 
       Scheduler k3s global letsencrypt server:     prod
       Scheduler k3s global namespace:              default
       Scheduler k3s global network interface:      eth0
       Scheduler k3s global rollback on failure:    false
       Scheduler k3s image pull secrets:            
       Scheduler k3s letsencrypt server:            
       Scheduler k3s namespace:                     
       Scheduler k3s rollback on failure:           
=====> example.io storage information
       Storage build mounts:                                   
       Storage deploy mounts:                                  
       Storage run mounts:                                     
=====> example.io traefik information
       Traefik api enabled:           false                    
       Traefik api vhost:             traefik.dokku.me         
       Traefik basic auth password:                            
       Traefik basic auth username:                            
       Traefik dashboard enabled:     false                    
       Traefik image:                 traefik:3.2.2            
       Traefik letsencrypt email:                              
       Traefik letsencrypt server:    https://acme-v02.api.letsencrypt.org/directory
       Traefik log level:             ERROR                    
       Traefik http entry point:      http                     
       Traefik https entry point:     https  

How (deb/make) and where (AWS, VirtualBox, physical, etc.) was Dokku installed?:

Ubuntu 24.04 @hetzner using x64 with vCpus.

Additional information

dokku nginx:show-config expample.com

upstream example.com-5000 {

  server 172.17.0.4:5000;
}

[josegonzalez]

The port map you set is ttp as the protocol, not http. Fix that and this will probably work.

[bitsmyth]

Indeed, there was a typo - thank you for pointing that out. I have corrected it and restarted the container, but unfortunately, nothing has changed in that regard. It is still stuck in the "Wait for certificate" process. I have disabled and enabled letsencrypt for that application as well, without any success either.

[josegonzalez]

What is the nginx config after the updated port map was set

[bitsmyth]

server {
  listen      [::]:80;
  listen      80;
  server_name www.example.io example.io; 
  access_log  /var/log/nginx/example.io-access.log;
  error_log   /var/log/nginx/example.io-error.log;
  underscores_in_headers off;

  client_body_timeout 60s;
  client_header_timeout 60s;
  keepalive_timeout 75s;
  lingering_timeout 5s;
  send_timeout 60s;

  location    / {

    gzip on;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_types    text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/wasm application/json application/xml application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
    gzip_vary on;
    gzip_comp_level  6;

    proxy_pass  http://example.io-5000;
    proxy_http_version 1.1;
    proxy_connect_timeout 60s;
    proxy_read_timeout 60s;
    proxy_send_timeout 60s;
    proxy_buffer_size 4k;
    proxy_buffering on;
    proxy_buffers 8 4k;
    proxy_busy_buffers_size 8k;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Request-Start $msec;
    
  }

  client_max_body_size 1m;

  error_page 400 401 402 403 405 406 407 408 409 410 411 412 413 414 415 416 417 418 420 422 423 424 426 428 429 431 444 449 450 451 /400-error.html;
  location /400-error.html {
    root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
    internal;
  }

  error_page 404 /404-error.html;
  location /404-error.html {
    root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
    internal;
  }

  error_page 500 501 502 503 504 505 506 507 508 509 510 511 /500-error.html;
  location /500-error.html {
    root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
    internal;
  }
  include /home/dokku/example.io/nginx.conf.d/*.conf;

}

upstream example.io-5000 {

  server 172.17.0.3:5000;
}

I didn't set any port besides now, like you suggested, http:80:5000 but viewing the nginx configuration, I seem that 443 is not set, so I ran dokku ports:add example.com https:443:5000 but got the following error:

 !     Ignoring detected https port mapping without an accompanying ssl certificate (https:443:5000)
-----> Configuring example.io...(using built-in template)
-----> Configuring www.example.io...(using built-in template)
template: nginx.conf.sigil:4:19: executing "nginx.conf.sigil" at <index $port_map_list 1>: error calling index: reflect: slice index out of range
 !      !     Ignoring detected https port mapping without an accompanying ssl certificate (https:443:5000)
template: nginx.conf.sigil:4:19: executing "nginx.conf.sigil" at <index $port_map_list 1>: error calling index: reflect: slice index out of range

[josegonzalez]

Remove the extra https port mapping. That only works if there is an ssl cert.

Can you show the output of letsencrypt:enable for that app now?

[bitsmyth]

I have removed the https port mapping like you suggested, but still having no success. My domain has valid records (A and CNAME as www).

letsencrypt:disable

-----> Disabling letsencrypt for app
       Removing letsencrypt files for example.io
       Removing SSL endpoint from example.io
-----> Skipping DOKKU_PROXY_SSL_PORT, it is not set in the environment
-----> Configuring example.io...(using built-in template)
-----> Configuring www.example.io...(using built-in template)
-----> Creating http nginx.conf
       Reloading nginx
-----> Done

letsencrypt:enable

=====> Enabling letsencrypt for example.io
-----> Enabling ACME proxy for example.io...
-----> Getting letsencrypt certificate for example.io via HTTP-01
        - Domain 'example.io'
        - Domain 'www.example.io'
2025/01/03 15:32:46 No key found for account [email protected]. Generating a P256 key.
2025/01/03 15:32:46 Saved key to /certs/accounts/acme-staging-v02.api.letsencrypt.org/[email protected]/keys/[email protected]
2025/01/03 15:32:47 [INFO] acme: Registering account for [email protected]
       !!!! HEADS UP !!!!
       
       Your account credentials have been saved in your Let's Encrypt
       configuration directory at "/certs/accounts".
       
       You should make a secure backup of this folder now. This
       configuration directory will also contain certificates and
       private keys obtained from Let's Encrypt so making regular
       backups of this folder is ideal.
2025/01/03 15:32:47 [INFO] [example.io, www.example.io] acme: Obtaining bundled SAN certificate
2025/01/03 15:32:48 [INFO] [example.io] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178641314/15582920074
2025/01/03 15:32:48 [INFO] [www.example.io] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/178641314/15582920084
2025/01/03 15:32:48 [INFO] [example.io] acme: Could not find solver for: tls-alpn-01
2025/01/03 15:32:48 [INFO] [example.io] acme: use http-01 solver
2025/01/03 15:32:48 [INFO] [www.example.io] acme: Could not find solver for: tls-alpn-01
2025/01/03 15:32:48 [INFO] [www.example.io] acme: use http-01 solver
2025/01/03 15:32:48 [INFO] [example.io] acme: Trying to solve HTTP-01
2025/01/03 15:32:54 [INFO] [example.io] The server validated our request
2025/01/03 15:32:54 [INFO] [www.example.io] acme: Trying to solve HTTP-01
2025/01/03 15:32:58 [INFO] [www.example.io] The server validated our request
2025/01/03 15:32:58 [INFO] [example.io, www.example.io] acme: Validations succeeded; requesting certificates
2025/01/03 15:32:59 [INFO] Wait for certificate [timeout: 720h0m0s, interval: 12h0m0s]

I want to point out that I appreciate your help and guidance very much at this stage!

[josegonzalez]

Interesting. Try setting the letsencrypt server property to default and retrying.

[bitsmyth]

Indeed, it is sure that it works flawlessly in contrast. I guess the whole issue is rather connected with having global vhosts cleared using the letsencrypt staging server.