Deploy Getting Stuck

[schechterab]

I've got "spin up" working on my local, and now moving on to test "spin deploy", but the process is just getting stuck, stalled, hanging.

Any tips on how to troubleshoot this, or any similar experiences?

The attached screenshot shows where the process hung for about 3 hours, before I killed it.

[jaydrogers]

Strange! It definitely should not take 3 hours.

Sometimes it can be an issue with an enviornment variable not being set correctly, or a migration that fails, etc.

We put this guide together: https://serversideup.net/open-source/spin/docs/server-access/troubleshooting-your-application

Hopefully this can help a bit? See if you can SSH into the server and view the logs of the services to see if there are any clues

[kirilldakhniuk]

Hey Jay,

We're able to SSH into the server, however php and mysql services aren't running. I started fresh using the link above and I was able to see some errors during spin deploy. Our docker-compose.prod.yml modified just a bit to create an additional databases for our project. Worth mentioning that there's zero issues running Spin locally. I attached a couple of screenshots which hopefully could give a bit more information.

Thanks.

[jaydrogers]

The invalid mount config for type "bind" may also be pointing out the docker-compose.prod.yml is using bind volumes instead of named volumes https://docs.docker.com/reference/compose-file/volumes/

If MySQL is not starting, that could be the reason why PHP wont start if it depends on it

[kirilldakhniuk]

So I reverted docker-compose.prod.yml to the initial state (we're using spin pro, all files were created with spin init laravel-pro ). Despite the invalid mount config for type "bind" is gone we are back to initial issue when spin deploy is stuck... From what I can tell all required variables are set.

Is there anything else I can check?

[jaydrogers]

The services will not start because they are not considered healthy. I would highly recommend looking at this section of the docs and seeing if you can find anything: https://serversideup.net/open-source/spin/docs/server-access/troubleshooting-your-application#services-wont-start-or-are-not-accessible

If you have more output of those debugging commands, that may help too

[kirilldakhniuk]

Hey Jay, thanks for your quick response.

I've tried to troubleshoot using spin docs, however I wasn't able to find anything helpful to get understanding of what is wrong. There's no errors from what I can tell.

MySQL, Redis, SSH and Traefik containers are running, others don't.
I attached the response I got.

sudo docker service ps --no-trunc spin-production_php
ID                          NAME                        IMAGE                                                                                                                  NODE       DESIRED STATE   CURRENT STATE            ERROR     PORTS
mi382bbb6cl5090et8fpa0mjn   spin-production_php.1       127.0.0.1:5080/dockerfile.php:20250128204759@sha256:b08ba91ee40b66673d273b871cd4c8b077173d4dad798b0f2a9568eeda1ff2f9   spin-app   Shutdown        Shutdown 2 minutes ago             
kp7re6r06u6ffgcqaykems8fi    \_ spin-production_php.1   127.0.0.1:5080/dockerfile.php:20250128204759@sha256:b08ba91ee40b66673d273b871cd4c8b077173d4dad798b0f2a9568eeda1ff2f9   spin-app   Shutdown        Shutdown 2 minutes ago             
g77ae38aes9599szpt8dehb8f    \_ spin-production_php.1   127.0.0.1:5080/dockerfile.php:20250128204759@sha256:b08ba91ee40b66673d273b871cd4c8b077173d4dad798b0f2a9568eeda1ff2f9   spin-app   Shutdown        Shutdown 3 minutes ago             
ps47ox95sblawml92rh6esqex    \_ spin-production_php.1   127.0.0.1:5080/dockerfile.php:20250128204759@sha256:b08ba91ee40b66673d273b871cd4c8b077173d4dad798b0f2a9568eeda1ff2f9   spin-app   Shutdown        Shutdown 5 minutes ago             

sudo docker service inspect spin-production_php --pretty

ID:		t21feej4u8mlyq7j53anjajqb
Name:		spin-production_php
Labels:
 com.docker.stack.image=127.0.0.1:5080/dockerfile.php:20250128204759
 com.docker.stack.namespace=spin-production
 traefik.enable=true
 traefik.http.routers.laravel-web.entrypoints=websecure
 traefik.http.routers.laravel-web.rule=Host(`dev.app.contractorcommerce.com`)
 traefik.http.routers.laravel-web.tls=true
 traefik.http.routers.laravel-web.tls.certresolver=letsencryptresolver
 traefik.http.services.laravel-web.loadbalancer.healthcheck.interval=30s
 traefik.http.services.laravel-web.loadbalancer.healthcheck.path=/up
 traefik.http.services.laravel-web.loadbalancer.healthcheck.scheme=https
 traefik.http.services.laravel-web.loadbalancer.healthcheck.timeout=5s
 traefik.http.services.laravel-web.loadbalancer.server.port=8443
 traefik.http.services.laravel-web.loadbalancer.server.scheme=https
Service Mode:	Replicated
 Replicas:	1
Placement:
UpdateConfig:
 Parallelism:	1
 Delay:		5s
 On failure:	rollback
 Monitoring Period: 5s
 Max failure ratio: 0
 Update order:      start-first
RollbackConfig:
 Parallelism:	0
 On failure:	pause
 Monitoring Period: 5s
 Max failure ratio: 0
 Rollback order:    stop-first
ContainerSpec:
 Image:		127.0.0.1:5080/dockerfile.php:20250128204759@sha256:b08ba91ee40b66673d273b871cd4c8b077173d4dad798b0f2a9568eeda1ff2f9
 Env:		APP_ENV=production AUTORUN_LARAVEL_MIGRATION=false HEALTHCHECK_PATH=/up LOG_OUTPUT_LEVEL=debug PHP_OPCACHE_ENABLE=1 SSL_MODE=full 
Mounts:
 Target:	/var/www/html/storage/app/private/
  Source:	spin-production_storage_private
  ReadOnly:	false
  Type:		volume
 Target:	/var/www/html/storage/app/public/
  Source:	spin-production_storage_public
  ReadOnly:	false
  Type:		volume
 Target:	/var/www/html/storage/framework/sessions
  Source:	spin-production_storage_sessions
  ReadOnly:	false
  Type:		volume
 Target:	/var/www/html/storage/logs
  Source:	spin-production_storage_logs
  ReadOnly:	false
  Type:		volume
Resources:
Networks: spin-production_web-public 
Endpoint Mode:	vip

[jaydrogers]

Sorry for all the back and forth, but I am trying my best to help without being able to see it in front of me.

Few thoughts:

Can you "start over" with the deployment?

As long as you're okay with deleting all services ⚠️and ALL APPLICATION DATA⚠️ and try another deployment, you can follow these steps to reset the server back to a vanilla Docker Swarm server without any services or data on it: https://serversideup.net/open-source/spin/docs/server-access/troubleshooting-your-application#starting-over

Check your ENV

Many things I see when this issue comes up is the .env or .env.production values are not set correctly to match the values in the docker compose files:

  mariadb:
    environment:
      MARIADB_DATABASE: "${DB_DATABASE}"
      MARIADB_USER: "${DB_USERNAME}"
      MARIADB_PASSWORD: "${DB_PASSWORD}"

In the example above, I would need these in my .env.production file:

Warning

You might want to avoid using special characters in your passwords. Instead, use longer alphanumeric passwords instead.

DB_DATABASE=laravel
DB_USERNAME=laraveluser
DB_PASSWORD=myrandompasswordDezMXebMIgyhgCrgnLfHMG8PlE2DeRk