Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker_build_with_restart not restarting service after the first time #6424

Open
jeremyk opened this issue Aug 8, 2024 · 2 comments
Open
Labels
bug Something isn't working needs repro case

Comments

@jeremyk
Copy link

jeremyk commented Aug 8, 2024

Expected Behavior

The service should restart every time a sync file changes

Current Behavior

The first time a sync file changes it restarts as expected but does not on subsequent changes.

First time (with some minor redactions):

Will copy 1 file(s) to container: [app-65b7fdb4f9-9hcn/app]
- '/Users/j/dev/mi-repos/app/r/grpc_router.py' --> '/opt/service/app/r/grpc_router.py'
RUNNING: tar -C / -x -f -
[CMD 1/1] sh -c date > /tmp/.restart-proc
RUNNING: date > /tmp/.restart-proc
Hot reload on, skipping container restart: app-65b7fdb4f9-9hcn/app
  → Container app-65b7fdb4f9-9hcn/app updated!
2024-08-08 00:20:53.923 | INFO     | __main__:serve:18 - Initializing server
2024-08-08 00:20:53.932 | INFO     | __main__:serve:31 - Added services: grpc.reflection.v1alpha.ServerReflection
2024-08-08 00:20:53.933 | INFO     | __main__:serve:34 - Added reflection
2024-08-08 00:20:53.936 | INFO     | __main__:serve:37 - Started server on port: 50051

The second time it looks exactly the same but stops after → Container app-65b7fdb4f9-9hcn/app updated! line.
I verified that /tmp/.restart-proc is being updated and not sure what else could be causing this.

Steps to Reproduce

I'm not 100% sure it is specific or not to my setup.

Context

tilt doctor Output

Tilt: v0.33.19, built 2024-08-05
System: darwin-arm64
---
Docker
- Host: unix:///Users/j/.docker/run/docker.sock
- Server Version: 27.0.3
- API Version: 1.46
- Builder: 2
- Compose Version: v2.28.1-desktop.1
---
Kubernetes
- Env: docker-desktop
- Context: docker-desktop
- Cluster Name: docker-desktop
- Namespace: default
- Container Runtime: docker
- Version: v1.29.2
- Cluster Local Registry: none

About Your Use Case

Pretty frustrating as it took me a while to realize this was happening. For now I have disabled sync and fall back on container restart so this is not a critical issue but would definitely like some help resolving.

@jeremyk jeremyk added the bug Something isn't working label Aug 8, 2024
@nicks
Copy link
Member

nicks commented Aug 8, 2024

sorry to hear you're having trouble. i was not able to reproduce this problem. here are the repro steps i tried:

are you able to repro with the same project? that will help narrow down if it's specific to your project or specific to your machine.

@ethunk
Copy link

ethunk commented Aug 9, 2024

I am also facing this exact same issue!

here is a copy of my output from the tilt ui dashboard. I can see that Tilt correctly identifies that files have changed, and on the first pass will rerun a startup command. (You can see my print statements "THIS HAS STARTED").

But then new changes occur, but the entry command doesn't get rerun (as indicated by the lack of my print statement)

2024-08-09 21:21:56.117 | INFO     | __main__:serve:21 - Initializing server
THIS HAS STARTED
something
2024-08-09 21:21:56.211 | INFO     | __main__:serve:36 - Added services: SubjectLineService, SummaryService, ToneService, grpc.reflection.v1alpha.ServerReflection
2024-08-09 21:21:56.212 | INFO     | __main__:serve:39 - Added reflection
2024-08-09 21:21:56.233 | INFO     | __main__:serve:42 - Started server on port: 50051
1 File Changed: [app/router.py]
Will copy 1 file(s) to container: [gen-server-app-64d6fcbfc7-jc65j/gen-server]
- '/Users/ehunkler/MI/gen-server/app/router.py' --> '/opt/service/router.py'

RUNNING: tar -C / -x -f -
[CMD 1/1] sh -c date > /tmp/.restart-proc

RUNNING: date > /tmp/.restart-proc
  → Container gen-server-app-64d6fcbfc7-jc65j/gen-server updated!

something
something
2024-08-09 21:22:11.605 | INFO     | __main__:serve:21 - Initializing server
THIS HAS STARTED
2024-08-09 21:22:11.629 | INFO     | __main__:serve:36 - Added services: SubjectLineService, SummaryService, ToneService, grpc.reflection.v1alpha.ServerReflection
2024-08-09 21:22:11.630 | INFO     | __main__:serve:39 - Added reflection
2024-08-09 21:22:11.636 | INFO     | __main__:serve:42 - Started server on port: 50051


1 File Changed: [app/router.py]
Will copy 1 file(s) to container: [gen-server-app-64d6fcbfc7-jc65j/gen-server]
- '/Users/ehunkler/MI/gen-server/app/router.py' --> '/opt/service/router.py'
RUNNING: tar -C / -x -f -
[CMD 1/1] sh -c date > /tmp/.restart-proc
RUNNING: date > /tmp/.restart-proc
  → Container gen-server-app-64d6fcbfc7-jc65j/gen-server updated!


1 File Changed: [app/router.py]
Will copy 1 file(s) to container: [gen-server-app-64d6fcbfc7-jc65j/gen-server]
- '/Users/ehunkler/MI/gen-server/app/router.py' --> '/opt/service/router.py'
RUNNING: tar -C / -x -f -
[CMD 1/1] sh -c date > /tmp/.restart-proc
RUNNING: date > /tmp/.restart-proc
  → Container gen-server-app-64d6fcbfc7-jc65j/gen-server updated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs repro case
Projects
None yet
Development

No branches or pull requests

3 participants