tofu logs are not streamed anymore and just appears once plan is completed

[jgournet]

Describe the bug

As the title suggests, the tofu outputs when planning used to be displayed as things would progress, like saying which resource it's refreshing.
Since 0.71.x, those logs just get printed in one block at the end, once the plan is complete.
Note: the logs from the init phase are still being printed in a stream

Steps To Reproduce

Run any plan with basic terragrunt run-all plan command

Expected behavior

Logs should appear in real time (as mentioned in https://github.com/gruntwork-io/terragrunt/blob/main/docs/_docs/04_reference/logging.md :

Here you see two different units being run by Terragrunt concurrently, and stdout/stderr for each being emitted in real time.

Versions

• Terragrunt version: 0.71.2
• OpenTofu/Terraform version: tofu v1.8.8
• Environment details (Ubuntu 20.04, Windows 10, etc.): Ubuntu 24.04

Additional context

Not sure if it's only me or not; hopefully, it's easily reproducible

[yhakbar]

Could you give us more details on your setup, @jgournet ? I'm seeing logs stream concurrently on the latest Terragrunt:

$ terragrunt --version
terragrunt version v0.71.2

$ eza -T
.
├── unit1
│   ├── main.tf
│   └── terragrunt.hcl
└── unit2
    ├── main.tf
    └── terragrunt.hcl

$ terragrunt run-all plan
08:23:01.534 INFO   The stack at . will be processed in the following order for command plan:
Group 1
- Module ./unit1
- Module ./unit2


08:23:01.569 INFO   [unit1] tofu: Initializing the backend...
08:23:01.569 INFO   [unit1] tofu: Initializing provider plugins...
08:23:01.569 INFO   [unit1] tofu: OpenTofu has been successfully initialized!
08:23:01.569 INFO   [unit1] tofu:
08:23:01.569 INFO   [unit1] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see
08:23:01.569 INFO   [unit1] tofu: any changes that are required for your infrastructure. All OpenTofu commands
08:23:01.569 INFO   [unit1] tofu: should now work.
08:23:01.569 INFO   [unit1] tofu: If you ever set or change modules or backend configuration for OpenTofu,
08:23:01.569 INFO   [unit1] tofu: rerun this command to reinitialize your working directory. If you forget, other
08:23:01.569 INFO   [unit1] tofu: commands will detect it and remind you to do so if necessary.
08:23:01.569 INFO   [unit2] tofu: Initializing the backend...
08:23:01.569 INFO   [unit2] tofu: Initializing provider plugins...
08:23:01.569 INFO   [unit2] tofu: OpenTofu has been successfully initialized!
08:23:01.569 INFO   [unit2] tofu:
08:23:01.569 INFO   [unit2] tofu: You may now begin working with OpenTofu. Try running "tofu plan" to see
08:23:01.569 INFO   [unit2] tofu: any changes that are required for your infrastructure. All OpenTofu commands
08:23:01.569 INFO   [unit2] tofu: should now work.
08:23:01.569 INFO   [unit2] tofu: If you ever set or change modules or backend configuration for OpenTofu,
08:23:01.569 INFO   [unit2] tofu: rerun this command to reinitialize your working directory. If you forget, other
08:23:01.569 INFO   [unit2] tofu: commands will detect it and remind you to do so if necessary.
08:23:01.589 STDOUT [unit2] tofu: No changes. Your infrastructure matches the configuration.
08:23:01.589 STDOUT [unit2] tofu: OpenTofu has compared your real infrastructure against your configuration and
08:23:01.589 STDOUT [unit2] tofu: found no differences, so no changes are needed.
08:23:01.589 STDOUT [unit1] tofu: No changes. Your infrastructure matches the configuration.
08:23:01.589 STDOUT [unit1] tofu: OpenTofu has compared your real infrastructure against your configuration and
08:23:01.589 STDOUT [unit1] tofu: found no differences, so no changes are needed.

Are you using any additional flags? Terragrunt will buffer log output if you've configured it so that it's harder to determine what unit is running when.

[jgournet]

Thanks @yhakbar ,
Day to day, we use quite a few flags.
But to make it easy for this ticket, I did try with barebone command:

terragrunt run-all plan

With version 0.70.3: the "plan" logs are streamed
With version 0.71.2: logs are NOT streamed (note: the "init" logs are streamed though).

This can only be seen on a module with many resources (or least quite a few).
in 0.70.3, we see the lines such as "STDOUT tofu: aws_apigatewayv2_api.webhook: Refreshing state" lines appear as they are executed
in 0.71.2, those "refreshing state" appear all in 1 block with the plan itself (ie: when the whole plan is finished)

[jgournet]

Highly compressed and horrible gif of 0.70.3 that shows the logs being streamed
(init was already done; start is truncated to avoid being bored)

=> we see the refresh lines being streamed and finally the "No changes. Your infrastructure matches the configuration" appear

And the equivalent run with 0.71.2:

=> we see the CLI hanging for a while, and everything appearing at once

Please someone tell me I'm not the only one :)

[yhakbar]

@jgournet , I believe you!

I'll look into this. I think I might have misunderstood what we changed here to be honest. Once I have more information, I'll get back to you.

[yhakbar]

I was able to reproduce this issue with these configurations:

fd -tf -x bash -c 'echo "# {}" && cat {} && echo'
# ./unit2/terragrunt.hcl

# ./unit1/main.tf
resource "null_resource" "empty" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "echo 'sleeping...'; sleep 10; echo 'done sleeping'"
  }
}

# ./unit2/main.tf
resource "null_resource" "empty" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "echo 'sleeping...'; sleep 10; echo 'done sleeping'"
  }
}

# ./unit1/terragrunt.hcl

[lorengordon]

I believe this is impacting me also, and many CI "plans" are failing due to a timeout from lack of output. Running Terragrunt 0.71.2.