sunbeam/maas/terraform/networking: dial tcp 172.20.x.x:17070: i/o timeout

[javacruft]

Description

sunbeam: 2024.1/candidate

MAAS based deployment with multiple network spaces:

• internal

• infra (public)

and others which are not pertinent to this bug.

The client deploying the cloud has a connection to the infra network only; the Juju controller is deployed to a machine which has configured network interfaces on the internal and infra networks.

Both of these appear in the controller.yaml in the client but only one is addressable from the client (the infra one).

The juju command line tool seems to deal with this fine; however when applying terraform plans using the juju provider application will sometimes fail:

15:04:41,917 sunbeam.core.terraform WARNING
Error: Not Found

with juju_application.traefik-rgw[0],
on main.tf line 396, in resource "juju_application" "traefik-rgw":
396: resource "juju_application" "traefik-rgw" {

dial tcp 172.20.0.5:17070: i/o timeout

Error: Not Found

The 172.20.0.5 address is the controllers configured network interface on the internal network which the client will not be able to contact.

Urgency

Blocker for our release

Terraform Juju Provider version

0.15.1

Terraform version

1.5.7

Juju version

3.6.2

Terraform Configuration(s)

Unrelated to the terraform plan - the issue is around network address resolution and retires I think.

Reproduce / Test

See above

Debug/Panic Output

15:04:41,917 sunbeam.core.terraform WARNING
Error: Not Found

  with juju_application.traefik-rgw[0],
  on main.tf line 396, in resource "juju_application" "traefik-rgw":
 396: resource "juju_application" "traefik-rgw" {

dial tcp 172.20.0.5:17070: i/o timeout

Notes & References

No response

[SimoneDutto]

Hi @javacruft, thanks for filing this bug.
I have some questions for you:

• Did you manually configure the terraform provider?
  If you did, it should look something like this.

provider "juju" {
  controller_addresses = "<ip1>,<ip2>"
  username = "<redacted>"
  password = "<redacted>"
}

Or did you rely on environment vars or auto-discovery via the juju cli?

[javacruft]

Hi @SimoneDutto had a quick check

All of our plans leave the provider definition unset:

https://github.com/canonical/sunbeam-terraform/blob/main/main.tf#L27

provider "juju" {}

which I'm guessing is falling back on auto-discovery?

[SimoneDutto]

Hi @SimoneDutto had a quick check

All of our plans leave the provider definition unset:

https://github.com/canonical/sunbeam-terraform/blob/main/main.tf#L27

provider "juju" {}

which I'm guessing is falling back on auto-discovery?

i took a look at your repository, it could be that you have some environment variables set in your github actions.
Like: JUJU_CONTROLLER_ADDRESSES.
Can you check?
I would like to understand better what addresses the provider receives, because from my local tests, if there are two addresses configured and one is not responding it shouldn't be an issue.

The i/o timeout you are experiencing is inside a github runner or on a local machine?

Also, if you can point me to a failing github action with the io/timeout error would be helpful!

[SimoneDutto]

while we are investing the issue, you could set the address from within the provider definition.
Values in the provider definition have the precedence, so the other configs should remain the same but this address would be overriden.

Can you test it?

provider "juju" {
  controller_addresses = "<ip>:<port>"
}

[SimoneDutto]

@javacruft any luck with the previous suggestions?