Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for address_mode = "alloc" #14

Open
cneira opened this issue Jun 16, 2021 · 1 comment
Open

Add support for address_mode = "alloc" #14

cneira opened this issue Jun 16, 2021 · 1 comment
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed
Milestone

Comments

@cneira
Copy link
Owner

cneira commented Jun 16, 2021

@cneira Thanks for your update.

Now also can not support address_mode = "alloc"

cni conf: /etc/cni/conf.d/firecracker.conflist

{
  "name": "firecracker",
  "cniVersion": "0.4.0",
  "plugins": [
    {
      "type": "ptp",
      "ipMasq": true,
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.60.0/24",
        "resolvConf": "/etc/resolv.conf"
      }
    },
    {
      "type": "tc-redirect-tap"
    }
  ]
}

job config

job "hello" {
    datacenters = ["dc1"]
    type = "service"

    group "sshd" {
        network {
            # mode = "cni/mynet"
            port "ssh" {
                to = 22
            }
        }
        service {
            name = "sshd"
            port = "ssh"
            address_mode = "alloc"
            check {
                type = "tcp"
                interval = "10s"
                timeout = "2s"
                address_mode = "alloc"
            }
        }

        task "sshd" {
            driver = "firecracker-task-driver"

            config {
                KernelImage = "/home/ox0spy/projects/nomad/study/firecracker/vmlinux.bin"
                BootDisk = "/home/ox0spy/projects/nomad/study/firecracker/rootfs.ext4"
                Firecracker = "/usr/local/bin/firecracker"
                Vcpus       = 1
                Mem         = 128
                Network     = "firecracker"
            }
        }
    }
}

docs for address_mode in service block: https://www.nomadproject.io/docs/job-specification/service#address_mode

run job

nomad status <alloc-id> got the below error message:

Setup Failure  failed to setup alloc: pre-run hook "group_services" failed: unable to get address for service "sshd": cannot use address_mode="alloc": no allocation network status reported

Originally posted by @ox0spy in #9 (comment)

@cneira cneira added bug Something isn't working help wanted Extra attention is needed labels Jun 16, 2021
@cneira cneira added this to the v1.3.0 milestone Jun 16, 2021
@cneira cneira changed the title add support for address_mode = "alloc" Add support for address_mode = "alloc" Jun 16, 2021
@cneira cneira added the enhancement New feature or request label Jun 16, 2021
@zh4n7wm
Copy link

zh4n7wm commented Jun 18, 2021

update more detail description.

add new capability

diff --git a/driver/driver.go b/driver/driver.go
index c5fc599..c7e65da 100644
--- a/driver/driver.go
+++ b/driver/driver.go
@@ -83,6 +83,7 @@ var (
                SendSignals: false,
                Exec:        false,
                FSIsolation: drivers.FSIsolationImage,
+               NetIsolationModes: []drivers.NetIsolationMode{drivers.NetIsolationModeGroup, drivers.NetIsolationModeTask},
        }
 )

add mode = "cni/firecracker" to network block, then did not report this error; but consul get the wrong ip address.

I think using address_mode = "alloc" and network -> mode = "cni/firecracker", nomad will assign the ip address for service, so firecracker-task-driver should use assigned ip address.

but, it seems that firecracker apply a new ip address: https://github.com/cneira/firecracker-task-driver/blob/master/driver/options.go#L165

containerd task driver use cfg.NetworkIsolation.Path for new container/allocation : https://github.com/Roblox/nomad-driver-containerd/blob/master/containerd/driver.go#L452

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants