Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support newer Docker API versions by ignoring additional properties in API responses #24

Closed
wants to merge 1 commit into from
Closed

Conversation

edigaryev
Copy link
Contributor

Recently Docker 19.03 was released and it introduced the new 1.40 API version with whole lot of new fields, which cause Bollard to fail due to it's use of deny_unknown_fields serde attribute.

Quote from the Docker Engine API reference:

The API uses an open schema model, which means server may add extra properties to responses.

Likewise, the server will ignore any extra query parameters and request body properties.

When you write clients, you need to ignore additional properties in responses to ensure they do not break when talking to newer daemons.

…n API responses

Recently Docker 19.03 was released and it introduced the new 1.40 API version
with whole lot of new fields[1], which cause Bollard to fail due to it's use of
deny_unknown_fields serde attribute.

Quote from the Docker Engine API reference[2]:

>The API uses an open schema model, which means server may add extra properties
>to responses.

>Likewise, the server will ignore any extra query parameters and request body
>properties.

>When you write clients, you need to ignore additional properties in responses
>to ensure they do not break when talking to newer daemons.

[1]: https://docs.docker.com/engine/api/version-history/#v140-api-changes
[2]: https://docs.docker.com/engine/api/v1.39/#section/Versioning
@fussybeaver
Copy link
Owner

Yes, you're right that the API introduces new fields breaking the old API. However, I previously believed that API pegging was sufficient to maintain API compliance - can you give a minimal example or an error log using RUST_LOG=debug where the upstream server was updated and the request fails ?

@edigaryev
Copy link
Contributor Author

examples/poc.rs:

extern crate bollard;
extern crate env_logger;
extern crate tokio;

use bollard::container::InspectContainerOptions;
use bollard::Docker;
use std::env;
use tokio::runtime::current_thread::Runtime;

fn main() {
    env_logger::init();

    let args: Vec<String> = env::args().collect();

    let docker = Docker::connect_with_unix_defaults().unwrap();
    let mut rt = Runtime::new().unwrap();
    let future = docker.inspect_container(&args[1], None::<InspectContainerOptions>);
    rt.block_on(future).unwrap();
}
$ docker version
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:22:03 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:20:35 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
$ RUST_LOG=debug cargo run --example poc 9ff515742d42
[2019-07-31T20:45:54Z DEBUG bollard::uri] Parsing uri: unix://2f7661722f72756e2f646f636b65722e736f636b/containers/9ff515742d42/json, client_type: Unix, socket: /var/run/docker.sock
[2019-07-31T20:45:54Z DEBUG tokio_reactor] adding I/O source: 0
[2019-07-31T20:45:54Z DEBUG tokio_reactor::registration] scheduling Write for: 0
[2019-07-31T20:45:54Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-31T20:45:54Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::io] flushed 126 bytes
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::io] read 4740 bytes
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::io] parsed 7 headers
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::conn] incoming body is chunked encoding
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::decode] incoming chunked header: 0x11A5 (4517 bytes)
[2019-07-31T20:45:54Z DEBUG hyper::proto::h1::conn] incoming body completed
[2019-07-31T20:45:54Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-31T20:45:54Z DEBUG tokio_reactor::registration] scheduling Read for: 0
[2019-07-31T20:45:54Z DEBUG tokio_reactor] dropping I/O source: 0
[2019-07-31T20:45:54Z DEBUG bollard::docker] Decoded into string: {"Id":"9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f","Created":"2019-07-31T20:43:09.556823261Z","Path":"bash","Args":[],"State":{"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":2604,"ExitCode":0,"Error":"","StartedAt":"2019-07-31T20:43:12.485498861Z","FinishedAt":"0001-01-01T00:00:00Z"},"Image":"sha256:8d31923452f8b79ae91b01568d28c90e7d667a9eaff9734c6faeb017b0efa8d0","ResolvConfPath":"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/resolv.conf","HostnamePath":"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/hostname","HostsPath":"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/hosts","LogPath":"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f-json.log","Name":"/silly_satoshi","RestartCount":0,"Driver":"overlay2","Platform":"linux","MountLabel":"","ProcessLabel":"","AppArmorProfile":"docker-default","ExecIDs":null,"HostConfig":{"Binds":null,"ContainerIDFile":"","LogConfig":{"Type":"json-file","Config":{}},"NetworkMode":"default","PortBindings":{},"RestartPolicy":{"Name":"no","MaximumRetryCount":0},"AutoRemove":true,"VolumeDriver":"","VolumesFrom":null,"CapAdd":null,"CapDrop":null,"Capabilities":null,"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IpcMode":"private","Cgroup":"","Links":null,"OomScoreAdj":0,"PidMode":"","Privileged":false,"PublishAllPorts":false,"ReadonlyRootfs":false,"SecurityOpt":null,"UTSMode":"","UsernsMode":"","ShmSize":67108864,"Runtime":"runc","ConsoleSize":[0,0],"Isolation":"","CpuShares":0,"Memory":0,"NanoCpus":0,"CgroupParent":"","BlkioWeight":0,"BlkioWeightDevice":[],"BlkioDeviceReadBps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteIOps":null,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpusetCpus":"","CpusetMems":"","Devices":[],"DeviceCgroupRules":null,"DeviceRequests":null,"KernelMemory":0,"KernelMemoryTCP":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":null,"OomKillDisable":false,"PidsLimit":null,"Ulimits":null,"CpuCount":0,"CpuPercent":0,"IOMaximumIOps":0,"IOMaximumBandwidth":0,"MaskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"ReadonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]},"GraphDriver":{"Data":{"LowerDir":"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f-init/diff:/var/lib/docker/overlay2/404978e99cf7ca0b546b1785718e25713ee080f80d5d985528f871eba73668a6/diff","MergedDir":"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/merged","UpperDir":"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/diff","WorkDir":"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/work"},"Name":"overlay2"},"Mounts":[],"Config":{"Hostname":"9ff515742d42","Domainname":"","User":"","AttachStdin":true,"AttachStdout":true,"AttachStderr":true,"Tty":true,"OpenStdin":true,"StdinOnce":true,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["bash"],"Image":"debian:9","Volumes":null,"WorkingDir":"","Entrypoint":null,"OnBuild":null,"Labels":{}},"NetworkSettings":{"Bridge":"","SandboxID":"08dde8e5e3a696e1693f0c892bcaedce4a801d3da083381ef8fb45b30314c5f2","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{},"SandboxKey":"/var/run/docker/netns/08dde8e5e3a6","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"ccc72b60802be1efd8a1200e155fb94920f66a346cc78301bf647d8e2ff0acc0","Gateway":"10.10.0.1","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"10.10.0.2","IPPrefixLen":24,"IPv6Gateway":"","MacAddress":"02:42:0a:0a:00:02","Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"f5f266376be904235945925b57d7e0f4f10b0153e3fdb585660846411304007b","EndpointID":"ccc72b60802be1efd8a1200e155fb94920f66a346cc78301bf647d8e2ff0acc0","Gateway":"10.10.0.1","IPAddress":"10.10.0.2","IPPrefixLen":24,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:0a:0a:00:02","DriverOpts":null}}}}

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: JsonDataError { message: "unknown field `Capabilities`, expected one of `Binds`, `Links`, `Memory`, `MemorySwap`, `MemoryReservation`, `KernelMemory`, `NanoCpus`, `CpuPercent`, `CpuShares`, `CpuPeriod`, `CpuRealtimePeriod`, `CpuRealtimeRuntime`, `CpuQuota`, `CpusetCpus`, `CpusetMems`, `BlkioWeight`, `BlkioWeightDevice`, `BlkioDeviceReadBps`, `BlkioDeviceWriteBps`, `BlkioDeviceReadIOps`, `BlkioDeviceWriteIOps`, `MemorySwappiness`, `OomKillDisable`, `OomScoreAdj`, `PidMode`, `PidsLimit`, `PortBindings`, `PublishAllPorts`, `Privileged`, `ReadonlyRootfs`, `Dns`, `DnsOptions`, `DnsSearch`, `VolumesFrom`, `CapAdd`, `CapDrop`, `GroupAdd`, `RestartPolicy`, `AutoRemove`, `NetworkMode`, `Devices`, `Ulimits`, `LogConfig`, `SecurityOpt`, `CgroupParent`, `VolumeDriver`, `ShmSize`, `ContainerIDFile`, `ExtraHosts`, `IpcMode`, `Cgroup`, `UTSMode`, `UsernsMode`, `Runtime`, `ConsoleSize`, `Isolation`, `DeviceCgroupRules`, `DiskQuota`, `CpuCount`, `IOMaximumIOps`, `IOMaximumBandwidth`, `MaskedPaths`, `ReadonlyPaths`, `Sysctls` at line 1 column 1427", contents: "{\"Id\":\"9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f\",\"Created\":\"2019-07-31T20:43:09.556823261Z\",\"Path\":\"bash\",\"Args\":[],\"State\":{\"Status\":\"running\",\"Running\":true,\"Paused\":false,\"Restarting\":false,\"OOMKilled\":false,\"Dead\":false,\"Pid\":2604,\"ExitCode\":0,\"Error\":\"\",\"StartedAt\":\"2019-07-31T20:43:12.485498861Z\",\"FinishedAt\":\"0001-01-01T00:00:00Z\"},\"Image\":\"sha256:8d31923452f8b79ae91b01568d28c90e7d667a9eaff9734c6faeb017b0efa8d0\",\"ResolvConfPath\":\"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/resolv.conf\",\"HostnamePath\":\"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/hostname\",\"HostsPath\":\"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/hosts\",\"LogPath\":\"/var/lib/docker/containers/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f/9ff515742d4220f6a688ecdcc69f973e91ecc118a7d606c2f8f4b4b163dd457f-json.log\",\"Name\":\"/silly_satoshi\",\"RestartCount\":0,\"Driver\":\"overlay2\",\"Platform\":\"linux\",\"MountLabel\":\"\",\"ProcessLabel\":\"\",\"AppArmorProfile\":\"docker-default\",\"ExecIDs\":null,\"HostConfig\":{\"Binds\":null,\"ContainerIDFile\":\"\",\"LogConfig\":{\"Type\":\"json-file\",\"Config\":{}},\"NetworkMode\":\"default\",\"PortBindings\":{},\"RestartPolicy\":{\"Name\":\"no\",\"MaximumRetryCount\":0},\"AutoRemove\":true,\"VolumeDriver\":\"\",\"VolumesFrom\":null,\"CapAdd\":null,\"CapDrop\":null,\"Capabilities\":null,\"Dns\":[],\"DnsOptions\":[],\"DnsSearch\":[],\"ExtraHosts\":null,\"GroupAdd\":null,\"IpcMode\":\"private\",\"Cgroup\":\"\",\"Links\":null,\"OomScoreAdj\":0,\"PidMode\":\"\",\"Privileged\":false,\"PublishAllPorts\":false,\"ReadonlyRootfs\":false,\"SecurityOpt\":null,\"UTSMode\":\"\",\"UsernsMode\":\"\",\"ShmSize\":67108864,\"Runtime\":\"runc\",\"ConsoleSize\":[0,0],\"Isolation\":\"\",\"CpuShares\":0,\"Memory\":0,\"NanoCpus\":0,\"CgroupParent\":\"\",\"BlkioWeight\":0,\"BlkioWeightDevice\":[],\"BlkioDeviceReadBps\":null,\"BlkioDeviceWriteBps\":null,\"BlkioDeviceReadIOps\":null,\"BlkioDeviceWriteIOps\":null,\"CpuPeriod\":0,\"CpuQuota\":0,\"CpuRealtimePeriod\":0,\"CpuRealtimeRuntime\":0,\"CpusetCpus\":\"\",\"CpusetMems\":\"\",\"Devices\":[],\"DeviceCgroupRules\":null,\"DeviceRequests\":null,\"KernelMemory\":0,\"KernelMemoryTCP\":0,\"MemoryReservation\":0,\"MemorySwap\":0,\"MemorySwappiness\":null,\"OomKillDisable\":false,\"PidsLimit\":null,\"Ulimits\":null,\"CpuCount\":0,\"CpuPercent\":0,\"IOMaximumIOps\":0,\"IOMaximumBandwidth\":0,\"MaskedPaths\":[\"/proc/asound\",\"/proc/acpi\",\"/proc/kcore\",\"/proc/keys\",\"/proc/latency_stats\",\"/proc/timer_list\",\"/proc/timer_stats\",\"/proc/sched_debug\",\"/proc/scsi\",\"/sys/firmware\"],\"ReadonlyPaths\":[\"/proc/bus\",\"/proc/fs\",\"/proc/irq\",\"/proc/sys\",\"/proc/sysrq-trigger\"]},\"GraphDriver\":{\"Data\":{\"LowerDir\":\"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f-init/diff:/var/lib/docker/overlay2/404978e99cf7ca0b546b1785718e25713ee080f80d5d985528f871eba73668a6/diff\",\"MergedDir\":\"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/merged\",\"UpperDir\":\"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/diff\",\"WorkDir\":\"/var/lib/docker/overlay2/84fd28192d99464d05ecbb3241d94c73423cb81d32b7255cff0d5aec14f7e22f/work\"},\"Name\":\"overlay2\"},\"Mounts\":[],\"Config\":{\"Hostname\":\"9ff515742d42\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":true,\"AttachStdout\":true,\"AttachStderr\":true,\"Tty\":true,\"OpenStdin\":true,\"StdinOnce\":true,\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Cmd\":[\"bash\"],\"Image\":\"debian:9\",\"Volumes\":null,\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":{}},\"NetworkSettings\":{\"Bridge\":\"\",\"SandboxID\":\"08dde8e5e3a696e1693f0c892bcaedce4a801d3da083381ef8fb45b30314c5f2\",\"HairpinMode\":false,\"LinkLocalIPv6Address\":\"\",\"LinkLocalIPv6PrefixLen\":0,\"Ports\":{},\"SandboxKey\":\"/var/run/docker/netns/08dde8e5e3a6\",\"SecondaryIPAddresses\":null,\"SecondaryIPv6Addresses\":null,\"EndpointID\":\"ccc72b60802be1efd8a1200e155fb94920f66a346cc78301bf647d8e2ff0acc0\",\"Gateway\":\"10.10.0.1\",\"GlobalIPv6Address\":\"\",\"GlobalIPv6PrefixLen\":0,\"IPAddress\":\"10.10.0.2\",\"IPPrefixLen\":24,\"IPv6Gateway\":\"\",\"MacAddress\":\"02:42:0a:0a:00:02\",\"Networks\":{\"bridge\":{\"IPAMConfig\":null,\"Links\":null,\"Aliases\":null,\"NetworkID\":\"f5f266376be904235945925b57d7e0f4f10b0153e3fdb585660846411304007b\",\"EndpointID\":\"ccc72b60802be1efd8a1200e155fb94920f66a346cc78301bf647d8e2ff0acc0\",\"Gateway\":\"10.10.0.1\",\"IPAddress\":\"10.10.0.2\",\"IPPrefixLen\":24,\"IPv6Gateway\":\"\",\"GlobalIPv6Address\":\"\",\"GlobalIPv6PrefixLen\":0,\"MacAddress\":\"02:42:0a:0a:00:02\",\"DriverOpts\":null}}}}\n", column: 1427 }', src/libcore/result.rs:1084:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

@fussybeaver
Copy link
Owner

Adopted in #25 Thanks!

@edigaryev edigaryev deleted the handle-open-schema-model branch August 14, 2019 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants