Debug linkerd2-proxy

[zhxie]

Hi, I am new to Linkerd and wish to learn the data plane of Linkerd. I use VS Code with CodeLLDB extension installed to launch linkerd2-proxy for debugging. The linkerd2-proxy launched successfully and the lldb was also attached, but I could not stop linkerd2-proxy on any breakpoint in the source. When I paused linkerd2-proxy, only the assembly code displayed and it showed there is no source.

Since there is not too much documentation about the data plane of Linkerd, I may need some help to walk through the code.

Here is the launch configuration:

{
    "type": "lldb",
    "request": "launch",
    "name": "Debug executable 'linkerd2-proxy'",
    "cargo": {
        "args": [
            "build",
            "--bin=linkerd2-proxy",
            "--package=linkerd2-proxy"
        ],
        "filter": {
            "name": "linkerd2-proxy",
            "kind": "bin"
        }
    },
    "args": [],
    "env": {
        "LINKERD2_PROXY_DESTINATION_SVC_ADDR": "linkerd-dst.linkerd.svc.cluster.local:8086",
        "LINKERD2_PROXY_IDENTITY_DISABLED": "true",
        "LINKERD2_PROXY_CONTROL_LISTEN_ADDR": "0.0.0.0:4190",
        "LINKERD2_PROXY_ADMIN_LISTEN_ADDR": "0.0.0.0:4191",
        "LINKERD2_PROXY_OUTBOUND_LISTEN_ADDR": "127.0.0.1:4140",
        "LINKERD2_PROXY_INBOUND_LISTEN_ADDR": "0.0.0.0:4143",
        "LINKERD2_PROXY_DESTINATION_GET_SUFFIXES": "svc.cluster.local.",
        "LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES": "svc.cluster.local."
    },
    "cwd": "${workspaceFolder}"
}

[hawkw]

Hi @zhxie,

The proxy project configures Cargo to not generate debug info for dev and test builds. This is because the debug symbols generated for the proxy have been quite large in the past, taking up a lot of disk space. This would explain why you can't see source code in your debugger, and can't set breakpoints on source lines.

You can change this configuration by removing the following lines from the Cargo.toml file in the repo root:

[profile.dev]
debug = false

[profile.test]
debug = false

Then, debug symbols should be generated, and (hopefully) your debugger should work. I haven't used the VS Code CodeLLDB extension, though, so it's possible there are other other issues, but removing those lines should cause the build to generate debug symbols.

[hawkw]

As a side note, I think that configuration is no longer relevant --- we've been disabling debug symbols for three years (see linkerd/linkerd2-proxy#135), and the proxy has changed significantly since then, including the reduction of some of the code patterns that were resulting in massive debug symbols.

It looks like the debug-mode proxy is now under 1GB with or without debug symbols, although the proxy with debug symbols is still significantly larger than the version without. On my machine, I get a 942 MB binary with debug symbols, and a 125 MB binary without.

# with debug symbols
:; ls -la target/debug/linkerd2-proxy  
.rwxr-xr-x eliza users 942 MB Tue Sep  7 09:31:49 2021 🗋  target/debug/linkerd2-proxy

# without debug symbols
:; ls -la target/debug/linkerd2-proxy  
.rwxr-xr-x eliza users 125 MB Tue Sep  7 09:37:32 2021 🗋  target/debug/linkerd2-proxy

# (and, of course, the release-mode proxy is significantly smaller than both)
:; ls -la target/release/linkerd2-proxy 
.rwxr-xr-x eliza users 37 MB Tue Sep  7 09:46:29 2021 🗋  target/release/linkerd2-proxy