Skip to content

Conversation

@ielshal
Copy link

@ielshal ielshal commented Nov 12, 2025

Summary

We are using rexec for recorded access for containers inside k8s clusters. For webapps and jobs we also need copy functionality. Since kubectl cp depends on exec, we implemented our own copy using rexec.

Security Reasons: Only copying FROM pods is supported (download only). Copying TO pods is intentionally disabled for security reasons.

Changes in this PR

  • Implement kubectl rexec cp for audited file transfers (download only)
  • Add comprehensive tests for cp functionality
  • Update documentation with cp command examples

Unit Testing

cd plugin && go test -v
=== RUN   TestParseFileSpec
=== RUN   TestParseFileSpec/local_file
=== RUN   TestParseFileSpec/pod_file
=== RUN   TestParseFileSpec/pod_file_with_namespace
=== RUN   TestParseFileSpec/file_path_with_colon
--- PASS: TestParseFileSpec (0.00s)
    --- PASS: TestParseFileSpec/local_file (0.00s)
    --- PASS: TestParseFileSpec/pod_file (0.00s)
    --- PASS: TestParseFileSpec/pod_file_with_namespace (0.00s)
    --- PASS: TestParseFileSpec/file_path_with_colon (0.00s)
=== RUN   TestValidateLocalDestination
=== RUN   TestValidateLocalDestination/existing_directory
=== RUN   TestValidateLocalDestination/existing_file
=== RUN   TestValidateLocalDestination/new_file_in_existing_dir
=== RUN   TestValidateLocalDestination/parent_does_not_exist
--- PASS: TestValidateLocalDestination (0.00s)
    --- PASS: TestValidateLocalDestination/existing_directory (0.00s)
    --- PASS: TestValidateLocalDestination/existing_file (0.00s)
    --- PASS: TestValidateLocalDestination/new_file_in_existing_dir (0.00s)
    --- PASS: TestValidateLocalDestination/parent_does_not_exist (0.00s)
=== RUN   TestExtractTarSingleFile
--- PASS: TestExtractTarSingleFile (0.00s)
=== RUN   TestExtractTarDirectory
--- PASS: TestExtractTarDirectory (0.00s)
=== RUN   TestExtractTarPathTraversal
--- PASS: TestExtractTarPathTraversal (0.00s)
=== RUN   TestExtractTarSymlinkSkipped
--- PASS: TestExtractTarSymlinkSkipped (0.00s)
=== RUN   TestExtractTarValidDoubleDotFilename
--- PASS: TestExtractTarValidDoubleDotFilename (0.00s)
=== RUN   TestRunWithArgsValidation
=== RUN   TestRunWithArgsValidation/local_to_pod_-_not_supported
=== RUN   TestRunWithArgsValidation/pod_to_pod_-_not_supported
=== RUN   TestRunWithArgsValidation/local_to_local_-_source_must_be_pod
=== RUN   TestRunWithArgsValidation/empty_pod_path
--- PASS: TestRunWithArgsValidation (0.00s)
    --- PASS: TestRunWithArgsValidation/local_to_pod_-_not_supported (0.00s)
    --- PASS: TestRunWithArgsValidation/pod_to_pod_-_not_supported (0.00s)
    --- PASS: TestRunWithArgsValidation/local_to_local_-_source_must_be_pod (0.00s)
    --- PASS: TestRunWithArgsValidation/empty_pod_path (0.00s)
=== RUN   TestExtractRemotePath
=== RUN   TestExtractRemotePath/standard_tar_error
=== RUN   TestExtractRemotePath/with_prefix
=== RUN   TestExtractRemotePath/no_match
=== RUN   TestExtractRemotePath/empty
--- PASS: TestExtractRemotePath (0.00s)
    --- PASS: TestExtractRemotePath/standard_tar_error (0.00s)
    --- PASS: TestExtractRemotePath/with_prefix (0.00s)
    --- PASS: TestExtractRemotePath/no_match (0.00s)
    --- PASS: TestExtractRemotePath/empty (0.00s)
=== RUN   TestValidateCopySpecs
=== RUN   TestValidateCopySpecs/valid_pod_to_local
=== RUN   TestValidateCopySpecs/local_to_pod_blocked
=== RUN   TestValidateCopySpecs/pod_to_pod_blocked
=== RUN   TestValidateCopySpecs/empty_remote_path
--- PASS: TestValidateCopySpecs (0.00s)
    --- PASS: TestValidateCopySpecs/valid_pod_to_local (0.00s)
    --- PASS: TestValidateCopySpecs/local_to_pod_blocked (0.00s)
    --- PASS: TestValidateCopySpecs/pod_to_pod_blocked (0.00s)
    --- PASS: TestValidateCopySpecs/empty_remote_path (0.00s)
PASS
ok      github.com/adyen/kubectl-rexec/plugin   0.639s
cd rexec/server && go test -v
=== RUN   TestExecHandlerUnsupportedContentType
--- PASS: TestExecHandlerUnsupportedContentType (0.00s)
=== RUN   TestExecHandlerBadJSON
--- PASS: TestExecHandlerBadJSON (0.00s)
=== RUN   TestExecHandlerAllowsNonExecKinds
--- PASS: TestExecHandlerAllowsNonExecKinds (0.00s)
=== RUN   TestExecHandlerBypassedUser
--- PASS: TestExecHandlerBypassedUser (0.00s)
=== RUN   TestExecHandlerSecretSauce
--- PASS: TestExecHandlerSecretSauce (0.00s)
=== RUN   TestExecHandlerExecDenied
--- PASS: TestExecHandlerExecDenied (0.00s)
=== RUN   TestCanPassBypassUser
--- PASS: TestCanPassBypassUser (0.00s)
=== RUN   TestCanPassSecretSauceMatch
--- PASS: TestCanPassSecretSauceMatch (0.00s)
=== RUN   TestCanPassNoMatch
--- PASS: TestCanPassNoMatch (0.00s)
=== RUN   TestWaitForListenerReady
--- PASS: TestWaitForListenerReady (0.00s)
=== RUN   TestRexecHandlerMissingUser
--- PASS: TestRexecHandlerMissingUser (0.00s)
PASS
ok      github.com/adyen/kubectl-rexec/rexec/server     0.353s

Manual Testing

Setup Test Environment

# 1. Create Kind cluster
kind create cluster --name rexec-test --image kindest/node:v1.30.0

# 2. Build and load image
docker build -t kubectl-rexec:test .
kind load docker-image kubectl-rexec:test --name rexec-test

# 3. Deploy with test image
cd manifests
kustomize edit set image ghcr.io/adyen/kubectl-rexec:latest=kubectl-rexec:test
kubectl apply -k . --context kind-rexec-test

# 4. Build plugin
go build -o kubectl-rexec main.go
chmod +x kubectl-rexec

# 5. Create test pod
kubectl run test-pod --image=ubuntu:22.04 -- sleep 3600
kubectl wait --for=condition=ready pod/test-pod --timeout=60s

Test Cases

1. Basic Exec

./kubectl-rexec exec test-pod -- echo "hello"
hello

kubectl -n kube-system logs -l app=rexec --tail=5
{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"echo hello","time":"2026-01-23T14:42:19Z"}

2. File Download

./kubectl-rexec exec test-pod -- sh -c 'echo "from pod" > /tmp/download.txt'
./kubectl-rexec cp test-pod:/tmp/download.txt /tmp/download.txt
Copied test-pod:/tmp/download.txt to /tmp/download.txt

cat /tmp/download.txt
from pod

3. Directory Download

./kubectl-rexec exec test-pod -- sh -c 'mkdir -p /tmp/testdir/sub && echo "file1" > /tmp/testdir/file1.txt && echo "file2" > /tmp/testdir/sub/file2.txt'
./kubectl-rexec cp test-pod:/tmp/testdir /tmp/downloaded
Copied test-pod:/tmp/testdir to /tmp/downloaded

ls -R /tmp/downloaded

file1.txt sub

/tmp/downloaded/sub:
file2.txt

4. Multi-Container Pod

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: multi-pod
spec:
  containers:
  - name: c1
    image: ubuntu:22.04
    command: ["sleep", "3600"]
  - name: c2
    image: ubuntu:22.04
    command: ["sleep", "3600"]
EOF

kubectl wait --for=condition=ready pod/multi-pod --timeout=60s
pod/multi-pod condition met

./kubectl-rexec exec multi-pod -c c2 -- sh -c 'echo "from c2" > /tmp/test.txt'
./kubectl-rexec cp multi-pod:/tmp/test.txt /tmp/from-c2.txt -c c2
Copied multi-pod:/tmp/test.txt to /tmp/from-c2.txt

cat /tmp/from-c2.txt
from c2

5. Upload Blocked (Security)

./kubectl-rexec cp /tmp/local-file.txt test-pod:/tmp/dest.txt
error: copying to pods is not supported for security reasons; only pod to local copy is allowed

Audit Logs

kubectl -n kube-system logs -l app=rexec --tail=5

{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"tar cf - -C /tmp -- download.txt","time":"2026-01-23T14:43:04Z"}
{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"sh -c mkdir -p /tmp/testdir/sub && echo \"file1\" > /tmp/testdir/file1.txt && echo \"file2\" > /tmp/testdir/sub/file2.txt","time":"2026-01-23T14:43:30Z"}
{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"tar cf - -C /tmp -- testdir","time":"2026-01-23T14:43:34Z"}
{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"sh -c echo \"from c2\" > /tmp/test.txt","time":"2026-01-23T14:44:29Z"}
{"level":"info","facility":"audit","user":"kubernetes-admin","session":"oneoff","command":"tar cf - -C /tmp -- test.txt","time":"2026-01-23T14:44:34Z"}

Cleanup

kubectl delete pod test-pod multi-pod
kind delete cluster --name rexec-test
rm -f /tmp/download.txt /tmp/from-c2.txt
rm -rf /tmp/downloaded

Fixed issue:

As mentioned we are introducing new feature so nothing to be fixed, the idea is to implement kubectl cp using rexec.

@ielshal ielshal requested a review from a team as a code owner November 12, 2025 22:15
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from 22f5266 to 17a06a7 Compare November 13, 2025 16:02
@sonarqubecloud
Copy link

@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from 17a06a7 to 7b37706 Compare January 23, 2026 14:56
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from 7b37706 to f895e6a Compare January 23, 2026 17:18
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from f895e6a to aedf702 Compare January 23, 2026 18:28
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from aedf702 to 5e606bb Compare January 23, 2026 18:58
- Enforce strict security: block uploads and path traversal

- Add comprehensive tests and docs
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from 5e606bb to 732c4f1 Compare January 23, 2026 20:25
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from f317146 to 6d9ea55 Compare January 23, 2026 20:40
@ielshal ielshal force-pushed the ibrahim/implemenet-copy-command-into-kubectl-rexec branch from 6d9ea55 to 5753cf2 Compare January 23, 2026 20:44
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant