Added trace, build allowlist, and enforce modes #2

carminecesarano · 2024-10-07T14:48:35Z

Todo:

test three modes on real packages

algomaster99

I suggested one change and some comments. Do answer them when you have time.

algomaster99 · 2024-10-08T16:24:20Z

track_syscalls/binanalyzer/binanalyzer.go

+		if sym.Value != 0 {
+			Cache = append(Cache, SymbolInfo{
+				Name:  sym.Name,
+				Start: sym.Value,
+				End:   sym.Value + sym.Size,
+			})
+		}


What symbols from elf file are you storing here and for what?

Since goleash retrieves only the addresses of invoked functions from the stack, we need to translate these addresses into function symbols from the elf binary. Function symbols from the binary come along with package names, which is how goleash can infer the caller dependency.

note: i don't resolve symbols at runtime during enforcement; instead, I resolve and cache all the function symbols once during initialization, for performance reasons.

algomaster99 · 2024-10-08T16:32:22Z

track_syscalls/main.go

+func runTraceMode(binaryPath string) {
+	setupAndRun(binaryPath, func(event ebpfEvent, stackTrace []uint64, objs *ebpfObjects) {
+		// Just log the event
+		logEvent(event, stackTrace, objs)


We would only want to log the violation of the event, eventually right? This would log complete stacktrace based on all eBPF comments, right?

For now, for each execution mode, I am logging everything for debugging and testing purpose. We can eventually add a debug mode flag for this.

But yes, in general, we want to log only violation events for the enforcement mode.
We will log everything, as logEvent does, for a "monitoring" mode (now called 'trace'), if we stille need this mode.

algomaster99 · 2024-10-08T16:32:47Z

track_syscalls/main.go

+	syscalls := make(map[string]map[int]bool)
+	setupAndRun(binaryPath, func(event ebpfEvent, stackTrace []uint64, objs *ebpfObjects) {
+		callerPackage := stackanalyzer.GetCallerPackage(stackTrace)
+		if callerPackage != "" {


In what case callerPackage is ""?

The callerPackage can be empty in cases where the package name doesn't have the prefix 'github.com'. The logic is implemented into stackanalyzer.GetCallerPackages()

In this current implementation, I was allowlisting (and also enforcing, in runEnforceMode) only third-party packages hosted on github. This implementation skips the generation and enforcement of the allowlist for packages from the Go runtime, the main package, and other local custom packages.

TODO: we should work on a more robust logic for package tracing. For example, third-party packages can be hosted elsewhere (e.g., k8s.io, google.golang.org, etc).

elsewhere (e.g., k8s.io, google.golang.org, etc).

Is there an exhausitve list? Could you host packages on custom URL and download go packages from there?

algomaster99 · 2024-10-08T16:35:41Z

track_syscalls/main.go

-				log.Println("Received signal, exiting..")
-				return
+func runBuildMode(binaryPath string) {
+	syscalls := make(map[string]map[int]bool)


I am bit confused about what this make function does. Could you explain? 😅

make in Go create a slice, a dynamic data structure.

We are creating a map of maps. The key of the outer map is the callerPackage name (string). For each callerPackage, we have an inner map, which the keys are syscall numbers (int). Each syscall number have an associated boolean value representing "allowed" or "denied". This structure allows for efficient allowlist check.

algomaster99 · 2024-10-08T16:37:06Z

track_syscalls/main.go

+	case "build":
+		runBuildMode(*binaryPath)


Let's rename this to augment because we take input as allowlist from capslock and we will augment it as we discussed here.

I agree and i like this new name.

algomaster99 · 2024-10-08T18:19:20Z

@carminecesarano I tried adding a test resource in order to test augment mode. The test resource lies in testpks/basiccgo. The allowlist.json is created using capslock. The augment mode should ideally add CAPABILITY_FILES to the allowlist and that is what we want to test for.
However, running sudo ./bpf_loader -binary $(realpath ../testpkgs/basiccgo/basiccgo) -mode build returns an empty allowlist.

{
  "dependencies": {}
}

What changes should I make to tracer to achieve this?

carminecesarano · 2024-10-09T16:41:58Z

@algomaster99 there is one more hard-coded parameter to set, which is target_comm in the eBPF program.

This is the name of the command to trace. In your case, it should be basiccgo. We should make this configurable from the CLI.

algomaster99 · 2024-10-09T17:47:27Z

Still unable to get a non-empty allowlist. Let's sit on Friday to pair program. I get some output from the eBPF program but nowhere it prints the stack trace that has go function for read system call.

…sh into syscall-enforcement

algomaster99 · 2024-10-20T16:41:37Z

The problems with test as of now:

Does not capture after "Tracking syscalls".
We need "sudo" to run the tests.

algomaster99 · 2024-10-21T10:06:33Z

I managed to solve 1 will push test soon :) The fix was putting time.Sleep(2 * time.Second) before executing basiccgo.

algomaster99 · 2024-10-21T10:12:01Z

Now we need CI

algomaster99 · 2024-10-21T11:44:58Z

We have a working CI now. I will fix the functionality for building allowlist and then merge.

algomaster99 · 2024-10-21T12:13:13Z

Gives a good failure message:

Expected output to contain Go caller function: executeMaliciousCGO, got 12:12:36 Tracking syscalls...
        
        12:12:38 Invoked syscall: 12	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60ea8b
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 158	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e602cb4
        0x1
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 21	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60f9ab
        0x7fc55e60883c
        0x40
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 257	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fb18
        0x7fc55e5f3982
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 262	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60f8de
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 17	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fb9e
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 10	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fd2b
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 3	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60f9db
        0x7fc55e5f3601
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
        
        12:12:38 Invoked syscall: 9	pid: 5069	comm: basiccgo
        12:12:38 Stack Trace:
        0x7fc55e60fcb7
        12:12:38 Go caller function: 
        12:12:38 Go caller package: 
FAIL
exit status 1
FAIL	github.com/chains-project/goleash/track_syscalls	2.007s

algomaster99 · 2024-10-24T09:01:54Z

@carminecesarano This is the structure of our local go package

├── basiccgo
│   ├── allowlist.json
│   ├── basiccgo
│   ├── go.mod
│   └── main.go
├── example.com
│   ├── filereader
│   │   ├── filereader.go
│   │   ├── hello.c
│   │   └── hello.h
│   └── go.mod
└── Makefile

go.mod file for module basiccgo looks like this now:

module basiccgo

go 1.23.2

require "example.com" v0.0.0
replace (
	"example.com" v0.0.0 => "../example.com"
)

…tions to get the Caller Package from the stack trace

algomaster99 · 2024-10-27T21:47:20Z

track_syscalls/main_test.go

+	StopTracer(tracer)
+
+	var expectedCallerFunction = "Go caller function: example.com/filereader.ExecuteMaliciousCGO"
+	var expectedCallerPackage = "Go caller package: example.com"


@carminecesarano should this not be "Go caller module: example.com"?

The package would be 'Go caller package: example.com/filreader".

added trace, build allowlist, and enforce modes

a3f0e82

algomaster99 requested changes Oct 8, 2024

View reviewed changes

test: add test resource for testing augment mode

dfdf52d

carminecesarano and others added 6 commits October 10, 2024 17:15

go module refactoring

a214663

Merge branch 'syscall-enforcement' of github.com:chains-project/golea…

65be221

…sh into syscall-enforcement

utils file refactored

174487b

coredns commit

505a606

refactor: encapsulate all args in struct

f1f4238

test: propose infrastructure for test

b89fedf

test: add test for build

ed5775c

chore: add workflow

ce01662

algomaster99 force-pushed the syscall-enforcement branch from f4ce2e6 to ce01662 Compare October 21, 2024 11:26

algomaster99 added 7 commits October 21, 2024 13:31

chore: install llvm-strip

24fd877

Fix syntax

9649af6

Fix syntax

0c4996f

Fix package name

457101e

Install llvm

9f33f63

Install eBPF

c97efcf

up

02d0bb0

algomaster99 added 2 commits October 21, 2024 13:45

Add go.mod

dc97ce4

Test if the exact function is being called

3d8fa17

Separate malicious CGO into a separate package

9db5d9c

carminecesarano and others added 9 commits October 27, 2024 17:45

modified basiccgo Makefile

90da01a

modified name Populate function

3acb045

added LoadModuleCache function for loading go.mod files, updated func…

bb47d23

…tions to get the Caller Package from the stack trace

added some dependencies

3157b02

modified basiccgo. Now it is a working example

66d4a30

removed unused files

db7a718

align workflow

b94e3a8

up

82a4556

Make test pass :)

09aca1c

algomaster99 reviewed Oct 27, 2024

View reviewed changes

algomaster99 mentioned this pull request Oct 28, 2024

example.com is considered as package and not module #3

Open

algomaster99 merged commit 1fcd1aa into main Oct 28, 2024
2 checks passed

algomaster99 deleted the syscall-enforcement branch October 28, 2024 15:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added trace, build allowlist, and enforce modes #2

Added trace, build allowlist, and enforce modes #2

carminecesarano commented Oct 7, 2024

algomaster99 left a comment

algomaster99 Oct 8, 2024

carminecesarano Oct 9, 2024

algomaster99 Oct 8, 2024

carminecesarano Oct 9, 2024

algomaster99 Oct 8, 2024

carminecesarano Oct 9, 2024

algomaster99 Oct 10, 2024 •

edited

Loading

algomaster99 Oct 8, 2024

carminecesarano Oct 9, 2024

algomaster99 Oct 8, 2024

carminecesarano Oct 9, 2024

algomaster99 commented Oct 8, 2024

carminecesarano commented Oct 9, 2024

algomaster99 commented Oct 9, 2024

algomaster99 commented Oct 20, 2024

algomaster99 commented Oct 21, 2024 •

edited

Loading

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 24, 2024

algomaster99 Oct 27, 2024 •

edited

Loading

Added trace, build allowlist, and enforce modes #2

Added trace, build allowlist, and enforce modes #2

Conversation

carminecesarano commented Oct 7, 2024

algomaster99 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

algomaster99 Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

algomaster99 commented Oct 8, 2024

carminecesarano commented Oct 9, 2024

algomaster99 commented Oct 9, 2024

algomaster99 commented Oct 20, 2024

algomaster99 commented Oct 21, 2024 • edited Loading

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 21, 2024

algomaster99 commented Oct 24, 2024

algomaster99 Oct 27, 2024 • edited Loading

Choose a reason for hiding this comment

algomaster99 Oct 10, 2024 •

edited

Loading

algomaster99 commented Oct 21, 2024 •

edited

Loading

algomaster99 Oct 27, 2024 •

edited

Loading