Describe DESIGN (performance, filtering and (thread/memory) controllability tradeoffs

[matu3ba]

It is unclear, if 1. lldb tracing of the complete program state is being with according performance implications used or 2. something more lightweight akind to rr with ~1.3x time needs or 3. custom code generation or 4. custom debug tracer functionality is loaded into the memory address space of the process.

The other question would be how much compiletime or runtime filtering can be mode to get only the relevant results and/or things can be filtered by only collecting in one thread etc.

Thanks for your effort. I do very much like the control flow visualization, but I think the memory visualization is unfeasible for bigger structures.

[tyt2y3]

Thank you for the thread. I hope I am not getting your question wrong, but I think it is 1: we set software breakpoints using lldb on parts of the code that we want to trace, and just "pause and resume" as fast as possible. But actually the backend implementation can be changed, it's possible to have an implementation based on rr, with a different set of trade offs - e.g. rr is not truely parallel.

Compile-time filtering: it's quite fast right now, at least it didn't bother me to parallelize or optimize the static analysis stage. Because we access cargo (rust's package manager)'s metadata. Basically the purpose of the static-analysis stage is to categorize each symbol into either "user code", "vendored code" and "system" (standard library and OS).

Thank you. The control flow visualization is pretty scalable, we can handle trees say with 100k nodes. I agree with you the variables rendering is for "neat visualizations" and it can definitely chok on MBs of data. So we basically truncate (with ..) when there are too many items.

Hope this answers your questions. What's your imagination to what time-travel debugging can be? I do want to bring FireDBG to other languages and would love to collaborate on those.

One thing I am working on is to trace heap allocations and detect memory leaks.

[matu3ba]

But actually the backend implementation can be changed, it's possible to have an implementation based on rr, with a different set of trade offs - e.g. rr is not truely parallel

Yes, rr simulates threading and multiple processes in one process to get accurate performance counter timings for accurate reproducibility. This also rules out searching for tricky race conditions between threads.
Did you already play with the Rust rewrite of rr (https://github.com/sidkshatriya/rd) to show feasability?

What's your imagination to what time-travel debugging can be?

You asked for it, so here is my egg-laying wull milk pig with mind-blowing complexity and scope.

A from my viewpoint optimal system would be a graphical + text editor with shortcuts to serialize and deserialize 1. program control flow state by relative conditions to another (in what source file area which thread or process is), and 2. program state state conditions (relative or absolute program conditions) and ideally would generalize both to 3. arbitrary programs [in which for example kernel information like memory usage could be queried] for all introspection use cases (debugging, memcheck/validators, race condition checkers, tracers)
to have seperate overlays for debugging to reproduce the state of one program state condition within the other tool even under stochastic events.

As far as I understand, currently language semantics leak into debuggers besides debuggers/systems with repls can not be designed well for third party interactivity, which is why gdb and lldb do not expose pretty printer as c api or formatting library etc.

More ideally, the semantics of the program itself could be reasoned about, but this would require compiler support with complete enough memory models + simulator with debug api etc on top of that (say Miri) to initialize the simulator from arbitrary program state.

As of now, we neither have a cross-usable snapshotformat for programs (on the same platform) nor tooling to compare program states or traces.

Does any of this make sense to you?

[tyt2y3]

Thank you for your sharing. I wasn't aware of the rust port of rr yet, I'll see what I can do with it.

I tried thinking about the other points for a few days... happy holiday.. can you define the following terms, i.e. "conditions", "relative or absolute program conditions", "overlays"?

Regarding "semantics leak", I generally agree. e.g. While the standard library is not part of the compiler, it's generally considered part of the language from the point of view of the programmer.

[matu3ba]

Sorry for late reply. I got sick and wanted to finish another thing, which took longer and revealed weird Windows behavior on LoadLibrary (taking virtual memory, even in case of error during loading the library).

"conditions"

Pardon, I misspelled and meant https://en.wikipedia.org/wiki/Conditional_(computer_programming)

relative or absolute program conditions

Here I meant that the condition can either contain equations (</<=/==/>=/>) about variable states or program locations (think of the source file) or the execution unit (thread with threadrelation to source location where it originated or process origin) itself.
Threads and processes have always a tree relation, variable states are bits with type information (for specific or generalized types) and program locations are source locations with the overlay rule being applied.

overlays

The program source locations may contain generic code or may be used in multiple contexts, say in debugger or tracer and in different sessions one may want to compare (one of the worst experience is not being able to step back in debugger, so it is usually favorable to take a snapshot etc).
The idea of overlay was here to be able to combine program states and/or locations and/or execution contexts and/or tools, since I was not aware of a better name.

standard library

I was not speaking of the standard library, but an interpretation of the type information on the underlying bits getting leaked as those are necessary for pretty printing or doing any kind of semantic reasoning. Think of it as initializing the interpreter from defined execution timepoints from the compiled program to prevent waiting for the very slow interpreter for longer running programs.