eBPF stack trace trie design

[chadbrewbaker]

https://github.com/iovisor/bcc/blob/master/tools/profile.py (what Pyroscope uses under the hood) is a far from optimal logging format for stack traces. I have a design in mind for a binary logger to save network bandwidth when exporting traces.

The logger would emit references to immutable trie pointers. When done, or when the trie overfills the memory limit - it would dump the trie tagged with the memory pointers.

Does this sound feasible? A more compressed version would have base64 counters on each trie node.

[chadbrewbaker]

You could probably do something equivalent to https://engineering.fb.com/2018/12/19/core-data/zstandard/ too when using the same workload over and over. Have a stock trie you pre-load that is already trained with your data layout - then you only need to give a root pointer address and serialize the parts of the trie that grow from it.