systemtap 3.3 release

From: Frank Ch. Eigler
Date: Fri Jun 08 2018 - 18:35:48 EST


The SystemTap team announces release 3.3!

eBPF backend extensions, easier access to examples, adapting to
meltdown/spectre complications, real-time / high-cpu-count
concurrency fixes


= Where to get it

https://sourceware.org/systemtap/ - our project page
https://sourceware.org/systemtap/ftp/releases/systemtap-3.3.tar.gz
https://koji.fedoraproject.org/koji/packageinfo?packageID=615
git tag release-3.3 (commit 48867d1cface944)

There have been over 237 commits since the last release.
There have been over 19 bugs fixed / features added since the last release.


= How to build it

See the README and NEWS files at
https://sourceware.org/git/?p=systemtap.git;a=tree

Further information at https://sourceware.org/systemtap/wiki/


= SystemTap frontend (stap) changes

- The "stap --sysroot /PATH" option has received a revamp, so it
works much better against cross-compiled environments.

- A new "stap --example FOO.stp" mode searches the example scripts
distributed with systemtap for a file named FOO.stp, so its whole
path does not need to be typed in.


= SystemTap backend changes

- The eBPF backend now supports uprobes, perf counter, timer, and
tracepoint probes.

- The eBPF backend has learned to perform loops - at least in the
userspace "begin/end" probe contexts, so one can iterate across BPF
arrays for reporting. (The linux kernel eBPF interpreter precludes
loops and string processing.) It can also handle much larger probe
handler bodies, with a smarter register spiller/allocator.

- Systemtap's runtime has learned to deal with some of the collateral
damage from kernel hardening after meltdown/spectre, including more
pointer hiding and relocation. The kptr_restrict procfs flag is
forced on if running on a new enough kernel.

- Several low level locking-related fixes were added to the runtime
that used uprobes/tracepoint apis, in order to work more reliably on
real-time kernels and on high-cpu-count machines.


= SystemTap tapset changes

- Runtime/tapsets were ported to include up to kernel version 4.17.
(The syscall tapsets are broken on kernel 4.17-rc, and will be fixed
in a next release coming soon; PR23160.)

- Some MIPS support has been added.


= SystemTap sample scripts

All 178 examples can be found at https://sourceware.org/systemtap/examples/

- io_submit.stp has been optimized for larger systems

- new example capture_ssl_master_secrets.stp is just as naughty as it sounds


= Examples of tested kernel versions

2.6.32 (RHEL 6 x86 and x86_64)
3.10.0 (RHEL 7 x86_64)
4.16.5 (Fedora 27 x86_64)
4.18-rc0 (Fedora rawhide x86_64)


= Known issues with this release

- The syscall tapset is broken for kernels >= 4.17. Use the
kernel.trace("sys_enter") probe until we get this fixed. (PR23160)

- Some post-meltdown/spectre kernel versions have broken uprobes
(resulting in SIGILL in userspace programs) and kernel tracepoints.
Kernel fixes are underway. (RHBZ1579521)

- Some kernel crashes continue to be reported when a script probes
broad kernel function wildcards. (PR2725)

- An upstream kernel commit #2062afb4f804a put "-fno-var-tracking-assignments"
into KCFLAGS, dramatically reducing debuginfo quality, which can cause
debuginfo failures. The simplest fix is to erase, excise, nay, eradicate
this line from the top level linux Makefile:

KBUILD_CFLAGS += $(call cc-option, -fno-var-tracking-assignments)


= Coming soon

- http and systemtap coming together, like peanut butter and chocolate


= Contributors for this release

Aaron Merey, *Aryeh Weinreb, *Bernhard Wiedemann, David Smith, Frank
Ch. Eigler, *Gustavo Moreira, *Igor Gnatenko, *Iryna Shcherbina, *Jafeer
Uddin, Jeff Moyer, *Lukas Herbolt, Mark Wielaard, Martin Cermak, *Petr
Viktorin, Serhei Makarov, Stan Cox, Stefan Hajnoczi, Timo Juhani
Lindfors, Victor Kamensky

Special thanks to new contributors, marked with '*' above.


= Bugs fixed for this release <https://sourceware.org/PR#####>

21107 a few more access_ok tweaks needed
21890 bpf uprobes support
22004 dyninst does not handle R_*_IRELATIV in .rela.plt
22141 The RPM specfile needs an update handling the bpf bits
22248 failure processing linux-vdso64.so.1
22311 bpf: drop the copy of the bpf map logic & snapshot-based pre-post begin {} synch
22313 bpf: exit-state checking prologue
22314 bpf: add support for uprobes, uretprobe and tracepoint events
22323 bpf: format string tags appearing in output when wildcards are used
22327 the loadavg tapset no longer works on recent kernels
22328 bpf: add timer probes
22462 quoted include path
22536 Add shorthand option --bpf for --runtime=bpf
22551 on rawhide, we're getting a compile error that init_timer() doesn't exist
22695 "make rpm" broken by commit 3128ca27f67476fdd5f26a44bc3809fa8396e749
22698 bpf translator: add support for array $context variables
22822 bpf translator lacks register spilling
22857 vfs.do_mpage_readpage doesn't work as expected on ext4 filesystems
23226 stap foo.stp should search a path for sample foo.stp