system call tracing/auditing?

Anil Somayaji (
03 Dec 1996 17:42:02 -0500


Hello everyone!

I have a straightforward problem: I need to be able to record the
system calls made by a process and its children. I do not need (most)
of the arguments to the system calls, but I do need their relative
order. The method needs to be robust and fast, because I want to be
able to do this in a production environment, on a heavily loaded

Up to this point, I've made do with strace. Now, strace is a
beautiful program, but it has two flaws from my perspective: it is
slow, and it sometimes breaks (i.e. the traced program freezes, or it
outright crashes, when the untraced version exhibits no problems).

Now, it may be possible to figure out why strace sometimes breaks, but
fixing that won't address the speed issue. So, I believe I need some
sort of kernel-level system call tracing.

I believe some people were working on a security auditing package that
might provide the kind of data I'm looking for; does anybody know
whether any progress has been made on that project?

In any case, I have been thinking about how to implement a flexible,
robust, and efficient solution, and after some discussion I've had the
following thoughts on possible approaches.

First, you would need to add a system call that would enable and
disable tracing for a process and its children, and you would have to
modify the task structure to contain a flag concerning system-call
tracing. I'm not that familiar with the kernel sources, but from what
I can tell both of these changes are not too hard to do.

Now, the trick is this: how do you get the data out of kernel space in
an efficient manner? One way would be to use the standard kernel
logging facilities. Now, in order to minimize the costs, you would
want to buffer the system call information, so you only end up making
a logging call once every 100 system calls, for example. (Does the
Linux kernel already do this?)

Another approach would be to create some sort of procfs interface,
where a random user-space process could start reading a pseudo-file
(one for each process), and the kernel would report system calls
through this pseudo-file as long as it was open. Additional
information could be provided during forks, allowing the user-space
monitoring program to know which new pseudo-files it should be

The kernel could dump information directly into the memory space of a
process, or maybe it could just write directly to a file.

I've started working on an implementation using the standard logging
facilities - it seems like the easiest approach. (I haven't really
thought about the other options.) Before I really work on getting
this to work, though, I thought I would send out this message and see
whether anyone has any suggestions.

So, has anyone already solved this problem? Should I instead play
with strace, maybe stripping it down to its bare essentials? Am I
overlooking some potentially serious problems?


- --
Anil Somayaji (

Version: 2.6.2