Glen Turner writes:
> "Albert D. Cahalan" wrote:
> I've written a fair bit of code to tune clusters and the problems
> weren't caused by so much by /proc values being in ASCII, as by the
>
> - lack of regularity in file formats
> - inconsistent use of "tag: data", lists of values, and
> files containing only one value.
Use of "tag: data" is bad because it encourages spelling "fixes"
and other stupid tweaks. Lists of values are great, as long as
they don't contain strings with ambiguous whitespace. Files with
only one value are slow.
> - no standard for repeated values, eg: multiple CPUs
> - no standard for nil values (some variables even overloading
> the zero value, so there is way to create an average for
> that variable).
> - unnecessary differences across architectures
> - inconsistent marking of decimal and hex values, and using
> leading-zero padding (which is interpreted as octal by many
> parsers)
All numbers should be leading-zero hex, with spaces used to pad lines
out to a power of two. :-) Then I could treat the data as binary.
> - lack of discipline and consistency in file and tag naming conventions
> - lack of a (user mode) data dictionary containing meta-data such as
> types, ranges and meanings of values. Without it, you can't reliabily
> plot arbitrary performance variables (is the value continuous or
> discrete?) or even create percentages.
> Performance in my code was limited by the inability to grab a
> whole lot of related variables at once[1]. I'd be interested in
> the profile of the "ps" code: is most of the time spent parsing
> ASCII, or stuffing about opening and closing a large number of
> files and coping with the inconsistent formats?
Can you suggest a good tool? I suppose kernel time is of interest,
along with user-space stack traces, etc.
> On the other use of /proc, using a binary format for setting
> and querying configuration variables would be more inconvenient
> than setting them through text. We'd see a proliferation of
> utilties that do nothing more than the present
> "echo 0 > /proc/sys/something". (These C utilities would also
> be slower, but that's hardly an advantage as configuration
> programs are run too infrequently for that argument to fly. [2])
The C utilities would be much faster. You can forget about the
time required to start them, since it only happens once.
The sysctl program, for example, can read in a whole file of
settings to load.
>> My fix would be to add the exact same binary files that
>> Solaris has and to make all non-process information invisible.
>
> Would this imply that "ps" and other performance utilities
> would have to carefully track the kernel sub-version?
The kernel version must be carefully checked already.
BTW, one can make all values be 64-bit right from the start.
> This could be a bit painful in a cluster, especially if the
> purpose of clustering is high uptime (eg: you'd be faced with
> having to reboot all the machines if a security fix came
> out for the procps package).
Security fixes for procps are very rare. There is no need to
run procps tools setuid -- if you do, STOP RIGHT NOW. It is
only overflows caused by data in /proc that are of concern.
Well, no change there. It happens with ASCII too.
>> Real code is C anyway.
>
> Unless you really care about performance. Then you'll find the
> tools are written in prototyping langauges so that you can hack
> together something to monitor a particular aspect of performance
> without the monitoring code taking all year to write.
Huh? Perl takes 10x as long to write and runs much slower.
> [2] A C (or any) program invoked from the shell is likely to take
> more resources than executing a built-in shell statement.
> Especially once the likelyhood of I/O is factored in.
This is why the sysctl program can read a file full of values.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Feb 29 2000 - 21:00:10 EST