"A month of sundays ago Ricky Beam wrote:"
> On Thu, 24 Feb 2000, Peter T. Breuer wrote:
> >> procfs is for process information (I'll be playing the part of purist today.)
> >I disagree. (a) humans are processes. (b), what's in a name? /proc has
> >come to mean "the place where the kernel outputs most of its info". Fine.
>
> No, humans are people. :-) Humans are self adaptive where process aren't
> (unless you've got some AI the rest of us don't.)
I meant that to a computer, they're much of a muchness. Human,
peripheral, what they heck. They have their good and bad points.
People are processes not spawned by the kernel :-). Anyway, amusing
as it may be, it's not a weighty or telling insight. It's only
use is to emphaise that both get a lot out of the kernel info produced
in proc, either directly or indirectly.
You now wish to separate this valuable source of info into either
a) info about procs
b) info for procs (/people)
I have nothing against it. It just doesn't seem worth the effort yet to me.
More dubiously, you seem also to be saying that you want to
make proc into a source of binary info only, so as to remove
inefficiencies connected with writing and reading back the same info.
Sorry. I disagree. You base your premise on "only processes read proc".
Well, tough, if you were worried about that you wouldn't ever let info
out of the kernel into user space in the first place. Why, it's just
copying info from one buffer to another and back again!
You are perhaps not aware of how important a source of information proc
is to me personally! I read it. I read it every time I want to see
what's going on, really. I put code in that prints real time info on
demand in proc. I even use it as a control point. It's convenient,
because _I_can_see_the_effect_of_my_action_immediately. It's
interactive. Printks don't compare.
I contrast, I have ever looked at the process info. You can throw that
away as far as I care. It's only use to me is that killall traverses it
while searching for the target I told it to look for.
> >the information has to be somewhere. How does being in proc in itself
> >somehow make the information less secure than in a file somewhere else?
>
> And if this untrusted thing has a buffer overflow or other flaw making it
You get a kernel oops. Anyway, there aren't any buffers in proc. That
was a valid complaint of yours. That's why it's "inefficient". Data
is recalculated at every access. How is this different from a buffer
overflow in sysctl, anyway?
> i.e. a procfs_chroot()? Linus wants all the sysctl() stuff to be a file
> system tree. That's a reasonable thing to do, but I don't want it attached
> to a static point in the file system making it difficult to cage processes.
Mmmm .. don't get it. Explain it to me later.
> At the core of this issue is the necessity of /proc for such everyday
> ordinary things as "how many cpu's are in this thing?", for example.
That's fair and appropriåte use. Slowly changing status info. EIther
count once at startup and print on request, or check at every access :-).
> There are far too many things pushed into /proc out of convenience without
> regard to fast, tiny, efficient non-fs access to them. Also, people
An inefficient /proc is about as harmful as bad cornering ability on a
panzer tank. The point is that it does what it is supposed to do, and
does it well. Speeding up the panzer's cornering by taking off the
armor cladding wouldn't help! If you want a binary interface, put
in a char device, or some such thing.
> quickly throw together a bunch of sprintf()s to hand back some status
> or debuging information without spewing stuff via printk(). As a debug
That's because you can control when you get info from proc.
> tool, that's perfectly ok -- I do it myself. But the fact that people's
> debugging tools become the sole point of inforamtion access is a bad thing.
They set up the debugging tool precisely because it was the way to
get good and useful information on demand. Obviously, it continues to be that
when they finished debugging. You benefit from the work they put into
debugging if you get it preserved as a proc interface in the end!
> As I said originally, I'd have to kill too many people to reverse this
> evolution, and it would do little to it from happening again. All I can
> hope to do is stop people from making it the sole interface for information.
> And to stop them from dumping everything in sight as english text.
Why not? You beef is that processes read the text instead of doing some
weird ioctls and sysctls, not that the text is there.
> Not at all. Just because YOU and I can write text parsers doesn't mean
> the entire world can. There are hunreds of examples of text parsing gone
> wrong throughout history. I don't care about the number of papers, books,
> classes, or monkeys you've taught to do text parsing. There's a reason
> lex and yacc exist and it's not because people are lazy. (They are, but
> that's not the point.)
Well, lex and yacc have always been somewhat obscurantist tools (but
strangely, the conditions for which yacc was designed, while extinct
outside the kernel, are still extant within it: guaranteed low memory
requirements. Hmm .. large code size is a buster for yacc in kernel,
though). I could probably whip up a programmable interpreted parser
in the kernel if people really wanted. So you could paste in the
attributed grammar spec as one argument and the string to be parsed
as the other, and get back the spec'd attribute. A sort of super
printk. The question is what you do on error :-).
> >consumption. Proc's job is to facilitate my gathering of the information
> >I need. It does that in the classical spirit of language design, by
> >providing the information in convenient _readable_ formats. It's just a
> >lot of ascii info. Just like configuration files. Are you arguning for
> >a binary registry?
>
> Let's take, for example, /proc/net/dev. You can 'cat' it and get all the
> numbers and things you want/need right there. End of rant. However, the
> instant you run 'netstat -i' you're wasting your time... netstat reads in
> the text, converts it to binary numbers, and prints it back out as text
> in a different format. You could argue netstat should skip the conversion
> and play with the text from dev, but the point is it (and almost everything
> else pulling data from /proc) DOES NOT DO THAT.
But that data came from the kernel, went to netstat, then went back
into the kernel via printf and the tty just to be put on your screen.
It already went back and forth a few times, never mind the
str->bin->str bit in netstat itself. The bit you're complaining about is
(most probably) linear complexity. It often happens whenever any two
processes commuicate, whether one of them is human or not. They
traslate between the interface format and their internal format.
> So, YES, a human will, in most cases, eventually, somewhere down the line,
> see the information that came from /proc. However, if the application
> accessing /proc does any work (e.g. math) on the data (such as printing
> an IP address in deciaml dot-quad format) then there was no point of
> originally presenting the application with human readable text.
If it does any real work, then that eclipses the work in the data
conversion. If it does no real work, then the whole thing is
lightweight,
> >You don't get the argument. Language design. You can write your code in
> >assembler, or in C, or in Python. /proc is there to provide the
> >interface that allows us to read the data with the minimum or
>
> 'us' is the program reading from /proc, not the human setting at the console.
> ...
I disagree. If I had to use a tool to read the files in proc, almost
all its value would disappear for me. It would cease to be a convenience.
> >effort appropriate to a high-level interfaces: humans and
> >their scripts or minor programs. It's there to make life easy for US.
>
> The PROGRAM is there to make it easy for 'us' to understand what the
The zillions of programs you want to interpose are unnecessary. By
all means add some to pretty up the output, or repretty it. But you
didn't really need any in the first place. Most proc output is pretty
readable by those humans interested in it.
> computer understands. By passing human centric information between
> computer programs, you're creating extra (unnecessary) work.
For whom? The computer? Fine! That's what I want.
> This arguement would then become one of "doing in kernel space what belongs
> in user space." The kernel's job is not to interact with humans. Humans
> interact with programs which interact with, among other things, the kernel.
> (When's the last time you keyed in a SCSI "START STOP UNIT" command to
> eject a disk?)
Mmm ...pass. Your argument here is that the kernel must burrow deeper
into anonimity to protect itself from those awful humans. MAKE them
use tools to talk to it. Why? We don't do it so very often. And the
kernel doesn't have to go to much effort to talk to us that way. No
more than it put into producing printks, anyway.
> >Why should I care if it takes 10ms or 20ms to cat a file from proc?
>
> (dammit) That is the most damnable statement anyone in the computer science
> world can say. It's attitudes like this that will require a 500GHz processor
> to run "hello world" in a few years.
Sorry, this argument has always been wrong. Don't worry about the speed
first. Chip density doubles every 18 months. That's 30% shorter
traces every 18 months, plus other speedups. The parsers I wrote in
1990 were too fast then (8MHz). Nowadays, the same parse code will run
approximately 100 times faster. And parsing is trivial in constrast to the
cost of lexing (those silly huge state machines). And that in turn is
trivial compared to the cost of actually doing something. You are
arguing about next to nothing, which is not to say that the argument
is without merit!
> "The processor is fast enough; we don't need to make it efficient."
>
> As I said before, if that is your attitude then kindly step away from the
> keyboard and never (ever) touch kernel code again. (already, so toned it
> down this time.)
However, the statement is true for the application you are considering:
/proc outputs.
> >??? No it isn't. This is called parsing. It's easy.
>
> And everytime someone changes it ever so slightly -- almost unnoticed by a
> human -- the parser fails to process it. If you can transfer the adaptive
No, parsers are as good at being forgiving as are humans.
> parsing capabiities of a human into a C/C++/perl/sed/whatever application,
> then be my guest -- just let me know how many hard drives I'll need to
> hold it. (I know some image processing people who would make a solid gold
Zero. Just be sure that you write the grammar nicely and ambiguously.
What's wrong with
TAGA A\x TAGB B\y | TAGB B\y TAGA A\x
?? Or ?* { TAGA A\x , TAGB B\y } ?*
(to invent a bit of notation for the construct). Anyway, the proper
cure is to beat the hell out of the guy who changed it and send a patch
in to Linus to change it back.
> >Embedded systems don't, by definition, have humans running them. And I
> >presume you in particular have no idea at all if your embedded system
> >is working correctly :-(.
>
> You'd be surprised. Granted, most of them don't have a human on a keyboard
> standing directly over them -- there's a master control station where the
> person sets with the keyboard to montitor >1 systems. And yes, I do know
> if my embedded systems are doing their job(s)... and I do it without spewing
I doubt it. If you could predict the failure modes you wouldn't have to
have humans watch out for them!
> volumes of human readable text.
You can only prepare traps for the situations you can predict. A human
has to read the information in order to spot the unexpected.
> >> The human brain can parse text very effectively. Computers don't! Having
> >
> >Computers are faster parsers. Perhaps you'd like the speed figures for
>
> Did I say anything about speed? NO. Computers do exactly what you tell
You said efficiently, and you've been using that word all along in a way
that I took to mean speed. Are you saying that you object to a computer
doing something TWICE, rather tha SLOW? Bear in mind that doing it
twice may be the fastest/best way to go. We just had that discussion
over boolean or. Apparently it pays to get both branches executed
speculatively.
> them to do in the manner in which you've instucted them to do so. That is
> all. If they haven't been programmed to handle a slight change of their
> input, then they fail. You can completely reformat the input and a human
That's why humans need to look at the diagnostics.
> You know how to parse text; I know how to parse text; I cannot say the same
> about the rest of the world. (And based on my former teaching experience,
> I'm taking the safe route and saying they cannot.)
The rest of the world doesn't know how to click a mouse, relatively
speaking.
> And adding "fast" and/or "efficient" means your code is highly specific to
Not necessarily. Some parsing techniques are both fast and forgiving.
Yes, backtracks generally cost, the same way that mistaken branches
cost in microcode. But the architecture of modern computers may make
that irrelevant too. I am refereeing more and more papers on quantum
computing these days, and the semantics are _not_ sequential.
> it's input. I've written a number of data file analysis tools. Their parsing
> of text is _extremely_ fast but completely intolerant of changes to the
> input.
I can imagine. But they're also extremely fast to change when the
format changes!
> >Ifconfig doesn't fail. netstat might. mount still works. Modutils still
> >work (apart from lsmod).
>
> - ifconfig by itself will not work -- it cannot find any devices to lookup.
Mine did. Sorry, but I tried it. ifconfig eth0 gave me the data.
> - netstat will fail as it reformats data from /proc/net/*.
Yes.
> - mount consults /proc/filesystems for a list to try.
Sorry, but I was able to do mount /proc just fine!
> - rmmod (-a) looks at /proc/modules to see if the module is loaded (and to
insmod worked fine. Didn't try rmmod. I think. I know I checked that
lsmod would fail.
> find the modules to purge.) lsmod just puts a header on /proc/modules.
> - free will not work as it reformats /proc/meminfo
Didn't try. I expect so.
> - sysconf(_SC_NPROCESSORS_ONLN) will fail as it counts /proc/cpuinfo.
> - some X servers will fail as they look in /proc/pci (or /proc/bus/pci)
When? At startx? I was running X when I umounted proc. and I didn't
die. I had to kill a couple of tasks in proc first ... I don't recall
which.
> >lmsensors won't work, by definition (In case you are wondering, yes, I am
> >typing this mail with /proc unmounted ;-).
>
> I didn't say the machine would crash :-)
Well, you pretty nearly did.
> >that is not there is that there is no buffer in the interface between
> >VFS and the registered proc_info (or whatever) functions. Thus the
> >proc codes have to recalculate their data every time they are asked for
> >a bit of it, because they don't know how else to get to the wanted bit.
>
> And generally speaking, they shouldn't have to. The case could be argued that
> drivers with a specified size should be able to seek to any point in their
> data, but there are problems with that. (The entire notion of file sizes
> in /proc is a grey area anyway.)
These files are really streams of data. They might as well be pipes
rather than files, except that there is no practical difference ...
> >I sort of agree. Is it open and close that are the markers
> >you need? You need to maintain "process x has read up to here". I.e.
> >you need to maintain a page of output until all its current readers
>
> I would say "reader_s_"... there will only be one fd attached to this
> reading instance. Ok, maybe there'd be dup()s of it? How do duplicated
> fd's get handled?
Good question, Moriarty. Magically. Somewhere. There has to be a
reference count. Oh, anyway, the buffer we write into won't go
away until its readers have left it, since it was handed down by the
VFS.
> While this is certainly "the right way"... now may not be the best time to
> go break procfs... softnet has made a big enough mess to cleanup. Altering
> the handling of procfs will potentially break _alot_ of code.
>
> I'll let Linus have the final word on this... this is gonna be an ISDN
> sized patch <grin>
>
Peter
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Feb 29 2000 - 21:00:11 EST