Re: OS Masters

From: Jesse Pollard (pollard@tomcat.admin.navo.hpc.mil)
Date: Wed Apr 19 2000 - 16:18:34 EST


Mel <mel@csn.ul.ie>:
> Hello,
>
> My name is Mel Gorman. I'm currently studying final year computer systems
> in University of Limerick, Ireland. I have the opportunity to study a
> masters (2 years) in the college next year on operating systems and the
> direction of the masters is largely up to myself. Ideally, I will be using
> Linux as a base. Before I start though, I should mention though that;
>
> 1. I'm not a currently a kernel developer and have a bit to learn
> before I could be
> 2. I won't be starting work on this for a while (if at all if the Masters
> falls through and I can't find the time to work on it anyway)
> 3. I'm not fully up to date with current development because final year of
> college doesn't allow reading hundreds of emails a day.
> 4. I've subscribed to the list for the moment until I get some information
> or get laughed off the list
> 5. I am wearing a BIG kernel newbie stamp on my forehead that shields me
> from flames. Be nice and remember that even looking at kernel development
> is daunting at first and the volume of information is hard to absorb so
> try and keep snide comments about clueless newbie to a minimum please.
>
> I took a look through the FAQ and the archives but didn't see much in the
> line of what needs to be done next. I'm looking for a list or even
> pointers to discussions on what needs to be developed in the kernel that
> would be long term and would be suitalbe for a masters - muliple goals are
> welcome. Of the top of my head and from asking my local LUG, I got

First, determine the size of the project vs the available time .....;-)
I base that on:

   short-term: 3-5 months
   medium-term: 4-8 months (when project might take 4 months but could drag
                            on as long as 8 months)
   long-term: 7-24 months
   very-long term: anything else

> 1. Research OOM issues and ways of resolving it possibly based on
> profiling individual users and using system metrics as a limit. IRCC,
> there isn't a way implemented yet that everyone is happy with

(this would be my preference)

This is a limited sized project in that a known (relatively limited) size
amount of kernel code is affected. There are sevral areas that can be
looked at as part of the project, or as additional projects:

a. per-user memory resource limits or memory quota controls
        This can be used to control overcommitting memory. It can also be
        used to aid in optimizing memory usage. Base needs are a way for
        job start (login/cron/at) to set a user limit; consistant process
        memory usage accounting (COW handling, zero page allocation, ...);
        memory resource accounting logs (per process).
b. scheduling optimization - using memory accounting to aid in page fault
        optimization and CPU scheduling.
c. Resource accounting to prevent overloading the system with users
        (login/cron/at activation control feedback when insufficient
        resources are available for a new user).
d. potential expansion of fair-share schedulting to include memory parameters

items b,c,and d depend on what is provided by item a.
(a) should be a short-term project, b,c could be a short or medium term project
and d should be a short-term project.

> 2. POSIX compliance. Not all of Linux is POSIX complient. Identify the key
> areas and update them

This is a moving target since much of the code keeps being updated. I place
this as a medium to long-term project.

> 3. Optimize the dynamic library load for large numbers of libraries ro
> encourage code reuse. Establish a metic loader for judging when static
> executables should be used instead of dynamic

no opinions here

> 4. Look into making Linux a distributed system. Provide a
> kernel/userland/some combination of easily distributing tasks across a
> multiple number of machines whether task farming or whatever. Possibly
> even distributing scheduling and memory management across a number of
> machines rather than one machine. Note, this is not RPC, ideally, userland
> would barely be aware of the distribution

There have been some patches in the past for distributed shared memory that
can relate to this. This too is a long term project

> 5. Develop an XML file systems that used a compression scheme that
> made the assumption that all files stored are XML files and provide some
> support kernel side for quickly parsing them. Probably implement as an
> extenstion to an existing filesystem using a mount switch to turn on
> compression.

(I assume this doesn't correspond to the eXtended Markup Language for web
use). This could refer to a compressed file systems with decompression
done on a file-by-file basis on open. This could also be the basis for
a heirarchical filesystem with 1) data compressed at one stage; 2) compressed
data copied to offline storage with automatic retrieval/decompression on open.
(another long term project, where (1) might be a medium-term project.

> 6. Strip down the kernel to act as only a firewall/router making it as
> lightweight as possible. Would allow a headless box to be turned into a
> firewall or possibly a router easily.

There is a project for this already.

> 7. Someone suggested implemented the BeOS filesystem to have a multimedia
> ready file system for Linux - I'm not definite if this has been done
> already or not

Don't know anything here.

> So, if anyone has suggestions on things I could start researching into and
> trying to implement, I'd highly welcome them as well as comments on the
> above idea.. Bear in mind, I'll have to get my degree first (2 months)
> before I get to look in detail but I'm starting discussions with my
> college now on what could be researched so any information you can provide
> is helpful.

There is also a former project for a userfs filesystem interface to allow
the actual file system code to be implemented in a user space daemon. This
could be used to prototype a number of additional file systems without
having to modify the kernel each time.
(I think this could be a medium term project)

> Thanks for listening - apologies of this is noise

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sun Apr 23 2000 - 21:00:16 EST