Re: File system for scratch space (in HPC cluster)

[Paul Menzel]

Dear Andreas,


On 2019-10-24 19:51, Andreas Dilger wrote:
> On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@xxxxxxxxxxxxx> 
> wrote:

>> In our cluster, we offer scratch space for temporary files. As 
>> these files are temporary, we do not need any safety requirements
>> â especially not those when the system crashes or shuts down. So
>> no `sync` is for example needed.
>> 
>> Are there file systems catering to this need? I couldnât find any? 
>> Maybe I missed some options for existing file systems.
> 
> How big do you need the scratch filesystem to be?  Is it local to
> the node or does it need to be shared between nodes?

In this case local.

> If it needs to be large and shared between nodes then Lustre is 
> typically used for this.  If it is local and relatively small you 
> could consider using tmpfs backed by swab on an NVMe flash device 
> (M.2 or U.2, Optane if you can afford it) inside the node.
> 
> That way you get RAM-like performance for many files, with a larger 
> capacity than RAM when needed (tmpfs can use swap).
> 
> You might consider to mount a new tmpfs filesystem per job (no 
> formatting is needed for tmpfs), and then unmount it when the job is 
> done, so that the old files are automatically cleaned up.
That is a good idea, but probably not practical for 10 TB. Out of
curiosity, what is the limit for ârelatively smallâ in your
experience?


Kind regards,

Paul

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature