[RFC PATCH v2 00/10] Rust PuzleFS filesystem driver

From: Ariel Miculas
Date: Wed Jul 26 2023 - 12:58:02 EST

Next message: Ariel Miculas: "[RFC PATCH v2 06/10] rust: file: pass the filesystem context to the open function"
Previous message: Ido Schimmel: "Re: [PATCH v1 01/11] selftests: forwarding: custom_multipath_hash.sh: add cleanup for SIGTERM sent by timeout"
Next in thread: Ariel Miculas: "[RFC PATCH v2 02/10] kernel: configs: enable rust samples in rust.config"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hello,

This is a proof of concept driver written for the PuzzleFS
next-generation container filesystem [1]. This patch series is based on
top of the puzzlefs_dependencies [4] branch, which contain the
backported filesystem abstractions of Wedson Almeida Filho [2][3] and
the additional third-party crates capnp [5] and hex [6]. The code is
also available on github [11]. I've adapted the user space puzzlefs code
[1] so that the puzzlefs kernel module could present the directory
hierarchy and implement the basic read functionality. For some
additional context, puzzlefs was started by Tycho Andersen and it's the
successor of atomfs. This FOSDEM presentation from 2019 [10] covers the
rationale for a new oci image format and presents a higher level
overview of our goals with puzzlefs.

Changes since v1:
* the main change is that I've replaced serde_cbor with Cap'n proto at
Kent Overstreet's suggestion; this simplifies the dependency tree of
puzzlefs and results in cleaner code
* I've also modified the filesystem abstractions to follow the latest
"Rust for Linux" best practices (in the puzzlefs_dependencies_v2
branch [4])

I've split the rest of the cover letter in following sections (using a
markdown style):
* Example: it describes a practical example of what was achieved
* Limitations: it presents the existing limitations of this POC
* Setup: it shows how to setup the necessary environment for testing the
puzzlefs driver

# Example
An example is provided below:

```
~ # cat /proc/filesystems | grep puzzlefs
nodev puzzlefs
~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="c4
3e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09" none /mnt
~ # ls -lR /mnt/
/mnt/:
total 0
drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-1
drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-2
drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-3
drwxr-xr-x 2 0 0 0 Jun 8 12:26 dir-4
-rw-r--r-- 1 0 0 0 Jun 8 12:26 file1
-rw-r--r-- 1 0 0 0 Jun 8 12:26 file2

/mnt/dir-1:
total 0

/mnt/dir-2:
total 0

/mnt/dir-3:
total 0

/mnt/dir-4:
total 0
~ # cat /mnt/file2
ana are mere bla bla bla
~ # wc /mnt/file1
202 202 5454 /mnt/file1
```

In this example, /home/puzzlefs_oci is the puzzlefs oci directory:
```
~ # ls -lR /home/puzzlefs_oci/
/home/puzzlefs_oci/:
total 8
drwxr-xr-x 3 1000 1000 0 Jul 26 14:37 blobs
-rw-r--r-- 1 1000 1000 267 Jul 26 14:37 index.json
-rw-r--r-- 1 1000 1000 37 Jul 26 14:37 oci-layout

/home/puzzlefs_oci/blobs:
total 0
drwxr-xr-x 2 1000 1000 0 Jul 26 14:37 sha256

/home/puzzlefs_oci/blobs/sha256:
total 16
-rw------- 1 1000 1000 272 Jul 26 14:37 c43e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09
-rw------- 1 1000 1000 5479 Jul 26 14:37 d86a87b19bd9a2fec0d31687c1d669cdb59e8fb579b0b2bfe354473699732bfb
-rw------- 1 1000 1000 752 Jul 26 14:37 fe6ee75ca93ded15da5f6e0abf335e6a4eb24277c59acd47282d8ed2475f3384
```

`c43e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09` is the puzzlefs image manifest
hash for the `first_try` tag:
```
❯ cat index.json | jq .
{
"schemaVersion": -1,
"manifests": [
{
"digest": "sha256:c43e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09",
"size": 272,
"media_type": "application/vnd.puzzlefs.image.rootfs.v1",
"annotations": {
"org.opencontainers.image.ref.name": "first_try"
}
}
],
"annotations": {}
}
```

I will describe how to build a puzzlefs image in step 5 of the `Setup` section.

# Limitations
One limitation is that the puzzlefs driver doesn't implement any lookup
functionality and instead it inserts every directory entry into the
dcache during init (see the `DCACHE_BASED` constant). This is similar to
how the sample `rust_fs` driver works, but the goal is to implement
proper lookup functions. However, more filesystem abstractions need to
be implemented before this can be achieved.

# Setup
My setup is based on Wedson's tutorial [8]. Next, I will describe the
necessary steps to build an initrd and run a custom kernel under qemu.

1. Get the code from the puzzlefs_rfc_v2 branch on github [11]

2. Follow the rust quickstart guide [9]

3. configure and build the kernel
```
$ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
$ make LLVM=1 -j$(nproc)
```

4. setup busybox
```
git clone git://git.busybox.net/busybox
cd busybox
make menuconfig # enable 'Build static binary' config
make
make install
```
This will create the `_install` directory with the rootfs inside it.

5. create a home directory in the rootfs and copy a puzzlefs oci
directory in home/puzzlefs_oci
To create a puzzlefs oci directory:
* clone the puzzlefs repository [1]

* run `make release`

* create a simple filesystem structure with a few directories and files
(I've created one at ../test-puzzlefs/simple_rootfs)

* build a puzzlefs oci image at
`~/work/busybox/_install/home/puzzlefs_oci` (replace this path with
your busybox path) with the tag `first_try`:
```
$ target/release/puzzlefs build ../test-puzzlefs/simple_rootfs \
~/work/busybox/_install/home/puzzlefs_oci first_try
```

* get first_try's image manifest from index.json (inside `puzzlefs_oci`)
```
$ cat index.json | jq . | grep digest
"digest": "sha256:c43e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09",
```

6. add the following 'init' script in the busybox rootfs (rootfs path
defaults to `./_install'):
```
#!/bin/sh
mount -t devtmpfs none /dev
mkdir -p /proc
mount -t proc none /proc

ifconfig lo up
udhcpc -i eth0

mkdir /mnt
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="c43e5ab9d0cee1dcfbf442d18023b34410de3deb0f6dbffcec72732b6830db09" \
none /mnt

setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
```

Make sure to replace the `image_manifest` with your own digest. This
init script will be passed to rdinit in the kernel command line.

7. generate the initramfs

Assuming busybox is in `~/work/busybox`:
```
cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
```
This will generate a compressed ramdisk image in
`~/work/busybox/ramdisk.img`.

8. run with qemu (this assumes the linux tree is at '../linux' and busybox
is at '../busybox'):
```
qemu-system-x86_64 \
-accel kvm \
-cpu host \
-m 4G \
-initrd ../busybox/ramdisk.img \
-kernel ../linux/arch/x86/boot/bzImage \
-nographic \
-append 'console=ttyS0 nokaslr debug rdinit=/init' \
-nic user,model=rtl8139 \
-no-reboot
```

9. Check whether puzzlefs has been successfully mounted
```
~ # mount | grep puzzlefs
none on /mnt type puzzlefs (rw,relatime)
~ # ls /mnt/
dir-1 dir-2 dir-3 dir-4 file1 file2
```

[1] https://github.com/project-machine/puzzlefs
[2] https://github.com/wedsonaf/linux/tree/fs
[3] https://github.com/Rust-for-Linux/linux/issues/1004
[4] https://github.com/ariel-miculas/linux/tree/puzzlefs_dependencies_v2
[5] https://docs.rs/capnp/latest/capnp/
[6] https://docs.rs/hex/0.4.3/hex/
[7] https://hackmd.io/@cyphar/ociv2-brainstorm
[8] https://www.youtube.com/watch?v=tPs1uRqOnlk
[9] https://docs.kernel.org/rust/quick-start.html
[10] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/
[11] https://github.com/ariel-miculas/linux/tree/puzzlefs_rfc_v2

Ariel Miculas (10):
samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
kernel: configs: enable rust samples in rust.config
rust: kernel: add an abstraction over vfsmount to allow cloning a new
private mount
rust: file: Add a new RegularFile newtype useful for reading files
samples: puzzlefs: add basic deserializing support for the puzzlefs
metadata
rust: file: pass the filesystem context to the open function
samples: puzzlefs: populate the directory entries with the inodes from
the puzzlefs metadata file
rust: puzzlefs: read the puzzlefs image manifest instead of an
individual metadata layer
rust: puzzlefs: add support for reading files
rust: puzzlefs: add oci_root_dir and image_manifest filesystem
parameters

kernel/configs/rust.config | 10 +
rust/alloc/vec/mod.rs | 17 +
rust/bindings/bindings_helper.h | 1 +
rust/helpers.c | 6 +
rust/kernel/error.rs | 6 +-
rust/kernel/file.rs | 146 +-
rust/kernel/fs.rs | 4 +-
rust/kernel/lib.rs | 1 +
rust/kernel/mount.rs | 71 +
samples/rust/Kconfig | 10 +
samples/rust/Makefile | 1 +
samples/rust/puzzle.rs | 5 +
samples/rust/puzzle/error.rs | 106 +
samples/rust/puzzle/inode.rs | 101 +
samples/rust/puzzle/oci.rs | 71 +
samples/rust/puzzle/types.rs | 340 ++
samples/rust/puzzle/types/manifest.capnp | 15 +
samples/rust/puzzle/types/manifest_capnp.rs | 773 ++++
samples/rust/puzzle/types/metadata.capnp | 69 +
samples/rust/puzzle/types/metadata_capnp.rs | 4458 +++++++++++++++++++
samples/rust/puzzlefs.rs | 220 +
samples/rust/rust_fs.rs | 3 +-
22 files changed, 6425 insertions(+), 9 deletions(-)
create mode 100644 rust/kernel/mount.rs
create mode 100644 samples/rust/puzzle.rs
create mode 100644 samples/rust/puzzle/error.rs
create mode 100644 samples/rust/puzzle/inode.rs
create mode 100644 samples/rust/puzzle/oci.rs
create mode 100644 samples/rust/puzzle/types.rs
create mode 100644 samples/rust/puzzle/types/manifest.capnp
create mode 100644 samples/rust/puzzle/types/manifest_capnp.rs
create mode 100644 samples/rust/puzzle/types/metadata.capnp
create mode 100644 samples/rust/puzzle/types/metadata_capnp.rs
create mode 100644 samples/rust/puzzlefs.rs

--
2.41.0

Next message: Ariel Miculas: "[RFC PATCH v2 06/10] rust: file: pass the filesystem context to the open function"
Previous message: Ido Schimmel: "Re: [PATCH v1 01/11] selftests: forwarding: custom_multipath_hash.sh: add cleanup for SIGTERM sent by timeout"
Next in thread: Ariel Miculas: "[RFC PATCH v2 02/10] kernel: configs: enable rust samples in rust.config"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]