HW7: Pantry

Part 0: Formatting and mounting disks (0 points)

A loop device is a pseudo-device that makes a file accessible as a block device. Files of this kind are often used for CD ISO images. Mounting a file containing a file system via such a loop mount makes the files within that file system accessible.

Tasks: ext2

  1. Create a loop device, build & mount an ext2 filesystem, and try creating directories and files. Below is a sample session you can follow. It goes without saying that you need to understand what’s going on at each step. Look at the man pages. Google stuff. You may need to boot into the LTS kernel for this to work.

    jae@vm04 ~/tmp $ sudo su
    vm04 /home/jae/tmp # dd if=/dev/zero of=./ext2.img bs=1024 count=100
    100+0 records in
    100+0 records out
    102400 bytes (102 kB) copied, 0.000626388 s, 163 MB/s
    vm04 /home/jae/tmp # losetup --find --show ext2.img
    vm04 /home/jae/tmp # mkfs -t ext2 /dev/loop0
    mke2fs 1.42.9 (28-Dec-2013)
    Filesystem label=
    OS type: Linux
    Block size=1024 (log=0)
    Fragment size=1024 (log=0)
    Stride=0 blocks, Stripe width=0 blocks
    16 inodes, 100 blocks
    5 blocks (5.00%) reserved for the super user
    First data block=1
    1 block group
    8192 blocks per group, 8192 fragments per group
    16 inodes per group
    Allocating group tables: done
    Writing inode tables: done
    Writing superblocks and filesystem accounting information: done
    vm04 /home/jae/tmp # mkdir mnt
    vm04 /home/jae/tmp # mount /dev/loop0 ./mnt
    vm04 /home/jae/tmp # cd mnt
    vm04 /home/jae/tmp/mnt # ll
    total 17
    drwxr-xr-x 3 root root   1024 Apr 20 11:19 ./
    drwxr-xr-x 5 jae  users  4096 Apr 20 11:20 ../
    drwx------ 2 root root  12288 Apr 20 11:19 lost+found/
    vm04 /home/jae/tmp/mnt # mkdir sub2
    vm04 /home/jae/tmp/mnt # ll
    total 18
    drwxr-xr-x 4 root root   1024 Apr 20 11:26 ./
    drwxr-xr-x 5 jae  users  4096 Apr 20 11:20 ../
    drwx------ 2 root root  12288 Apr 20 11:19 lost+found/
    drwxr-xr-x 2 root root   1024 Apr 20 11:26 sub2/
    vm04 /home/jae/tmp/mnt # cd sub2
    vm04 /home/jae/tmp/mnt/sub2 # ll
    total 2
    drwxr-xr-x 2 root root 1024 Apr 20 11:26 ./
    drwxr-xr-x 4 root root 1024 Apr 20 11:26 ../
    vm04 /home/jae/tmp/mnt/sub2 # mkdir sub2.1
    vm04 /home/jae/tmp/mnt/sub2 # ll
    total 3
    drwxr-xr-x 3 root root 1024 Apr 20 11:27 ./
    drwxr-xr-x 4 root root 1024 Apr 20 11:26 ../
    drwxr-xr-x 2 root root 1024 Apr 20 11:27 sub2.1/
    vm04 /home/jae/tmp/mnt/sub2 # touch file2.1
    vm04 /home/jae/tmp/mnt/sub2 # ll
    total 3
    drwxr-xr-x 3 root root 1024 Apr 20 11:28 ./
    drwxr-xr-x 4 root root 1024 Apr 20 11:26 ../
    -rw-r--r-- 1 root root    0 Apr 20 11:28 file2.1
    drwxr-xr-x 2 root root 1024 Apr 20 11:27 sub2.1/
    vm04 /home/jae/tmp/mnt/sub2 # cd ../../
    vm04 /home/jae/tmp # umount mnt
    vm04 /home/jae/tmp # losetup --find
    vm04 /home/jae/tmp # losetup --detach /dev/loop0
    vm04 /home/jae/tmp # losetup --find
    vm04 /home/jae/tmp # ll mnt/
    total 8
    drwxr-xr-x 2 root root  4096 Apr 20 11:20 ./
    drwxr-xr-x 5 jae  users 4096 Apr 20 11:20 ../
    vm04 /home/jae/tmp # exit
    jae@vm04 ~/tmp $
  2. In the sample session shown above, files and directories are created. Make sure you see the number of links each file or directory has, and make sure you understand why.

    Also try creating some hard links and symlinks. Make sure you understand how they affect the link counts.

  3. Assuming you built your 4.4.50-UNI kernel with a reduced .config using make localmodconfig, you won’t be able to create and mount a loop device. Identify the kernel config option(s) required for creating and mounting loop devices. Rebuild your 4.4.50-UNI kernel and try running through the session again.

Tasks: PantryFS

In this part, we will mount a loop device and format it as PantryFS. Then, we’ll use the compiled solution code (pantry.ko) to read our newly formatted disk.

  1. Create a disk image and assign it to a loop device:

    dd bs=4096 count=200 if=/dev/zero of=~/pantry_disk.img
    losetup --find --show ~/pantry_disk.img
  2. Format the disk as PantryFS.

  3. Mount the disk at /mnt/pantry using the reference implementation:

    mkdir /mnt/pantry
    insmod pantry.ko
    mount -t pantryfs /dev/loopN /mnt/pantry
  4. Explore the newly created filesystem. Edit hello.txt and create some new files.


Part 1: Creating additional files upon formatting (30 points)

The formatting utility creates the new filesystem’s root directory and places hello.txt in that directory. In this part, we will create another directory and file.

PantryFS Specification

View slides


  1. Read the provided formatting utility, format_disk_as_pantryfs.c. Make sure you understand the on-disk format and what each line contributes toward creating the filesystem. Test your understanding by thinking about the following questions:

  2. Extend the program to create a subdirectory called members. The directory should contain a single file, names.txt, that lists the names of your team members.

  3. Create and format a new disk using your modified program. Use the reference implementation, pantry.ko, to verify that the new file and directory were created correctly. You can use the stat command to see the size, inode number, and other properties.


Part 2: The filesystem awakens (50 points)

The rest of this assignment is structured to guide you while implementing your filesystem driver. Each part represents a milestone toward a working driver, forcing you to develop incrementally. It also allows the graders to award partial credit if you don’t get the whole thing working in the end.

In this part, we will begin by writing the code that mounts disks. Later, we’ll add the ability to read files, modify existing files, create new files, and delete them. There is an optional part that adds support for creating and removing directories.

Don’t worry about concurrency in your filesystem driver. That is, you may assume that each PantryFS disk will only have one thread of one program performing operations at any given time.

You are welcome to search the web for info. The linux-4.4.50/Documentation directory contains a wealth of information as well. Here are some resources that might be useful:

  1. LKD chapter 13

  2. LKD chapter 14: pages 289 - 294

  3. Page cache overview slides

  4. Linux Kernel Internals chapter 3 (Virtual Filesystem)

  5. Documentation/filesystems/vfs.txt

  6. Linux VFS tutorial at lwn.net


  1. Read the skeleton code starting from its entry point, pantryfs_init(). This module is written so that it can be loaded simultaneously with the reference implementation. How is this accomplished?

  2. Unmount your PantryFS drive and try to mount it with the skeleton code instead. The following error message indicates that you are no longer mounting with the reference implementation:

    mount: mount /dev/loop0 on /mnt/pantry failed: Function not implemented

    This error is returned from pantryfs_fill_super() via pantryfs_mount().

  3. Implement pantryfs_fill_super() so that you can mount disks. Note that we’re only interested in making the mount and umount commands work cleanly in this part. We won’t be reading any files or directories at this time.


Part 3: Listing the contents of the root directory (50 points)


  1. In the previous part, we created a VFS inode without associating it with the corresponding PantryFS inode from disk. Update your code to associate the root VFS inode with the root PantryFS inode.

  2. Add support for listing the root directory.


Part 4: Accessing subdirectories (30 points)

In this part, we’ll implement the lookup function of inode_operations. The kernel calls this function repeatedly as it walks through a filepath such as /a/b/c/d/e/f.txt. For example, once it knows the inode of c, it will ask you for the inode associated with the name d in the directory c. Your job is to retrieve the inode for d from the filesystem.

To avoid repeated work when looking up similar paths, the kernel maintains a cache called the dentry cache. Learn how the dentry cache works by reading the materials given earlier.


  1. Add support for looking up filepaths.


Part 5: Reading the contents of regular files (20 points)

In this part, we are going to implement reading the contents of files. Our implementation is made very simple by the fact that each file occupies exactly one block.


  1. Add support for reading the contents of files.

    [archie@vm17 hw7]$ sudo cat /mnt/pantry/hello.txt
    Hello world!
    [archie@vm17 hw7]$ sudo cat /mnt/pantry/members/names.txt
    Kevin Chen
    Mitchell Gouzenko
    John Hui
    [archie@vm17 hw7]$ sudo dd if=/mnt/pantry/hello.txt
    Hello world!
    0+1 records in
    0+1 records out
    13 bytes copied, 4.5167e-05 s, 266 kB/s
    [archie@vm17 hw7]$ sudo dd if=/mnt/pantry/hello.txt bs=1 skip=6
    7+0 records in
    7+0 records out
    7 bytes copied, 5.1431e-05 s, 117 kB/s


Part 6: A full-featured, read-only PantryFS implementation (20 points)


  1. Fix all of the corners we cut in the previous section by making sure your code returns correct metadata for all files and directories. These include size, link count, timestamps, permissions, owner, and group.

  2. At this point, you should stress test your PantryFS implementation. The rest of this assignment will be easier if you can depend on the reading functionality to report things correctly.

    Here are some ideas:


HW7b: Writable Pantry

Below is HW7b (i.e., parts 7–10 of HW7). Before you start, please complete the following.

Please download the updated skeleton code. We fixed some bugs in pantry.ko, and made the following bugfix to the bitvector macros in pantryfs_sb.h. If you modified that header file, simply make the change yourself:

-#define SETBIT(A,k)     ( A[(k/32)] |=  (1 << (k%32)) )
-#define CLEARBIT(A,k)   ( A[(k/32)] &= ~(1 << (k%32)) )
-#define IS_SET(A,k)     ( A[(k/32)] &   (1 << (k%32)) )
+#define SETBIT(A,k)     ( A[((k) / 32)] |=  (1 << ((k) % 32)) )
+#define CLEARBIT(A,k)   ( A[((k) / 32)] &= ~(1 << ((k) % 32)) )
+#define IS_SET(A,k)     ( A[((k) / 32)] &   (1 << ((k) % 32)) )

Review the ops structs in the header files ending with *_ops.h. Ensure that you have commented out all unimplemented function pointers.

By the way, you may set the setattr function pointer to NULL or remove it entirely. This causes VFS to run simple_setattr() instead, which is fine for our purposes.

Part 7: Overwriting existing files (40 points)

So far, we’ve only been reading what’s already on the filesystem. In this part, we’ll begin implementing functions for modifying the filesystem contents.


  1. Add support for overwriting the contents of existing files.

  2. Properly update the VFS inode’s size if the length of the write exceeds the file’s length.

  3. You may have noticed that although the nano editor works, vim complains about fsync() failing and refuses to save the file. Fix it.


Part 8: Creating new files (50 points)


  1. Implement creating new files. That is, user programs should be able to call open() with a mode that includes O_CREAT.


Part 9: Deleting files (60 points)

While testing the previous part, you probably created lots of files that are now cluttering your disk. Let’s implement a way to delete those files.


  1. Review how the VFS dentry and inode caches interact with each other using the resources given earlier in this assignment.

  2. Implement the unlink and evict_inode ops so that you can delete files.


Part 10: Implementing odds and ends (0 points)

This part is optional. However, if you successfully complete one or more tasks in this part, Jae will take it into consideration for boosting borderline grades at the end of the semester.


  1. Implement creating new directories. mkdir should work.

  2. Implement removing directories. rmdir should work on empty directories.

  3. Implement hard and soft links. ln should work on regular files, and ln -s should work on both regular files and directories.


The PantryFS assignment and reference implementation are designed and implemented by the following TAs of COMS W4118 Operating Systems I, Spring 2017, Columbia University:

Last updated: 2017–04–17