Introduction

The purpose of this project is to add a new filesystem to GeekOS, as well as the standard operations for file management.

A working Project 4 is not required for this project. You can implement Project 5 on top of the provided base kernel. Note however that if you do not build this project on top of project 4, you must make the following changes:

You may also have to make a few small changes if you do not implement on top of project 2. These were listed in the project 3 description. In short, you will have to make Sys_RegDeliver, Sys_Signal, and Check_Pending_Signal return 0, and you will have to provide a minimal implementation of Get_Next_Runnable in kthread.c; you can replace the TODO with

    best = Find_Best(&s_runQueue[0]);
    KASSERT(best != 0);
    Remove_Thread(&s_runQueue[0], best);

GOSFS - GeekOS FileSystem

The main part of this project is to develop a new filesystem for the GeekOS. This filesystem will reside on the second IDE disk drive in the Bochs emulator. This will allow you to continue to use your existing PFAT drive to load user programs while you test your filesystem. The second IDE disk's image is called diskd.img. It is 10MB by default.

GOSFS will provide a filesystem that includes multiple directories and long file name support.

VFS and file operations

Since GEEKOS will have two types of filesystems (PFAT and GOSFS), it will have a virtual filesystem layer (VFS) to handle sending requests to an appropriate filesystem (see figure below). We have provided an implementation of the VFS layer in the file vfs.c. The VFS layer will call the appropriate GOSFS routines when a file operation refers a file in the GOSFS filesystem.

The implementation of PFAT is in pfat.c. You will implement GOSFS in gosfs.c and relevant system calls in syscall.c. VFS picks the functions to call based on supplied structures containing function pointers, one for each operation. For example, see Init_PFAT in pfat.c: this initializes the PFAT filesystem with VFS by passing it a pointer to s_pfatFilesystemOps, a structure that contains function pointers to PFAT routines for mounting (and formatting) a filesystem.  Other PFAT functions are stored in different structures (e.g., look at the PFAT_Open routine, which passes the s_pfatFileOps structure to VFS).  You will analogously use s_gosfsFilesystemOps, s_gosfsMountPointOps, s_gosfsDirOps, and s_gosfsFileOps in gosfs.c. You should also add a call to Init_GOSFS provided gosfs.c in main.c to register the GOSFS filesystem.  In general, use the PFAT implementation as your guide to interfacing GOSFS with VFS.

Open files are tracked by the kernel using a file descriptor, defined as struct File.  Each user space process will have an associated file descriptor table that records which files the process can currently read and write. A user process can have up to USER_MAX_FILES files open at once. The file descriptor table is implemented as a struct File *fileList[USER_MAX_FILES] array that you must add to struct User_Context. Note that not all the entries in the fileList are necessarily open files, since usually a process has less than USER_MAX_FILES files open at once. If fileList[i] is NULL, it represents a free slot (file descriptor is not used). This descriptor will be filled out by the code in VFS; e.g., see Open in vfs.h, whose pFile argument is a pointer to a free slot in the table.

Your filesystem should support long filenames (at most 128 bytes, which includes a null at the end). A file's full path (which includes directories and subdirectories) will be no more than 1024 characters in total.

You should keep track of free disk blocks using a bit vector (as described in the textbook, Section 11.5.1, page 429). A library called bitset is provided (see bitset.h and bitset.c) that manages a set of bits and provides functions to find bits that are 0, which correspond to free disk blocks (note that this is the opposite of the book, which uses a 1 to indicate a free block).

The block size for GOSFS is 4 KB; IDE sectors (a.k.a. disk blocks) are 512 bytes, so a block is 8 sectors.  Thus one bit in a bitset corresponds to a 4KB block. For example, a bitset that is 8192 bits (1024 bytes) will keep track of 8192 * 4KB = 32 MB of data.

Directories and Subdirectories

The filenode for a directory (i.e a subdirectory) is represented by the GOSFS_DIRENTRY_ISDIRECTORY flag set in the GOSFS_Dir_Entry->flags field. As we shall see later, a filenode for a file stores information about the blocks that the file is composed of. The filenode for a directory on the other hand just stores the location of the one block and contains all the information about that directory. The location of the block that holds the data for the directory will be stored in the first entry in the blocksList array of the directory's filenode (hence entries blocksList[1]..blocksList[GOSFS_NUM_BLOCK_PTRS-1] are unused).

Files

See the recitation slides for additional details on directory structure and how files are organized.

Buffer Cache

For this project, we have provided you will a buffer cache implementation, which you can use by including the <geekos/bufcache.h> header file. The Buffer_Cache data structure represents a buffer cache for a particular filesystem instance. You can create a new buffer cache by passing the block device to the Create_FS_Buffer_Cache() function. A buffer containing the data for a single filesystem block is represented by the FS_Buffer data type. You can request a buffer for a particular block by calling the Get_FS_Buffer() function. The actual memory containing the data for the block is pointed to by the data field of FS_Buffer. If you modify the data in a buffer, you must call the Modify_FS_Buffer() function to let the buffer cache know that the buffer is now dirty. When your code has finished accessing or modifying a buffer, it should be released using the Release_FS_Buffer function.

Buffers are accessed in a transactional manner. Only one thread can use a buffer at a time. If one thread is using a buffer and a second requests the same buffer (by calling Get_FS_Buffer()), the second thread will be suspended until the first thread releases the buffer (by calling Release_FS_Buffer()). Note that you need to be careful never to call Get_FS_Buffer() twice in a row, since that will cause a self-deadlock.

The GeekOS buffer cache writes dirty buffers to disk lazily. If you want to immediately write the contents of a buffer back to the disk, you can call the Sync_FS_Buffer() function. It is generally a good idea to write back modified filesystem buffers when they contain filesystem metadata such as directories or disk allocation bitmaps; by writing these blocks eagerly, the filesystem is less likely to be seriously corrupted if the operating system crashes or if the computer is turned off unexpectedly. You can flush all of the dirty buffers in a buffer cache by calling the Sync_FS_Buffer_Cache(); this is useful for implementing the Sync() operation for your mounted filesystems.

The Destroy_FS_Buffer_Cache() function destroys a buffer cache, first flushing all dirty buffers to disk. This may be useful in your implementation of GOSFS_Format().

Mounting

The Mount system call allows you to associate a filesystem with a place in the file name hierarchy. The Mount call is implemented as part of the VFS code we supply (see Mount function in vfs.c); you will implement your Init_GOSFS function so that VFS's mount code will call your function GOSFS_Mount() in gosfs.c. Among other things, it must ensure that the filesystem being mounted is GOSFS. The new syscalls section below contains more details.

New System Calls

You have to implement the semantics of the new system calls as described below. As you can see, the semantics is very similar to the UNIX file system.

• NOTE: All user-supplied pointers (e.g. strings, buffers) must be checked for validity.
• NOTE 2: Some of the checks in 'Reason for failure' column are already done by vfs.c so you don't have to perform the check if it's implemented for you already.

Call	User Function	Return on success	Return on failure	Reasons for failure	Comment
SYS_MOUNT	Mount(char *dev, char *prefix, char *fstype)	0	-1	a filesystem already mounted under name illegal value for one of the parameters
SYS_OPEN	Open(char *path, int mode)	new file descriptor number	-1	name does not exist (if permissions don't include O_CREATE ) path to name does not exist (if permissions include O_CREATE ) attempt to open a directory, (should use OpenDirectory)	there's no create syscall, so setting O_CREATE will create the file. If the file exists, the call succeeds (return >= 0) but its data contents is not affected. Should NOT create directories; e.g. if you do Open("/d/d1/d2/d3/xFile", O_CREATE) and the leading path /d/d1/d2/d3 does not exist, the syscall fails, returning -1 You should use the Allocate_File in vfs.c to get a File when opening. The permissions values are flags and may be or'ed together in a call. For example: O_CREATE | O_READ O_READ | O_WRITE O_CREATE | O_READ | O_WRITE
SYS_OPENDIRECTORY	Open_Directory(char *path)	new file descriptor number	-1	path does not exist attempt to open a file, (should use Open)	Should NOT create directories, use CreateDirectory instead. You should use the Allocate_File in vfs.c to get a File when opening.
SYS_CLOSE	Close(int fd)	0	-1	fd not within range 0-(USER_MAX_FILES-1) fd is not an open file
SYS_DELETE	Delete(char *path)	0	-1	name does not exist name is a non-empty directory	if Delete(file) is called and file is still open in other threads or even in the thread that called Delete(), all the subsequent operations on that file (except Close()) should fail
SYS_READ	Read(int fd, char *buffer, int length)	number of bytes read	-1	fd not in range fd is not an open file fd was not open with O_READ flag fd is a directory	it's OK if return value < length, for instance reading close to end of file increase the filePos (the current position in the file) by the number of bytes read (if successful)
SYS_READENTRY	Read_Entry(int fd, VFS_Dir_Entry *entry)	0 if entry is valid >0 if end of directory reached	-1	fd not in range fd is not an open directory	copy over directory entry info from a GOSF_Dir_Entry into a VFS_Dir_Entry skip over unused entries increase the filePos, if successful VFS_Dir_Entry is defined in fileio.h
SYS_WRITE	Write(int fd, char *buffer, int length)	number of bytes written	-1	fd not in range fd is not an open file fd was not open with O_WRITE flag fd is a directory	increases filePos by the number of bytes written, if successful "Grow on write"- allocate blocks "on the fly" if past end of file "Grow minimum" - allocate only the needed blocks. A seek to a position past the end of the file should be allowed and a write will allocate only a single block rather than all blocks in between.
SYS_STAT	Stat(char *path, VFS_File_Stat *stat)	0	-1	path is not valid	VFS_File_Stat is defined in fileio.h
SYS_FSTAT	FStat(int fd, VFS_File_Stat *stat)	0	-1	fd not within 0-9 fd is not an open file	VFS_File_Stat is defined in fileio.h
SYS_SEEK	Seek(int fd, int offset)	0	-1	fd not within 0-9 fd is not an open file	offset is an absolute position; could be equal to fileSize or greater, then Write appends, see above
SYS_CREATEDIR	Create_Directory(char *path)	0	-1	name already exists, as file or directory regular file encountered on the path to name	Should NOT create directories recursively; e.g. CreateDirectory("/d/d1/d2/d3/d4"), will fail if /d/d1/d2/d3 does not exist already.
SYS_SYNC	Sync(void)	0	-1	None	Forces any outstanding operations to be written to disk.
SYS_FORMAT	Format(char *dev, char *fstype)	0	-1	illegal value for dev dev is in use, i.e. mounted	You only have to implement the format of the GOSFS filesystem.

Disk Layout

Here are guidelines for how you should set up your filesystem on the disk.

• The first block (0) is called the SUPERBLOCK, and it contains filesystem housekeeping data. Blocks >= 1 contain files and directories.
• The Magic number at the very beginning could be 0xDEADBEEF, "GOSF" or the like. This tells you that the disk has a GOSFS filesystem on it. If you try to mount a drive and you don't find the magic signature, return error.
• Root Dir Pointer holds the block number of the block containing the root directory.
• Size is the size of the disk, in 4KB blocks. (32M / 4K = 8K for the example above)
• Free Blocks Bitmap is : Size bits large, that is Size/8 bytes large. (8K / 8 = 1K for the example above). Every block has an associated bit.

  When you do a Format() , you make a raw disk usable with GOSFS. That is:

  1. Get drive's size, convert it in # of blocks. Get_Num_Blocks() tells you that.
  2. Figure out Free Blocks Bitmap size, mark them all free.
  3. Create a valid, but empty directory. That will be the root directory. Make Root Dir Pointer point to it.
  4. Mark superblock and block for root directory as used in the Free Blocks Bitmap
  5. If everything went OK, write the Magic. Now the disk is ready to be mounted and used.
  Keep in mind that the superblock and root directory have no associated GOSFS_Dir_Entry.

How to create an arbitrarily big diskd.img

• Change the size/disk geometry by changing the diskd line in .bochsrc
• Change the argument to $(ZEROFILE) in Makefile. For $(ZEROFILE) one block is 512, so 4096 blocks = 2 MB, 65536 blocks = 32 MB and so on.

Notes

Requirements

• Make sure your Mount() works well, so that we can test your project. If we cannot Mount() a GOSFS, we cannot grade your project.
• You might also want to mount /d to ide1 automatically in main() to speed up your testing, but the code you submit should not mount /d automatically.
• You should support disk sizes of at least 32 MB. More than 32 MB is optional. Following the procedure described in the "How to create an arbitrary size big diskd.img" section above, in your submitted project, when someone types gmake, a 32 MB file should be created.
• You should support file sizes of between 0 and at least 5 MB (double indirect threshold crossed, yes). More than 5 MB is optional.

Testing