Homework 3

W4118 Fall 2012

UPDATED: Thursday 10/04/2012 at 7:00am EST

DUE: Monday 10/15/2012 at 11:59pm EST

Individual Written Problems:

The Git repository you will use for the individual, written portion of this assignment can cloned using: git clone gitosis@os1.cs.columbia.edu:UNI/hmwk3.git (replace UNI with your own UNI). This repository will be accessible only by you, and you will use the same SSH public/private key-pair used for Homework 1.

Exercise numbers refer to the course textbook, Operating System Concepts Essentials. Each problem is worth 5 points.

  1. Exercise 4.12

  2. Exercise 6.6

  3. Exercise 6.11

  4. Exercise 6.15

  5. Exercise 6.31

  6. Exercise 6.35

Group Programming Problems:

Group programming problems are to be done in your assigned groups. The Git repository your entire group will use to submit the group programming problems can be cloned using: git clone gitosis@os1.cs.columbia.edu:TEAM/hmwk3.git (Replace TEAM with the name of your team, e.g. team1). This repository will be accessible to all members of your team, and all team members are expected to commit (local) and push (update the server) changes / contributions to the repository equally. You should become familiar with team-based shared repository Git commands such as git-pull, git-merge, git-fetch. You can see the documentation for these by writing $ man git-pull etc. You may need to install the git-doc package first (e.g. $ apt-get install git-doc.

All team members should make at least five commits to the team's Git repository. The point is to make incremental changes and use an iterative development cycle. Follow the Linux kernel coding style and check your commits with the scripts/checkpatch.pl script included in the Linux kernel. Errors from the script in your submission will cause a deduction of points.

All kernel programming assignments in this year's class are done on the Android operating and targeting the ARM architecture. For more information on how to use adb and aliasing refer to Homework 2. Use "adb -e" for emulator commands and "adb -d" to direct the commands to the device. If you run in to a read-only disk error then type "adb remount" add -e or -d accordingly.

You can use Homework 2's VM for this assignment which can be downloaded from here.

The kernel programming for this assignment will be done using a Google Nexus 7 The Android platform can run on many different architectures, but the specific platform we will be targeting is the ARM CPU family. The Google Nexus 7 uses an ARMv7 CPU. The Android emulator can also be used and is built on a system called QEMU which emulates an ARMv7 processor.
  1. (5 pts.) Install a custom kernel on your Nexus 7

    Make sure you still have the Android SDK unpacked in your home directory on the VM as specified by Homework 2.

    The kernel source is located in your team's git repository in the android-tegra-3.1 directory, and an appropriate ARM cross-compiler has been installed in the VM, and resides in the w4118 user's path. In this homework, there are two configurations that can be used to complete the assignment - one configuration for the emulator, and one configuration for the device (these two devices have different [virtual] hardware and thus require different drivers enabled/disabled). To switch between the two is straight-forward using the kernel's default configuration mechanism.
    To compile for the Nexus 7 device use the custom tegra3_android default configuration: make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- tegra3_android_defconfig To compile for the emulator use the goldfish default config: make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- goldfish_armv7_defconfig
    These commands configure the Makefiles in the kernel to include all the proper drivers / support for the chosen platform. After you have configured the kernel (which is only necessary when switching between emulator and device), you can compile the kernel with the same command you used in Homework 2: make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- Modifications to the configuration can be done using: make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig however, these changes do not get saved into the default configuration, so if you modify the kernel config and then use one of the default configuration commands, your custom changes will be erased!

    The final kernel image (after compilation) is output to: arch/arm/boot/zImage. This file is the one you will be using to boot either the Nexus 7 or the emulator.

    Booting the kernel on the Nexus 7 requires the fastboot utility. This utility can boot a custom kernel on the device, and can also flash the custom kernel into the device's boot partition. Here are the steps you must follow in order to successfully boot a new kernel on your Nexus 7:
    1. Put the device into "fastboot" mode by holding down the lower key while powering-up the device. Alternatively, you can type "adb reboot bootloader" in your VM if you have enabled USB Debugging. You will know that the device has successfully been placed in "fastboot" mode when the screen shows a yellow android.

      NOTE:Your Nexus 7 must be unlocked. First, enable USB debugging by going to "Settings" then "Developer Options" on your Nexus 7 device. Make sure you check the "USB debugging" option. Your VM should be connected to your Nexus 7 USB device (go to settings on your VMware product and connect to your USB device). Next, unlock your Nexus 7 device by booting in to "fastboot" mode and then typing "fastboot oem unlock" in your VM. Follow the instructions on your Nexus 7 screen. Finally, boot in to fastboot mode again. Unlocking is a one-time process.

    2. Once the device is in "fastboot" mode, check the connection to the VM with: w4118@osvm:~$ fastboot devices If you see a line which looks like: HT851N002807 fastboot then you are ready to boot your new kernel.
    3. To boot the new kernel (and leave the existing kernel in-place for subsequent reboots of the device) use the boot fastboot command: w4118@osvm:~/hmwk3$ fastboot boot arch/arm/boot/zImage ramdisk.img The device should eventually boot up with your new kernel.
    4. To write the new kernel image into the boot partition of the device's flash memory, use the following fastboot command:  w4118@osvm:~/hmwk3$ fastboot flash:raw boot arch/arm/boot/zImage ramdisk.img This will allow your system to boot the new kernel after a power-cycle. Be careful though - you will want to ensure system stability before committing your work to flash!


    In the tools directory there is a custom emulator that you should run with your kernel: cp emulator-hmwk3 ~/utils/android-sdk-linux/tools/ # copy binary once emulator-hmwk3 -avd NameOfYourAVD -kernel arch/arm/boot/zImage -ramdisk ramdisk.img -show-kernel
  2. (10 pts.) Write a user-space daemon to pass device orientation into the kernel

    On the Android platform, device orientation is accessed via an on-board "digital compass" device (Asahi Kasei AK8975) You are to write a daemon process, called orientd, which polls the compass and updates the devices orientation in the kernel using the system call interface below:

    /*
     * sets current device orientation in the kernel.
     * syscall number 376
     */
    int set_orientation(struct dev_orientation *orient);
    
    struct dev_orientation {
    	int azimuth; /* angle between the magnetic north
    	                and the Y axis, around the Z axis
    	                (0<=azimuth<360)
    	                0=North, 90=East, 180=South, 270=West */
    	int pitch;   /* rotation around the X-axis: -180<=pitch<=180 */
    	int roll;    /* rotation around Y-axis: +Y == -roll,
    	                -90<=roll<=90 */
    }; 

    Only the administrator (root user) should be able to update the device orientation. All Orientation related functions should be placed in kernel/orientation.c, and include/linux/orientation.h

    A basic command-line daemon project has been provided in your homework repository under the orientd directory. To compile the program simply type make. To install the program on the device (or emulator) type make install_dev (or make install_emu). To run on the device (or emulator) type make run_dev (or make run_emu).

    NOTE:
    This must be a true daemon i.e. it should place it self in the background using a call to fork() as opposed to forcing the program into the background using shell techniques such as the & symbol or the Ctrl-Z, bg command sequence.

    To add files to the project, edit the Makefile and add lines to the SOURCES variable. Please do not remove any of the existing source files.

    HINT: You don't have to have a custom kernel on the device in order for the user space daemon to read the orientation data and print them. We suggest investigating the compass interaction before writing your new system call in the kernel.
  3. (45 pts.) Orientation-based reader/writer locks

    Design and implement a new kernel synchronization primitive that will provides reader-write locks based on device orientation. Reader-writer locks works by allowing several readers to grab the lock at the same time, but only a single writer can grab the lock at any time. You should make sure not to starve writers: If readers hold the lock and a writer wants to take the lock, no more readers can take the lock.

    A lock is defined by a range of the orientation of the device. For example, if a writer grabs the lock for when the pitch is +-5 and the roll is +- 5 for all value of the azimuth, then readers cannot grab any locks in that area, but readers can grab a lock when the pitch is between 80 and 100, and when the roll is between between 20 and 30, for all values of azimuth (given no writer has grabbed a lock covering that area).

    If a process wants to grab a lock (either read or write lock) for an area, which does not cover the current physical device rotation, the process should block until the device is rotated into that area.

    A user space process can hold the lock as long as it wishes, and either eventually gives it up voluntarily or is forced to give it up when the process dies. While locks are only obtained when the device is in the corresponding orientation, locks can be released irrespective of the rotation.

    When the device orientation is updated in the kernel (see Problem 2), the processes that are waiting to grab a lock on an area entered by the new orientation should be allowed to grab the lock (making sure readers and writers don't grab the lock at the same time. If no processes are waiting for the particular orientation when it is updated in the kernel, then the operation has no effect. The API for this synchronization mechanism is the following set of new system calls which you will implement:

    /* Take a read/or write lock using the given orientation range
     * returning 0 on success, -1 on failure.
     * system call numbers 377 and 378
     */
     int orientlock_read(struct orientation_range *orient);
     int orientlock_write(struct orientation_range *orient);
    
    /* Release a read/or write lock using the given orientation range
     * returning 0 on success, -1 on failure.
     * system call numbers 379 and 380 */
    int orientunlock_read(struct orientation_range *orient);
    int orientunlock_write(struct orientation_range *orient);
    
    /*
     * Defines orientation range. The ranges give the "width" of
     * the range. If one of the range values is 0, it is not considered
     * as defining the range (ignored).
     */
    struct orientation_range {
    	struct dev_orientation orient;  /* device orientation */
    	unsigned int azimuth_range;     /* +/- degrees around Z-axis */
    	unsigned int pitch_range;       /* +/- degrees around X-axis */
    	unsigned int roll_range;        /* +/- degrees around Y-axis */
    };

    You should modify the set_orientation system call written in Problem 2 to handle processes blocking on a lock that covers the new orientation. You should choose to allow either all blocked readers to take the lock or a single blocked writer to take the lock. When a lock is taken by one or multiple processes, the processes are unblocked and return to user space. Your strategy to select either readers or a writer should consider consider fairness as a criteria.

    If there are no processes waiting on the event, then nothing happens. Modify the return value of the system call such that it returns -1 on error, and the total number of processes awoken on success (0 or more)

    You should begin by thinking carefully about the data structures that you will need to solve this problem. Your system will need to support having multiple processes blocking on different areas at the same time, so you will probably need a set of area descriptor data structures, each of which identifies an event. Those data structures will need to be put in a list from which your code can find the appropriate area descriptor corresponding to the area that you need. Space for the area descriptors should be dynamically allocated, most likely using the kernel functions kmalloc() and kfree().

    You should not make any assumptions about whether the system is a uniprocessor or multiprocessor system. Be sure to properly synchronize access to your data structures. Moreover, be sure to select the appropriate synchronization primitives such that they are both correct and efficient, in this order. For instance, you should prefer a spinlock over a semaphore if-and-only-if you don't plan to sleep while holding it.

    You can choose to work at the level of wait queues using either the associated low-level routines such as add_wait_queue(), remove_wait_queue(), or the higher-level routines such as prepare_to_wait(), finish_wait(). You can find code examples both in the book (pages 58 - 61 of Linux Kernel Development) and in the kernel. If you prefer to use functions such as interruptible_sleep_on() and sleep_on(), then plan carefully because they can be racy in certain circumstances.

    HINT: a useful method to guarantee the validity of a data structure in the presence of concurrent create, access and delete operations, is to maintain a reference count for the object. Then, the object should only be freed by the last user. The file system, for example, keeps a reference count for inodes: when a process opens a file the reference count of the inode is incremented. Thus if another process deletes the file, the inode remains valid in the system (albeit invisible) because its count is positive. The count is decremented when the process closes the corresponding file descriptor, and if it reaches zero then the inode is freed. A reference count must be protected against concurrent access, either using explicit synchronization or using atomic types (see atomic_t in Chapter 10 of the Linux Kernel Development book).

    You should properly handle any errors that may occur and report meaningful error codes e.g. -ENOMEM in the event a memory allocation fails. As always, you are encouraged to look for existing kernel code that performs similar tasks and follow the coventions and practices it provides.

  4.   (10 pts.) Write three C programs to test the system

    Determining the prime factors of a large number can be useful if, for example, you want to break an encryption system. Your device is just a computer like any other computer, so you may want to use your device for this purpose. However, you don't want your device to be calculating prime numbers when you are holding it and doing things with it. You only want it to calculate when lying on a table face down.

    HINT: The device is lying face up, when the pitch is around 0 and roll is around 0, for all values of azimuth. The device is lying face down, when the pitch is around 180 and the roll is around 0, for all values of azimuth. Remember that the values can also be negative and that -178 for the pitch is equivalent to 182.

    Your programs are going to do the following:


    Each program should pre-pend the output number with the method used (ie. pollard or trial).

    Verify by moving turning device face up and make sure no more numbers are output to the screen. Also make sure that the two calculating processes at no time are working on different numbers.



    HINT:
    The orientation sensor data output and polling interface is documented in the orientd/inc/hardware/sensors.h header file in your homework repository.

    Testing on the Device:

    Testing on the Emulator: