COMS W4118: Operating Systems I

Step 1. Using your CLIC account (Easiest)

Either log in physically on one of the CLIC lab machines (clic-lab.cs.columbia.edu), or make sure you have an X Windows server installed on your laptop if you wish to ssh into them. If you have a Mac, install the XQuartz server. If you're on Windows, the XMing server will probably be the easiest option. You can find more options for running X Windows on various platforms here. Once you have a local X Server installed, just ssh into the CLIC environment using "ssh -Y clic-lab.cs.columbia.edu".
The Android SDK (software development kit) and NDK (native development kit) are available in the ~krj/android directory. Create two symbolic links that point to the SDK and NDK directories in your home directory using ln -s ~krj/android/android-sdk-linux ~/android-sdk-linux and ln -s ~krj/android/android-ndk-r8d ~/android-ndk-r8d.
Add the following directories to your PATH in your ~/.profile file: ~/android-sdk-linux/tools, ~/android-sdk-linux/platform-tools, and ~/android-ndk-r8d/toolchains/x86-4.4.3/prebuilt/linux-x86/bin and set the following environment variable export CROSS_COMPILE=i686-linux-android-.

Step 1 (alternate). Using the provided VM (Supported)

A VM environment with the name sp13osvm has been created from you, and is available on the CLIC-lab machines at clic-lab.cs.columbia.edu:/scratch/space/sp13-w4118/sp13osvm.7z. The file is rather large (1.3GB 7zipped), so make sure you are connected to the CS network before downloading it to your laptop or to a USB stick. Upon unzipping the file, you may run the VM either on your own laptop or put it on a USB stick and run it on one of the CLIC machines (clic-lab.cs.columbia.edu). To run the VM on your own laptop, you will either need VMWare Fusion (for Mac OS X) or VMWare Player (for Linux/Windows) software available here through your CLIC account, or the free VirtualBox software (works on Mac/Linux/Windows). Once the VM has booted, you should log in as user w4118 and use the password provided to you in class. You will be required to change the password on first login. To use the VM image on the CLIC machines, run the vmware player application that is preinstalled on those machines.

Step 1 (alternate). Constructing your own Environment (Fastest)

Downloading and creating your own development environment on your laptop is the only option that allows QEMU to run Android natively (through virtualization) rather than emulating the x86 Atom CPU in software, and so is likely to result in much faster development cycles (faster reboot). However, we cannot support all the different environments and host OSes you may have, so expect to spend some time fiddling with paths/Makefiles and figuring out what to do if something goes wrong during the setup.

Download and install the Android SDK Tools and the NDK for your host platform (Windows, OS X, Linux) from here. You do not need to download the IDE that comes along with the SDK. Just go to the "Other Platforms" tab, and choose the SDK Tools only option.
Install the Android 4.1.2 platform and Atom-x86 system image. To do this, you need to run the tools/android command, and from the Android SDK Manager window that starts up, install the Android 4.1.2 (API 16) SDK platform and the Intel x86 Atom System Image. Accept any other dependencies that are installed with these packages.
Install the Hardware Acceleration Module (HAXM) to speed up the emulator. HAXM is an Intel provided extension that allows the emulated Android device to be run as a virtual machine (native x86 execution) instead of being emulated in software, instruction by instruction. Needless to say, this speeds up the emulator quite substantially, and will allow you to run it almost as fast as a real Android device. If your laptop is running a recent version of Linux with kvm installed, you don't have to do anything special except use the command line parameter described below. For Windows and Macs follow the instructions here to configure virtual machine acceleration.
Install git in your environment and set it up following the instructions here.
Set your PATH so that the Android SDK /tools and /platform-tools directories and the NDK toolchains/x86-4.4.3/prebuilt/linux-x86/bin directory are in your path and the following environment variable is set CROSS_COMPILE=i686-linux-android-. Note: if your host machine runs an OS other than Linux, then you will need to use the right cross-compiler toolchain for your host platform. E.g., the cross-compilers for OS X and Windows are located in the toolchains/x86-4.4.3/prebuilt/darwin-x86/bin and toolchains/x86-4.4.3/prebuilt/windows/bin directories of the NDK, respectively.

Step 2. Getting the Android kernel and working with git

A git repository has been created for your group on sp13-w4118-git.cs.columbia.edu and your username and password for this server has been sent to you already. If you wish to change the password, simply do ssh UNI@sp13-w4118-git.cs.columbia.edu.
Clone a working copy of the kernel by typing git clone UNI@sp13-w4118-git.cs.columbia.edu:groupN, where N is your group number. The kernel will download in a local directory called groupN. Your local copy has 5 branches: goldfish-2.6.29, hw2, hw3, hw4, and hw5. You can checkout a branch by typing git checkout <branch_name>. You and your teammates can work on your code from multiple machines - simply clone a working copy on each machine you want to work on, and use git push and pull (below) to synchronize with the central repository on the git server.
Note if you're using your own development environment: on OS X or Windows, working with the kernel is tricky because these OSes have case insensitive filesystems by default. The Linux kernel has several files whose names differ only in their case (yeah, don't ask). If you want to do development on OS X or Windows, you will have to create a special partition with a different filesystem that is case sensitive. For example, on OS X, you can do this easily by using the Disk Utility to create a case sensitive ".dmg" image, mount it, and git clone inside that volume (usually accessible through /Volumes/name_of_vol). Make sure the size of the volume is at-least 500MB (preferably more). The Android kernel itself will occupy about 450MB of space.
To add a file, you need to git add <filename> and to commit your changes to your local working copy, use git commit -a. You will be asked to enter a commit message for the log. Make sure you put meaningful comments in there - we will check the commit log to determine the contribution by each group member.
You can use the git server to synchronize your work with your groupmates. To pull other teammates updates from the server, do git pull and to push your changes back to the server, use git push
Before you compile the kernel, make sure the NDK is installed in the environment of your choosing and add the toolchain directory to your path. We will be using the x86-4.4.3 toolchain that is located in the android-ndk-r8d/toolchains/x86-4.4.3/prebuilt/linux-x86/bin directory of the NDK. You also need to set the $CROSS_COMPILE environment variable to i686-linux-android-. All this is already done for you in the VM environment. Subsequently, you need to configure the kernel (one time only) using the command:
```
        make ARCH=x86 CC="${CROSS_COMPILE}gcc -mno-android" goldfish_defconfig
    
```
Finally, you can compile the kernel using the command:
```
        make ARCH=x86 CC="${CROSS_COMPILE}gcc -mno-android" bzImage
    
```
If all goes well, the first compile should take a reasonable amount of time (a few minutes), and the compiled kernel will be written to arch/x86/boot/bzImage. When you change kernel code, you don't need to reconfigure the kernel every time unless you're changing the architecture from the emulator to a Nexus 7 (or vice-versa), but you do need to recompile it every time.
Kernel debugging: The Android kernel provides a lot of built in support for debugging, including a robust (but heavyweight) potential deadlock detector based on lock ordering. These options result in debug output to be written to the kernel log (dmesg), but can slow down your kernel substantially, so do not keep them always turned on. To enable debugging support, create a custom config based on the goldfish_defconfig configuration during the kernel compile. The command make menuconfig will pull up the kernel configuration tool. From the tool, select the option Load an Alternative Configuration File, and choose arch/x86/configs/goldfish_defconfig. Now, go to the Kernel Hacking subsection and enable Kernel debugging, Lock debugging: proving locking correctness, and Spinlock debugging: sleep-inside-spinlock checking. Finally, go back to the main section (via Exit) and choose Save an Alternate Configuration File. Save this new config as arch/x86/configs/goldfish_debug_defconfig. For your convenience, a prebuilt defconfig file for you to put in the arch/x86/configs is available here. Then, exit the config tool, and configure your kernel to enable debugging as follows:
```
    make ARCH=x86 CC="${CROSS_COMPILE}gcc -mno-android" goldfish_debug_defconfig
    
```
Additional notes for OS X and Windows: When using a Mac (or Windows) for kernel compilation, you also need to have a host gcc environment to compile some binaries needed during the compile itself. On Windows, this can be obtained through Cygwin following the installation instructions on that site. You will also need to install the following Cygwin packaged through the installer: gcc4-core, make, perl, gettext, libelf-devel, libgmp-devel, libmpc-devel, libmpfr-devel, zlib-devel, libncurses-devel, and libncursesw-devel.

On OS X, you can get a host environment through Apple's XCode. There's no need for the entire development environment - only the command line tools will suffice. Subsequently, you will also need to install an additional library libelf and a copy of GNU sed (the OS X supplied sed behaves differently from its GNU counterpart). The easiest way to do this is via HomeBrew. Installation is pretty straight-forward, but a detailed tutorial is available here.

After installing homebrew via:
```
$ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
```
install the libelf and sed packages using
```
$ brew install libelf gnu-sed
```
Then, copy this elf.h header file to /usr/include/ as root (sudo cp elf.h /usr/include) and this sysmacros.h file to /usr/include/sys/. Finally, make gnu-sed, i.e., gsed, the default by creating a symlink called sed to it somewhere in your PATH, e.g.,
```
ln -s $(brew --prefix gnu-sed)/bin/gsed $HOME/android-sdk-macosx/tools/sed
```
As part of your assignments, you will be writing command line test programs that will run inside Android. These programs have to be compiled using the Android compiler and libc. A sample Makefile.ndk that shows you how to do this has been provided for you in the VM, and is also accessible here. Note: The Makefile.ndk in the VM has a bug that can cause your code to seg-fault on quitting. Please use the updated version from the CLIC environment.
As you'll soon realize when you start working with the kernel, it contains a lot of code. LXR (Linux Cross Reference) is an invaluable tool to help navigate it. Amongst other things, you can use it to tell you where particular data structures and functions are defined. An LXR for the 2.6.29 Linux kernel can be found here.

Working with the emulator

The Android emulator can be found in the tools/emulator binary in the SDK. You need to make sure the tools and platform-tools directories are in your path. Before you use the emulator, you need to create an Android virtual device (AVD). An AVD is the emulator's terminology for an emulated phone/tablet. To create an AVD, type android avd to bring up the AVD manager. From the AVD manager, create a new AVD with the name "w4118" (or any other name you prefer). The emulator is a pretty slow environment already, and the more capable the device you're trying to emulate, the slower it gets. To make the AVD as fast as possible, choose a low screen resolution — a 3.3 WQVGA device with a screen size of 240x400 pixels (we won't be doing any GUI work in this class anyway) and 512MB RAM. Make sure you select "Android 4.1.2 - API Level 16" as the target platform and Intel Atom (x86) as the CPU/ABI and that the option to display a skin with hardware controls is turned on. Make sure you do not choose the Snapshot option - this will prevent your AVD from freshly booting the kernel every time your restart it. All of this is already done for you in the VM environment, and an AVD named "w4118" is already present in the ~/.android directory.
To start the emulator with your newly created AVD, type emulator @w4118 and to boot using a custom kernel, type emulator @w4118 -kernel <path_to_bzImage>. See here for more emulator options. In particular, the following options may be particularly useful:
- -qemu -enable-kvm to enable hardware acceleration (HAXM) if you're running the emulator natively on your own laptop.
- -no-boot-anim to disable the Android animation during boot. This usually speeds up the emulator boot time.
- -show-kernel to turn on kernel messages.
- -shell to enable access to the device through a shell even if the adb daemon (see next) isn't working.
- -logcat to display logcat messages.
Use the adb (Android Debug Bridge) command to interact with the device through the command line, and to transfer files to and from the device. If you are using a shared machine (e.g., the CLIC machines), make sure you are connecting to the correct device. adb devices will list the devices currently running on (or physically connected) to your computer and adb -s <device_name> can be used to connect to a specific one. The name of your device will be shown in the titlebar of the emulator. You can find details on how to use the adb tool here. In particular:
- adb shell will open a shell on the device.
- adb shell cmd can be used to run any shell command cmd on the device.
- adb push and adb pull can be used to transfer files to and from the device.
- adb shell dmesg can be used to show kernel messages (similar to the -show-kernel emulator option, but this method allows you to pull the messages on demand).
- Finally, if your device crashes or reboots unexpectedly (expect this often when working with the kernel), the file /proc/last_kmsg will show you the kernel log just before the crash. You can pull it using adb shell cat /proc/last_kmsg.

Working with a Nexus 7 tablet (Optional - no points)

Coming soon!

In the meantime, the high level steps are:

Checkout the tegra kernel from the Android Open Source Project.

Change the default architecture to arm-eabi by setting

        $ export ARCH=arm
        $ export SUBARCH=arm 
        $ export CROSS_COMPILE=arm-eabi-

Reconfigure the kernel for the Nexus 7 using the make grouper_defconfig command and built the kernel the usual way.
Boot your kernel using fastboot by following the instructions here.