There is a fairly complete list in the
course web page of Fall 2004. The working environment is almost
identical to that of Fall 2004, except that we are no longer working
with Linux but with OSKit.
You are going to build your own kernel from scratch.
1. Accessing your CLIC machine
Your team will be given one of the CLIC Lab machines (the host
machine), on which the virtual machine (VM) for your team has been
installed. The operating system of the host machine is Redhat Linux
9. Half of these machines are located on the left-hand side of the
CLIC Lab, and the rest of them are on the racks of the CRF machine
room. The latters must be accessed remotely. Even if the machine
assigned to your team is located in the CLIC Lab, more often than not
you cannot access directly from its console, so you need to know how
to remotely connect the machine that hosts your VM.
1.1 Secured shell (SSH)
If you are using a modern Unix-based operating system, (e.g., Linux
machines on the left-hand side of CLIC or Solaris-based Sun-Ray consoles
on the right-hand side, or MAC OS X) you should already have SSH installed.
Using the following command to login your VM host machine (replace
<machine> with the machine name assigned to
you):
ssh -X <machine>.clic. cs.columbia.edu
If you are running Windows, you need to install an SSH software. One
of the choices is Cygwin,
a software package emulating Linux systems
with most of the Unix commands included. You can also use a stand-alone
SSH client software, e.g.
PuTTY or
TeraTerm(with SSH2).
Use SSH protocol version 2 if you have a choice. SSH protocol version
1 is known to have security problems.
1.2 X Server
SSH only provides a telnet-like interface. To run GUI applications
such as VMware, you need an X server installed on your desktop. The
X-server displays windows and dialogs of (and also sends your keyboard
and mouse inputs to) the client process running on a remote machine
(known as X-client) in a so-called
>X protocol.
Here the remote machine
is your VM host machine. However, there is an alternative way, please
see the next section: using VNC.
- If you are using a Unix system (Linux, Solaris, FreeBSD, etc), you
need to do nothing more. The X server has already been installed.
With the -X option in the previous SSH command, you are able to use
the remote GUI applications. Note that the data transmitting between
the X-server and X-client are encrypted by using a tunnel of SSH.
- If you are using Mac OS X 10.3 or newer, you need to install the
X11 package. This package is
downloadable from Apple
.
- If you are using Windows and
- if you are using cygwin with everything (including cygwin/X) installed,
type 'startx' in the cygwin command window to initialize the X server.
You are then able to run GUI applications on the remote machine after
login with 'ssh -X' command.
- if you are using a stand-alone ssh client software, you need to install
a stand-alone X-server software. There are a few choices:
- Many commercial software packages are available, e.g.,
Exceed
and X-Win32.
You can get a free license for one-month from
X-Win32. The CS department has licensed X-Win32 for department-wide
use; if your computer resides in the CS network, you can use
the license key provided by CRF. There is no free X servers for
Windows until very recently cygwin/X and WierdX showed up.
- WierdX, originally known
as WiredX, is a Java-based X-server. If you are interested, you are
encouraged to have a try.
Firewall issues.
If your desktop computer is behind a firewall, you may have trouble
with your X server, since the X server is listening for incoming
network connections initiated by X-clients. These connections must go
from your VM machine to your desktop through the firewall. If you are
using Windows XP/SP2 with firewall enabled, you have to unblock the
network access to the X-server (e.g., cygwin whose X-server process is
`xwin.exe', or the X-Win32 process). If you have no privilege to
change the firewall configurations, or you are in some more complicated
scenarios (e.g., NAT), use VNC instead - see the next section.
X forwarding in SSH.
As mentioned earlier, the data transmitted in the X protocol are encrypted
if you are using `ssh -X'. If you are using a stand-alone SSH software
package on a Windows system, try to find and enable the option 'X
forwarding'. With X forwarding, you may circumvent the firewall issue
in the same way as using `ssh -X'. If your SSH software package does not
support X forwarding, you can type the following command after SSHing to your
VM host machine:
export DISPLAY=<your_desktop_pc_name_or_IP>:<your_X_server_display_number>
Usually <your_X_server_display_number> is 0.
Make sure that your X-server accepts connections from your VM machine.
With this solution, the data transmitted in the X protocol are NOT
encrypted, which means potentially some malicious programs on the network may
eavesdrop what you see and type in your X Window. Also, you may have to solve
the firewall issue.
1.3 Virtual Networking Computing (VNC)
In fact, there are two different VNC projects:
RealVNC and
TightVNC. They are both
backward compatible to the aged
VNC 3.x.
The recent Fedora Core 2/3 is using RealVNC, whereas in Redhat Linux 9,
TightVNC is installed.
There are several advantages to using VNC over X-server.
- With VNC, your desktop makes connections to the Internet instead of
the other way around, therefore it works even if you are behind a
firewall.
- With VNC, you get a `mobile working desktop window' which can be
shared between you and your teammates. The windows within the desktop
window and their corresponding applications will not be closed when
you close the VNC desktop window on your desktop machine, as long as
the VNC server keeps running. Moreover, when you are working with VNC,
your teammates can have read-only (or more precisely view-only)
windows to see what your are doing. This is particularly useful for
people who cannot always work with teammates in a same geological
location.
- If the previous connection is lost, uou can re-connect the VNC server
without losing your previous work. It is useful if you are on an unstable
network.
- The client program of VNC (VNC viewer) is a pretty small program.
For Windows, it is just an executable file - no installation is necessary.
However, there are a few drawbacks:
- You have to run a VNC server somewhere, usually on the same machine
as your VM.
- With TightVNC and the Free Edition of RealVNC, the data
transmitted between VNC server and client are NOT encrypted. There is
a simple way to encrypt data by using SSH tunneling - you may talk
with us if you really concern about security.
- All your group members will be accessing the same CS account.
VNC Infrastructure
Three computers are involved in VNC communication: the remote computer,
say A, your desktop computer, say C, and the computer where VNC server
resides, say B. The communication between A and B is in the X protocol,
where B is the X-server. The communication between B and C is in
a protocol defined by VNC itself, where B again is the server.
One may consider the VNC server, B, as an X proxy. Hence, if both
A and C are behind their firewalls, you can still find a way to access
A from C through a B which resides outside both firewalls. In our
cases A and B are the same computer.
Using VNC: A Quick Start
- Download and install VNC viewer on your desktop machine. It should
be included in the recent Linux distributions. The command name of
VNC viewer is `vncviewer'. VNC viewer has a Windows version that can
be downloaded
here.
- Login your VM host machine (-X option is not necessary here).
- Type `vncserver :1' where :1 is the display number. The display :1
may already be used by other people. In this case, try :2, :3 and
so on. Alternatively just type `vncserver' and let VNC server find a
free display number for you.
- Type `vncpasswd' to set a password. Without a password, anyone can
access your VNC desktop. We suggest a different password from your
CS account, because this password is shared by you and your teammates.
- Run your VNC viewer on your desktop machine and type the
<vm_hostname>:<display_number>
and the password in dialogs. Then the VNC desktop window shows up.
2. Collaborating with your teammates
Your team is free to choose methods regarding coordination of your
team development. However, we strongly suggest to use some source
code management systems. Your team can also use Instant Messaging for
quick communication, such as ICQ or MSN Messenger.
Source code management tools:
- Revision Control System (RCS).
RCS is an aged source code management system. For each of your source
file, say abc.c, RCS can create an RCS file for you, named abc.c,v in
the same directory, which stores all of your past changes. To save the
current change over the previous revision, you can use `ci' command to
check in a new revision to the RCS file. The changes with respect to
the previous revision are then added to the RCS file. With this RCS
file, you can review old changes, revert an unsuccessful change, or
store the change logs. Please type `man rcsintro' for more
information.
- Concurrent Version System (CVS).
CVS is currently the most popular open-source version control system.
CVS improves RCS by moving RCS files around. It separates the working
directory and the centralized repository directory that contains RCS
files. The repository directory may or may not reside on the same
machine as your working directory. Each of you may have your
own working directory whereas there is only one respository directory.
CVS can merge the changes and (partially)
solve the code conflicts due to simultaneous modifications by
different developers.
See Prof Stephen Edwards'
two-page introduction to the CVS version control system for a quick start.
CVS has been installed on all CLIC machines and is available in most Linux
distributions.
- Subversion (SVN).
CVS has some fundamental problems with the current implementation. For
examples, the directory is not versioned, and the version system is
per-file based (as well as in RCS). The file lock mechanism sometimes
raises problems that need to be manually corrected. Stressing on solving
these problems, the recent project Subversion is designed to be a
replacement of CVS. It relies on
Berkeley DB (A separate project which is not included in Subversion
distribution. Note that Berkeley DB is not an
SQL or relation-based DBMS). Binary versions of Subversion and
Berkeley DB are included in recent Redhat Fedora Core distributions
but not in Redhat Linux 9. Please see the Subversion website for more
details.
RCS, CVS, SVN are NOT backup tools.
You are advised to make regular backups for your project. If you are
using CVS, back up the CVS repository directory tree regularly. The
VMware directory (usually /vmware1 on your VM host machine) should be
fairly reliable, but it does not subject to human errors. If you plan
to place your CVS repository in the VMware directory, the simplest way
is to regularly compress your CVS repository to archive files, by using
`tar/gzip/bzip' or `zip', and then place these files in your home directory,
since your home directory is supposed to be nightly backed up.
$Id: tools.html,v 1.5 2005/01/27 20:45:26 hanhua Exp $