Computer Science W1007: Data Structures in Java. Spring 2010. Back to main 1007 page

Introduction to Cunix

By Christy Lauridsen
Modified by Lauren Wilcox, then David Elson, then Bert Huang

What is UNIX?
Getting to Cunix
Managing your Files
Editors
X-Windows
Compiling and Running your Code
Your .profile
Shortcuts
Restoring a Cunix File from a Backup
SFTP: File Transfer Protocol
Printing
Columbia Computer Labs

What is UNIX?

Unix is a popular operating system, originally developed by Bell Laboratories in 1969, and written in the C programming language. There are a variety of different versions of Unix in use today. Columbia's Cunix runs a version of Unix called Solaris (sometimes called SunOS) from Sun Microsystems.

Unix systems are multi-user and multi-tasking, so they are set up to:

Most of your interaction with a Unix system takes place in a shell, a program that is run every time you log in, and displays the "$" prompt. The shell is known as a command interpreter; you give it commands, and it runs them. It can also be called a terminal.

Unix was originally a command-line only operating system, used mostly for servers. Today, there are many graphical interfaces available, making it popular for home workstations, running implementations such as Linux. Mac OS X is based on Unix and has the largest number of users of any Unix derivative.

This tutorial will focus on getting you familiar with the Unix command-line programs, and some Cunix specific details.

Getting to Cunix

To get into Cunix, the computer you're working on needs to be set up with an SSH client.

Mac users can simply open Terminal, a program in the Utilities folder. Once Terminal launches, a shell with a command interpreter ($) should appear. To transfer from your Mac to Cunix, type:

$ ssh yourUNI@cunix.cc.columbia.edu

Type in your password when Cunix requests it. More information about using Mac Terminal to connect to Cunix can be found here.

Windows users need to download PuTTY. CUIT maintains a copy of PuTTY that is pre-configured to connect to Cunix. You can download it here, with further instructions here.

Managing your Files

ls (list)

This command lists the contents of your current working directory. When you first log in, you will be in your home directory. Your home directory has the same name as your user-name, for example, abc123, and is where your personal files and subdirectories are saved.

To see the contents of your home directory, type:

$ ls

There are some hidden files that are not listed with the ls command, you can see these files by using the '-a' option with ls. To see your hidden files, type:

$ ls -a

Hidden files begin with a dot (.), and usually contain important program configuration information for your account.

The ls command is an example of a command which can take options. The options change the behavior of the command. There are online manual pages that describe which options a particular command can take, and how each option modifies the behavior of the command (more later on manual pages).

Other useful options for the ls command include:

$ ls -l (this lists additional information about your files such as size, date last modified, and file permissions).
$ ls -R (this lists all of the files in each sub-directory, recursively)

You can also combine options, such as:

$ ls -la

mkdir (make directory)

This command creates a new directory with a name you specify. Directories in Unix are similar to folders in Windows, and help you organize your files. For example, if you want to create a directory to store your homeworks, you might type:

$ mkdir homework

Now you can use the ls command to see that your new directory exists.

rmdir (remove directory)

Similar to mkdir, the command rmdir directoryname will remove a directory if it is empty. If it is not empty, you will have to remove the files it contains first.

cd (change directory)

The command cd directory changes your current working directory to 'directory'. To change to the directory you just created, type:

$ cd homework

Type ls to see the contents of this directory (which should be empty).   Typing just $ cd will always bring you back to your home directory.

pwd (print working directory)

To find out which directory you are in, type:

$ pwd

This will show you the full path to your current working directory, relative to the root directory (/). This should look something like '/u/2/c/cl303/homework'.

rm (remove files or directories)

The command rm file will remove the specified file. Some useful options for rm include

$ rm -i    (interactive mode - you will be prompted to confirm file deletion)
$ rm -f    (force remove - overrides interactive mode and removes file without a prompt)
$ rm -r    (recursive - remove the contents of directories recursively)

cp (copy files and directories)

To copy a file, do:

$ cp oldfilename newfilename

Or to copy a file to a new directory, while keeping the old filename do:

$ cp filename directoryname/

To ensure you don't overwrite any existing files you can use the '-i' option:

$ cp -i oldfilename newfilename

This command will prompt you to confirm if there is already a file with the name newfilename.

mv (move files)

This command allows you to rename a file as in:

$ mv oldfilename newfilename

Or to move a file to a new directory:

$ mv filename directoryname/

As with cp, it is a good idea to use the '-i' option with mv, so you don't mistakenly overwrite your existing files.

The special directories ~ . ..

You may have noticed when you typed ls -a that there were listings dot (.) and dot-dot (..). These represent your current working directory (.) and its parent directory (..). So, if you are in the 'homework' directory, for example, typing:

$ cd ..

will bring you up one directory, back to your home directory, while 

$ cd .

will leave you where you are.

In addition the tilde (~ and ~user-name) always represent your home directory, so typing:

$ cd ~

will always bring you back to your home directory. And:

$ cd ~/homework

will bring you back to the 'homework' directory.

The special characters * and ?

The character * is called a wildcard and will match against none or more character(s) in a file or directory name. For example, if you type

$ ls h*

The ? character is similar to the * character, except that ? will match exactly one character. For example:

$ ls ?an

Will match 'man' and 'lan' but not 'stan'!

Your friend: tab-completion

It's pretty tedious to type out long directory and filenames every time you want to edit a file Tab-completion can help! Words (filenames, directory names, or even commands themselves) are automatically expanded on the command line if you enter the first few unique characters and then press the <TAB> key.

To test this out, from you home directory type:

$ cd ho<TAB>

If the name you're trying to complete is not unique (i.e. you have a 'homework' directory and a 'homes' file), then hitting the <TAB> key again will display a list of possible completions for you to choose from.

How do I find out more about xyz command?

Extensive documentation of Unix commands can be found in the manual pages. These on-line manuals tell you which options a particular command can take, and how each option modifies the behavior of the command.

For example, to find out what the wc (word count) command does type:

$ man wc

Or, to find out more about the manual pages themselves type:

$ man man

If you want a one line summary of a command, use the whatis command:

$ whatis wc

Or, if you can't remember the exact name of a command, but know a keyword related to it, do:

$ apropos keyword

Editors

Emacs

Emacs is an extremely versatile text-editor, and is particularly good for programming.   Emacs provides modes for various programming languages which allow for context sensitive indentation and layout, and allow you to compile your programs inside Emacs, with links from error messages to source code. It even has spell checking and a Rogerian therapist included!

To start Emacs, at the command prompt type:

$ emacs

Or, to work with a particular file type:

$ emacs filename

If you're using Emacs from within your shell (which you are at this point), you need to know these basic key-stroke-combinations. Here a "C-x" means "hold the Control key down while you hit the x key".

Quit C-x C-c Quit Emacs (prompts to save file changes)
Save C-x C-s Save to current filename (or prompt for filename if none specified)
Save as C-x C-w Prompts you to type a new filename at the bottom of the screen.
Open C-x C-f Prompts you to type an existing or new filename.
Help C-h t The emacs tutorial
Help C-h i Emacs Info mode
Search C-s Search for a string
Close C-k Kills the current buffer
Abort C-g Abort a partially executed command (this will free you when you're stuck mid-command or when you mistakenly get into the help page).

More useful emacs commands can be found here: http://www.columbia.edu/acis/publications/emacs.html
and at this tutorial

Pico

Pico stands for PINE composer, and is the text editor used by the email client PINE (a Program for Internet News & Email). Pico is easy to use, since it always displays a menu of commands at the bottom of the screen, but does not have the programming specific features of emacs.

You can open pico by typing either of:

$ pico (opens a blank file)
$ pico filename (opens specified file)

Here are some common pico commands (the ^ stands for the Control key):

You can find more information about Pico commands here http://www.columbia.edu/acis/publications/pico.html

Vi

The Vi editor has powerful features to aid programmers, but many emacs devotees avoid Vi at all costs. *

The Vi editor has two modes: command and insert. The command mode allows the entry of commands to manipulate text. The insert mode puts anything typed on the keyboard into the current file. Vi initially starts in command mode.

Open a file in Vi by typing either of:

$ vi (opens a blank file)
$ vi filename (opens specified file)

Here are some basic Vi commands:

Command mode
i, a Enter insert mode
:w Save the file without exiting
:q! Quit without saving
:x, ZZ Quit and save the file
h Move left one character
k Move up one line
j Move down one line
l Move right one character
Insert mode
ESC Enter command mode

For more information on VI visit this tutorial: http://www.eng.hawaii.edu/Tutor/vi.html


* Read more about the Emacs/Vi schism here: Vi vs. Emacs

X-Windows

X-Windows is a windowing system used almost exclusively on Unix platforms. X-Windows allow commands executed on a remote machine (i.e. the Cunix cluster), to display graphical information on your local machine (i.e. the workstation you're sitting at). With X-Windows, an X-server controls the display directly, and is responsible for all input/output via the keyboard, mouse or display. X-clients do not access the screen directly, but communicate with the server and do the actual 'work' of running applications.

To use X-Windows with Cunix you need an X-server. All of the AcIS lab machines already have X-servers installed. If you need an X-Server for your home computer you can download one free from:
http://www.xwin32.com/ (PC only)
http://www.microimages.com/freestuf/mix/ (PC and Mac)
Mac users need only start X11 (found in the Utilities folder) and log in to the X11 terminal like this:

$ssh -X username@cunix.cc.columbia.edu

Step 1: Start your X-server

For AcIS Sun workstations in 251 Mudd: you already have an X-server running. Skip to step two.

Step 2: Tell the remote machine (Cunix) where you want it to display programs. This is done by setting you display environment variable ($DISPLAY) to the name of your local machine.

First, find out what machine you're on. To do this type:

$ who am i

The last field of output (in parentheses) should be your machine, or hostname. Now, using this hostname, type:

$ export DISPLAY=hostname:0.0

[note: if you're using the C-shell (csh) use this line instead: 'setenv DISPLAY hostname:0.0' If you don't know what the C-shell is, you are most likely using the bash shell, which is the default for Cunix users]

Step 3: Tell your local machine that it's ok for another computer to open applications on your display. 

For AcIS Sun workstations in 251 Mudd: Open an X-term window (the X-term icon is located at the bottom of the screen) and type:

$ xhost
For all other machines: This step should not be necessary.

A note about xhost: Typing "xhost +" allows all other computers to connect and start applications on your display. "xhost -" removes that privilege. "xhost +machine_name" allows you to specify the machine to which you wish to grant X access.

Step 4: You're ready to open up an application. For example try:

$ emacs filename
To regain use of your shell while emacs is running, instead type:
$ emacs filename &
The '&' allows emacs to run as a background process. Note: You may not be able to run X sessions if you are behind a firewall.

One of the best reasons to use the X-Windows version of Emacs is for the text color coding. (We used this feature in class.) Variable names, class names and other special parts of your program are given different colors, so it is easy to navigate around your code and spot typing errors without even compiling.

The name of this Emacs feature is font lock mode. To turn it on in graphical (X-Windows) Emacs, type

Meta-X
where the "Meta" key is the Escape key. Hit Meta first, then X. Emacs should bring the cursor down to a line at the bottom of the screen that begins with M-x. Here, Emacs waiting for you to finish your command. Type:
M-X global-font-lock-mode
and Emacs will properly fill in the colors for your Java file. Note that this only works if your Java code is in a file with the proper name extension (i.e., ending in .java, which is how Emacs knows which color-coding scheme to use).

Compiling and Running your Code

To compile java code, use:

$ javac MyJavaProgram.java

When all errors have been addressed, compiling in this way will generate a .class file. Use:

$ java MyJavaProgram

to run the Java code (leave out the .java and .class extensions when running). Remember that a class must be named the same as the file in which it resides. For example, MyJavaProgram.java has to contain a public class called MyJavaProgram, with the correct capitalization.

Your .profile

Your .profile is a file in your home directory that configures your environment when you log in to Cunix. There are many useful settings you can put in your .profile, and here we will see just a few.

In Unix, your environment is defined by a set of environment variables which can be set by including a line like 'export VARIABLENAME=value' in your .profile. (see Java example below). [Note that if you're using the C-shell, use the file .login rather than .profile, and set varables with the line 'setenv VARIABLENAME value'].

To change your environment settings, you will need to edit your .profile (in emacs, pico, or vi).  
Once you've modified and saved your .profile, there are two ways to make the changes take effect. 1. Log in again, or 2. type:

$ ~/.profile   (or $ source ~/.profile)

This command forces the .profile to be executed or 'sourced', even though you have not logged in again.

Note that any command or environment variables in your .profile can also be used on the command line (and vice versa), Variables set on the command line will only remain set for the life of the shell in which they are entered, while those entered in the .profile are set each time you log in to a new shell.

Other useful additions to your .profile

Changing your prompt
To change your prompt, use the $PS1 environment variable, but including a line like this in your .profile:
export PS1='this is my new prompt! :)'
Recalling the commands from the beginning of this tutorial, can you guess what this prompt will look like?
export PS1='$PWD>'
Setting your TERM type
Sometimes Unix programs that need to display information in your shell get all funky and messed up, seemingly for no reason. This happens especially when you're using a terminal (i.e. telnet program) on a PC.   Different terminals use different control sequences to move the cursor around and change the display in your shell. The $TERM environment variable is used to communicate this information to emacs and to other Unix utilities. Since almost all terminal emulation software for PC's have a vt100 emulation mode, you can often fix the problem by typing at the command prompt:
export TERM=vt100
or better yet, adding it to your .profile so it's always set.
Aliases
An alias, as the name suggests, assigns an assumed or additional name to a given command. Aliasing long, frequently typed, commands can make your life a lot easier. To set an alias, type alias newname='old command'. These are some recommended aliases to keep your files safe: 
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

You can find some more common environment variables here: http://www.columbia.edu/acis/tutor/Unixhelp/tables_env_list.html

Shortcuts

When running the same command multiple times, remember that

$ ![char]
will correspond to the last command you typed in that began with that char. For example, to run $pine (given that your previous command was $pine), try typing
$ !p
and the command executed will be the same command last executed that began with that letter, in this case $pine. On cunix the command will appear and you will be required to press return again if that is the command you intended to run.

Shortcuts using {} to build strings

For example, if you want to create sets of subdirectories to store your homeworks, you might type:

$ mkdir hw

$ mkdir hw/{hw,submit}{1,2,3,4,5}

Now you can use the ls -F command to see that your new directories exists.

The indispensable history command

Instead of going through your command line history with the up-arrow command by command, you can type:

$ history
to see the last 500 commands you typed. Remember, the higher in number, the most recent the command. I also like using the pipe with more or less so that I can view the commands one screen at a time.
$ history | more

Restoring a Cunix File From a Backup

The AcIS .snapshot system has saved many a weary programmer from the occasional delirious 'rm -fr *' and other less devastating deletions. Cunix's backup systems is one of its best features. Though you won't often need it, at the right time, it can be a life saver.

ALL of your files are backed up every hour for the past several hours and every night for the past several nights!

The backups are stored in the .snapshot directory, a virtual directory that's hidden even from ls -a.

From any directory you can type:

$ cd .snapshot

cd into one of the time directories (i.e. hourly.0) to see what your files looked like the specified time ago, and use ls -l to view the timestamps.

To retrieve a file from the .snapshot directory you must copy it back to where you want it:

$ cp filename ~/path-to-file/filename-backup

Then go back home:

$ cd ~

SFTP: Secure File Transfer Protocol

SFTP allows you to transfer files between your local PC or Mac and Cunix (on the Suns there is no real 'local' file storage, so this is not necessary). An SFTP program will prompt you for your Cunix username and password, and then will give you access to transfer your Cunix files.

The two needed commands for SFTP are:

get -- transfer file from remote machine (i.e. Cunix) to local machine (i.e. your computer)
put -- transfer file from your local machine to remote machine

There are a few different methods for using SFTP:

SFTP Software

On the AcIS PCs, a program called WIN-SCP provides a fairly intuitive graphical user interface for transferring files.
WS-SCP can be downloaded for your home PC from here http://www.columbia.edu/acis/software/winscp/

On the AcIS Macs, a program called Fugu is similarly user friendly. (Note: chose Raw Data when given the transfer method option)
Fetch can be downloaded for your Mac here http://www.columbia.edu/acis/software/fugu/

Update: Fugu has not been updated in a few years. While the sftp protocol has not been changed much recently, you may experience problems with compatibility with newer versions of Mac OS X. If you do, you can try the very popular CyberDuck software (http://cyberduck.ch/). Alternatively you can use command line tools sftp or scp. Type man sftp for more info.

SFTP Through a Web Browser

This is a popular method because it is portable across all operating systems, and allows you to drag and drop your files.

In your browser's url field type: sftp://username@cunix.cc.columbia.edu (substituting your username)

Notes:
1. If there is a '/' at the end of the ftp line, you will be taken to the Cunix root directory instead of your home directory.
2. Do not use Microsoft Internet Explorer for this method because it will store your Cunix password in plain text in the browser history, leaving it for every other user of that computer to see!! A good alternative, found on all lab machines, is Firefox.

Command line SFTP

On a PC, open a command (DOS) prompt, navigate to the desired directory using cd and dir (the equivalent of Unix ls), and type:  
> sftp cunix.cc.columbia.edu

Or, alternatively type: 

sftp open cunix.cc.columbia.edu

If you are transferring any files that are not plain ascii text (i.e. executable programs, word docs, etc), you will want to make sure you're in binary mode. Before you transfer any files type: 

sftp> bin

You can navigate through your Cunix home directory, using the cd and ls commands. You will not be able to use tab-completion here. Use the get and put commands as described above like this: 

sftp> get cunixfile
sftp> put localfile

To exit, type:

sftp> quit

This method will also work when transferring files between two Unix systems.

Printing

To print to an AcIS printer use this command:

$ lpr -Pprinter_name filename

A full list of AcIS printer names (called Lpr server) and locations can be found here: http://www.columbia.edu/acis/facilities/printers/locations.html

Print quotas: Students with extended accounts get 100 pages per week. To check how many pages you have remaining in your quota type:

$ pages

Or, for more detailed information:

$ pages -v

To check what type of account you have, type:

$ groups

Your primary group is the first group that appears in the list. If this is student, then you have an extended account.

Find out more about account types and print quotas here http://www.columbia.edu/acis/facilities/printers/quota.html
Find out about upgrading your account here http://www.columbia.edu/acis/accounts/create/current.html

Columbia Computer Labs

Your Cunix account allows you to use the AcIS computer labs on campus: 251 Mudd and 213 Butler. These public labs are staffed with consultants that can assist you with non-homework related problems.

251 Mudd has both Solaris and Windows machines and is the best choice for coding; it usually has free machines (note: Sun workstations require an extended account). 213 Butler has mostly Windows PCs and some PowerMacs. 213 Butler is usually crowded.

These are the current public lab hours:

Student Labs

251 ET Lab
213 Butler Lab

Monday - Thursday:

9am - midnight
24 hours

Friday:

9am - 5pm
24 Hours

Saturday:

10am - 5pm
24 Hours

Sunday:

10am - midnight
24 Hours

Residence Halls Clusters

Broadway, Carman, East Campus, Furnald,
Hartley, McBain, Schapiro, Wien

24 Hours

Residence hall clusters and other electronic classrooms are un-staffed. In addition there are some electronic classrooms such as 407 Mathematics that are sometimes free for student use.

For more information about AcIS lab facilities see: http://www.columbia.edu/acis/facilities/labs/locations/