Unix intro
From Waisman Brain Imaging Wiki
| Table of contents |
A Quick and Dirty Intro to UNIX
This document is not intended as a UNIX manual of any sort; there are several of those in the computer room. Instead, it is a collection of handy tips and references to places to look for more information.
UNIX is a powerful, flexible OS (operating system) that allows you to do everything that you could do with Windows or a Mac, plus a lot more. It assumes that you know what you are doing, and that you are paying attention to what you are typing, and therefore will allow you to get into trouble a lot faster and more easily than a home computer will.
Intrinsic UNIX Philosophy
Small Is Beautiful: UNIX has a zillion little commands, one for every job. Most can be connected together to perform more complex tasks. Most commands under UNIX have very short, terse names, which is nice because you don't have to type as many letters, but often is confusing, because you have to remember some pretty goofy command names (such as "grep" for searching files for specific text, or "vi" for editing files).
The Command Prompt IS The Interface: UNIX originally only had a command-line prompt (like DOS). There was no Windows equivalent. Most UNIX commands were developed for this environment. While modern UNIX does have a desktop, menus, and window-based applications, you must know how to do things at the command prompt to get anything done.
Case Matters: Unlike Windows (and DOS), UNIX commands and filenames are case-sensitive. /exp/mystudy/myfile.txt and /EXP/MYSTUDY/MYFILE.TXT and /Exp/MyStudy/MyFile.Txt are all different to UNIX.
You KNOW What You Are Doing: UNIX was originally written by computer gurus for computer gurus. Most commands don't have any way to "undo" or "back up a step". When you accidentally delete a critical directory, it is gone. When you change a file, it is changed. Many commands run "silently" without telling you what they're doing. They assume that you *know* what they're doing.
Basic UNIX Commands
- pwd - What Directory Am I In? This command simply tells you which directory you are in. When you first log in, you will be in your "home" directory (this is the same as your "H:" drive under Windows):
LAN104% pwd
/export/home/users/yourname
- cd - Change Location To A Different Directory
cd ~ Change to your "home" directory. "~" is shorthand for your home directory.
cd Another way of changing to your home directory.
cd / Change to the top-level (root) directory. Note that, unlike Windows, disks don't each get their own directory; all disks are under / somewhere (even floppies and CD-ROM disks).
cd /exp/mystudy Change to my study under /exp. All experimental data is under /exp somewhere.
cd .. Move up one directory level (change to the current directory's "parent").
cd ../../whoohoo Move up two directory levels then into the "whoohoo" directory.
- more - Read A File This command lets you view the contents of a file:
LAN104% more myfile.txt
While viewing the file, press the "Enter" key to scroll down 1 line, the "Space" key to scroll down an entire screenful, and the "q" key to quit. - ls - What's In The Directory? This command shows the contents of a directory:
ls Show the contents of the directory that you are currently in.
LAN104% cd /exp/gadmri
/exp/gadmri
LAN104% ls
MRI/ README.txt forms/
ls -l Show the contents of the current directory in detail.
LAN104% ls -l
total 8
drwxrwxrwx 3 anderle lab 1024 Dec 9 02:40 MRI/
-rw-rw-rw- 1 joviko lab 244 Nov 22 16:27 README.txt
drwxrwxrwx 2 nitschke lab 2048 Dec 8 01:32 forms/
Whew! That's a lot of info! The first "d" on the line shows that the item is a directory ("-" means that it isn't). In this example, "MRI" and "forms" are both directories. "README.txt" is a file.
The permissions are shown right after the "d". There are 3 sets of 3 specific permissions, all bunched together. The first set is for the owner; the second is for the group; and the third is for anyone and/or everyone. Within each set, "r" means that the file can be read; "w" means that it can be written to, modified, created, and deleted; and "x" means that it can be moved in to (if it is a directory) or executed (if it is a program). Here are some more examples:
drwx------ This is a directory that only the owner can do anything in.
-rw-rw---- This is a file that only the owner and group can view/edit.
-rw-r--r-- This is a file that the owner can edit, all others only view.
After the directory/file flag and the permissions comes the number of links. In this example, "MRI" has 3 links, "README.txt" has 1, and "forms" has 2. You can ignore these for now.
Next comes the owner and group. "forms" is owned by "nitschke", and is associated with the "lab" group.
The size of the item (in bytes) comes next. "README.txt" is 244 bytes long.
The date and time that the item was last modified is next. "README.txt" was created (or last modified) on November 22 (of last year) at 4:27 PM.
The final field is the name of the item.
ls -l | more Show the detailed contents of the directory that you are currently in. Send (or "pipe") the output to the 'more' command, so that the listing for a big directory doesn't just scroll off the screen.
ls /exp/mystudy Show the contents of the /exp/mystudy directory.
- mv - Move A File Or Directory Here's how to rename "yow.abc" to "yow.def":
LAN104% mv yow.abc yow.def
This command is similar to, but definitely not the same as the "move" or "ren" commands under DOS. An important difference is that you can't "block rename" a group of files. For example:
LAN104% mv *.abc *.def
will *not* rename every file that ends in ".abc" to end with ".def". (We'll cover how to do this next time.)
However, it is capable of moving entire directory trees--even from one disk to another (be sure that the receiving disk has enough space before trying this). Let's say that you have a directory called "Results" which you want to move to /exp/BISPIL:
LAN104% mv Results /exp/BISPIL
That's all there is to it. There will now be a complete /exp/BISPIL/Results directory, and the copy in the current directory will be gone.
- cp - Copy A File Or Directory 'cp' works almost exactly like 'mv', except that it copies files instead of moving them. One difference is when you want to copy an entire directory tree (a directory and everything under it) from one place to another. To do this, for example:
LAN104% cp -r Results /exp/BISPIL
The "-r" flag tells 'cp' to copy the directory recursively; that is, to copy the directory and everything underneath it.
- rm - Remove A File Or Directory 'rm' works like 'mv' and 'cp', except that it deletes files--PERMANENTLY. There is no undo for 'rm' and it works very quickly and efficiently. You can remove a huge directory full of crucial files in seconds; the command will finish long before you've realized what you've done.
ALWAYS BE PARANOID WITH 'rm'!!!
To delete "abc_def", do this:
LAN104% rm abc_def
To remove the "final_results" directory and everything under it in the blink of an eye:
LAN104% rm -r final_results
Once again, ALWAYS BE PARANOID WITH 'rm'!!!
- lp - Print A File Printing under UNIX is a topic all to itself. Printing options that are relatively simple under Windows (printing on both sides of a page, printing tranparencies to the color printer, etc.) can get quite complex. Here's how to print a file on the default (black-and-white) printer:
LAN104% lp tiger.ps
request id is 465
Efficient File Management (Living Without Windows)
Making Typing Easier -- Filename And Command Completion Using UNIX means typing in some long, complex commands, often times over and over again. On the lab computers, you can use the following keys to make this easier:
- Arrow Keys -- Use the "Up Arrow" key to recall a prior command. You can then use the "Left Arrow" and "Right Arrow" keys to move around within the command and use the "Backspace" key and others to modify it. Press the "Enter" key when it is tweaked to your satisfaction and ready to run.
- Tab Key You can type in the first few letters of a command or filename, then press the "Tab" key, and UNIX will complete the name for you. This can be incredibly handy if your file names are really long or have spaces in them.
- Control-D If the computer beeps and just sits there after you type in a few letters and press "Tab", there could be more than one file that starts with the letters you've entered. Press the "Control" key and the "d" key together, and UNIX will display a list of all of the possible matches (or nothing if you've mis-typed, and there are no matches). You can then type one or more additional letters to uniquely select a file, then press "Tab" again to have UNIX complete the name.
Permissions and How To Change Them -- So, you've created a file, but no one else in the lab can get to it or edit it. (This is very common after you've used 'Fetch' on the Mac or 'FTP' on a PC to transfer files to the lab computers.) What's wrong? The file probably has the wrong permissions on it. To check, use the 'ls -l' command on the file. For example,
LAN104% ls -l tiger.ps
-rw------- 1 joviko lab 78519 Sep 8 1999 tiger.ps
In this example, only "joviko" can see or modify this file. To fix this sort of problem, you need to use the 'chmod' (or "change permission mode") command. Here are some examples: LAN104% chmod a+rw tiger.ps
This example allows everyone (owner, group, and others) to read and modify "tiger.ps".
LAN104% chmod a+rwx /exp/BISPIL/mydir/
This example gives everyone permission to use the "mydir" directory.
LAN104% chmod -R a+rw /exp/BISPIL/mydir/
This example gives everyone permission to view and modify all files anywhere under the "mydir" directory.
Selecting Groups of Files -- ls * List all of the files in the current directory. "*" is shorthand for "match any file".
ls *.txt List all of the files which end with ".txt"
Note: since UNIX is case-sensitive, files which end in ".TXT" will *not* be shown.
ls *.* List all of the files which contain a "." somewhere in the name. Files which do not contain a period in the name will not be selected. UNIX files don't need to have a period or extension like under DOS.
ls *148* List all of the files which have the string "148" somewhere in their name.
ls *[148]* List all of the files which have "1", "4", or "8" somewhere in their name.
ls *e[3-59]* List all of the files which have an "e", followed by any one of"3", "4", "5", or "9" in their name.
A Note On Aliases -- Sometimes when you use a UNIX command, it doesn't act exactly like you'd expect it to. This can be caused by an "alias", which is similar to a macro or keyboard accelerator in that it is a a translation from what you type to a more complex command.
To see if a command is aliased, simply type:
LAN104% alias | more
and see if the command is in the list. Many of the basic commands that can get you into trouble such as 'rm' and 'mv' are aliased at the lab to confirm your actions. To temporarily get around an alias without turning it off completely, put a "\" character before the command: LAN104% \mv my_critical_directory /very_small_disk/my_important_stuff
Of course, you should always BE CAREFUL AND THINK when doing this!
Disk Space: Used And Free
- df - How Much Space Is Free On The Disk? This command shows how much space is free on a specific physical disk. This is important if you're planning on running a command that will generate a lot of big files, and you want to make sure that it won't fill the disk that your study resides on. Here's how to see how much space your study has available: LAN104% df -k /exp/BISPIL Filesystem 1k-blocks Used Available Use% Mounted on lan105:/da 307294560 268788913 18966994 94% /da Note that you should always use the full directory path--/exp will have a different (and much smaller) result than /exp/BISPIL. (This is because /exp/BISPIL "lives" on a different disk than its parent /exp.) The two numbers of interest are the "Available" and "Use%" columns. The "Available" number is the amount of free or available space on this disk in K (1024 bytes). In this example, there are 18966994 K free, or about 18 gigabytes. That's a lot, but a single MRI analysis job can use up a good chunk of it. The "Use%" number shows roughly what fraction of the total disk has been used. In this case, it is 94% used up, so only 6% of the disk is free. That's a good indicator that researchers need to free up some space by archiving or deleting things that they've finished working on. Note that the "-k" option tells 'df' to report the available space in K, rather than in disk blocks.
- du - How Much Space Does My Directory Use? This command shows you how much disk space an entire directory (including all files and subdirectories in it) is using up. Since it needs to search the directory and add up all of the subtotals, it can take a few minutes to run on large directories: LAN104% cd /exp/BISPIL /exp/BISPIL LAN104% du -sk . 10892062 . In this example, /exp/BISPIL is using up 10892062 K, or about 10 gigabytes. Note that you should always 'cd' to the directory that you're interested in, then do a 'du' of the current directory. That way, you won't get a false summary caused by symbolic links (more about them next time). Note also that the "-sk" flags tell 'du' to report a summary of everything in the directory (it will list each subdirectory individually otherwise); and to report the total in K.
Searching For Files
find - Winner Of The Most Confusing Command Award Here is how to find a file in the current directory that has "123" somewhere in its name. Type it *exactly* like this:
find . -name \*123\* -print
Yuck, huh? Here's how to find any files that end in ".elf" under /exp/BISPIL:
find /exp/BISPIL -name \*.elf -print
How to list any files in the current directory that have been created or modified within the past 10 days. Once again, type it exactly like this:
find . -mtime -10 -ls
And here's how to list the files which have *not* been modified within the past 10 days (change the "-10" to "+10"):
find . -mtime +10 -ls
rgrepl - Find The File Which Contains This Text This command is unique to the lab. It lists any files which contain the text listed on the command line. This example lists all of the files in /d2/pccommon/joviko which contain the string "test" somewhere within them:
LAN104% cd /d2/pccommon/joviko LAN104% rgrepl test ./mail/mail$43c9e90a0005009b.mai;1 ./mail/mail$45780fec0005009b.mai;1 ./mail/mail$4975281a0005009b.mai;1 ./todo_gud.txt ./joviko/udec/udec.faq ./joviko/udec/register.for ./joviko/spite.txt ./todo.txt
rgrepil - Find The File Which Contains This Case-Insensitive Text 'rgrepil' works just like 'rgrepl', except that it is case-insensitive. In the previous example, it would list any files which contained "test" or "TEST" or "tExT", etc.
Computer / Network Interactions
X Connections You can use several different methods to "log in" to a lab UNIX computer (for example 'telnet' and 'ssh'). Most of these are fine for doing many tasks under UNIX, but they don't allow you to run graphical programs such as SPAMALIZE, SPM, or AFNI. To do this, you need to make an "X" connection. X is short for XWindows, the graphical system used under UNIX that is the equivalent to Windows on PCs and the Desktop on Macs.
The PCs and Macs in the lab are all set up to make connections via X-compatible programs. When you double-click on the "LAN104" icon on your desktop, a program such as Exceed (the one with the hummingbird) or XWin32 is started up, and makes an X-friendly connection to LAN104.
One of the nice things about having an X-based login is that you can easily open up more terminal windows, so that you can run multiple commands at once. You do this by typing in:
LAN104% xterm &
which tells LAN104 to start another terminal window (an 'xterm'), and to run it separately ("in the background" which is what the "&" character means) from your login terminal window. You can do this several times to quickly open many windows.
Where To Run A Command
Copying Files - You may have noticed that it can take a long time to copy or move files from one place to another on the W: drive under Windows. This is because the W: drive is actually a UNIX disk "pretending" to be a Windows drive. It has to translate from Windows to UNIX and back for every file that is moved or copied. When the UNIX servers are busy, simple copies can slow to a crawl.
The same command will take only a few seconds under UNIX, because the there is no translation involved. So, when you need to move an entire directory tree, or copy the contents of a CD or floppy (more about them next time), do it under UNIX rather than Windows.
When Things Get Busy - LAN104 is the main UNIX computer used by the entire lab to process and analyze MRI and PET data. During "lab rush hour" (roughly 10 AM to 4 PM), LAN104 can get swamped. You can see who and what are causing the slowdown by running the 'top' command. It will show you the programs running on LAN104, with the biggest CPU users at the top. Don't leave 'top' running for a long time, as it is a bit of a CPU hog itself!
The lab has other UNIX computers that you can use when LAN104 is busy. LAN103 is a Sun just like LAN104. LAN101, and LAN106 - LAN109 are similar, but they run the Linux flavor of UNIX instead of Solaris (more about the subtle differences next time). To see which UNIX computers are less busy, run the 'labbusy' command, which shows how busy each computer is. The bigger the number to the right of the computer name, the less busy it is.
If there isn't an icon for a computer your desktop, you can log in to LAN104 and then run
LAN104% rlogin LAN106
to log into one of the less busy computers. If you want to run a graphical program, you'll need to tell X how to find your PC by setting the DISPLAY environment variable. Look for the computer name on the front of your PC (for example, LAN345), then, after you've logged in, type in this command: LAN106% setenv DISPLAY LAN345:0.0
Don't forget the ":0.0" on the end; X needs this to know which display on your computer to connect to--UNIX computers often have several displays, so even though your PC has only one, you still need to tell X this.
Alternatively, many of the PCs in the lab can "dual boot" into either Windows or Linux UNIX. To see if the one that you're sitting at can do this, tell Windows to reboot. Now, watch carefully while the PC boots. If it stops for several seconds and displays this on the screen:
lilo:
then it can boot into Linux. To get it to do so, quickly type "linux":
lilo: linux
and you'll have an entire UNIX computer to yourself. You'll need to do this in order to 'cp' CDs and floppies to the lab servers.
Helpful Books
The lab "library" in 365 The lab has a handful of UNIX starter books. They tend to be borrowed most of the time, so you might consider one of these other options:
- Learning the Unix Operating System This is one of the great O'Reilly "nutshell" (or "animal") books. You can recognize them by the animal on the cover. This one is probably the best single introduction to UNIX that you can buy.
- UNIX® For Dummies® Yeah, it has that insulting title. It is still a pretty good intro to UNIX, and might be easier to find at an actual bookstore than the O'Reilly book.
by joviko@gmail.com
