Unix

STUDY OF UNIX OS A computer uses a set of programs, generally called the operating system, to manage its hardware resources (memory, disks, displays, input devices, etc.) on behalf of the user. UNIX is an example of such a system. It was originally developed as a research project at AT&T Bell Labs in 1969. It is a stable, multi-user, multi-tasking system for servers, desktops and laptops. UNIX systems also have a graphical user interface (GUI) similar to Microsoft Windows which provides an easy to use environment.  UNIX was designed to be easily ported (moved from one hardware platform to another). It has been ported to everything from desktop computers to room-sized supercomputers.  UNIX allows many people to share the resources of a single computer simultaneously.  UNIX allows users to run multiple programs at once. Unlike most desktop operating systems, UNIX imposes strict constraints which keep ill-behaved programs from affecting other programs and the operating system itself. There are many different versions of UNIX, although they share common similarities. The most popular varieties of UNIX are Sun Solaris, GNU/Linux, and MacOS X. The UNIX operating system is made up of three parts.  The kernel  The shell  The programs. The kernel The core of the UNIX system. It is loaded at system start up (boot). It is a memory-resident control program. It manages the entire resources of the system, presenting them to you and every other user as a coherent system. It Provides service to user applications such as device management, process scheduling, etc. Example functions performed by the kernel are: • Managing the machine's memory and allocating it to each process. • Scheduling the work done by the CPU so that the work of each user is carried out as efficiently as is possible. • Accomplishing the transfer of data from one part of the machine to other • interpreting and executing instructions from the shell • Enforcing file access permissions. The shell We can interact with UNIX through a special program called the shell. The shell prompts you for commands and hands these off to the operating system to be executed after you have typed them in. Commands are composed of two parts:  the name of the command itself  Arguments.

The UNIX shell comes in two major flavors: the Bourne shell, sh , and the C shell, csh. Other popular shells are derived from these (e.g., ksh, bash, tcsh). Each shell uses a particular character (or string of characters) to prompt the user for commands. The shell acts as an interface between the user and the kernel. When a user logs in, the login program checks the username and password, and then starts another program called the shell. The shell is a command line interpreter (CLI). It interprets the commands the user types in and arranges for them to be carried out. The commands are themselves programs: when they terminate, the shell gives the user another prompt. Files and processes Everything in UNIX is either a file or a process. A process is an executing program identified by a unique PID (process identifier). A file is a collection of data. They are created by users using text editors, running compilers etc. Examples of files:  a document ,the text of a program written in some high-level programming language  instructions comprehensible directly to the machine and incomprehensible to a casual user,a directory, containing information about its contents, which may be a mixture of other directories (subdirectories) and ordinary files. The Directory structure All the files are grouped together in the directory structure. The file-system is arranged in a hierarchical structure, like an inverted tree. The top of the hierarchy is traditionally called root (written as a slash / )

In the diagram above, we see that the home directory of the undergraduate student "ee51vn" contains two sub-directories

(docs and pics) and a file called report.doc. The full path to the file report.d Basic UNIX Commands

The list below presents some examples of basic UNIX commands. Files 

ls --- lists your files ls -l --- lists your files in 'long format', which contains lots of useful information, e.g. the exact size of the file, who owns the file and who has the right to look at it, and when it was last modified. ls -a --- lists all files, including the ones whose filenames begin in a dot, which you do not always want to see. There are many more options, for example to list files by size, by date, recursively etc.

Example % ls lists all of the files in your current directory, except those beginning with a '.' (period). % ls -a lists all the files in your current directory, including those beginning with a period (such as .login) % ls /usr/users lists all the files in the /usr/users directory. 

more filename --- shows the first part of a file, just as much as will fit on one screen. Just hit the space bar to see more or q to quit. You can use /pattern to search for a pattern.

Example % more gamma.seq displays the gamma.seq file on your terminal, one screen at a time.

File Creation/Deletion    

cat filename --- user can create a file. mv filename1 filename2 --- moves a file (i.e. gives it a different name, or moves it into a different directory (see below) cp filename1 filename2 --- copies a file rm filename --- removes a file. It is wise to use the option rm -i, which will ask you for confirmation before actually deleting anything.

Example % cat temp.txt gamma.seq > new.seq combines the temp.txt and gamma.seq files and puts the result in the new.seq file. % cp gamma.seq temp.txt creates a new file -- or writes over an existing one -- called temp.txt that contains exactly the same data as the gamma.seq file. % mv gamma.seq temp.txt changes the name of the gamma.seq file to temp.txt, writing over the contents of temp.txt, if any. % rm gamma* deletes all files that begin with the characters "gamma".

File Manipulation  

diff filename1 filename2 --- compares files, and shows where they differ wc filename --- tells you how many lines, words, and characters there are in a file

Directories Directories, like folders on a Unix, are used to group files together in a hierarchical structure.  



mkdir dirname --- make a new directory cd dirname --- change directory. You basically 'go' to another directory, and you will see the files in that directory when you do 'ls'. You always start out in your 'home directory', and you can get back there by typing 'cd' without arguments. 'cd ..' will get you one level up from your current position. You don't have to walk along step by step - you can make big leaps or avoid walking around by specifying pathnames pwd --- tells you where you currently are.

Example % mkdir project creates a subdirectory called project off your current directory. % pwd displays the full path of your current directory. Printing 

 

lpr filename --- print. Use the -P option to specify the printer name if you want to use a printer other than your default printer. For example, if you want to print double-sided, use 'lpr -Pvalkyr-d', or if you're at CSLI, you may want to use 'lpr -Pcord115-d'. See 'help printers' for more information about printers and their locations. lpq --- check out the printer queue, e.g. to get the number needed for removal, or to see how many other files will be printed before yours will come out lprm jobnumber --- remove something from the printer queue. You can find the job number by using lpq. Theoretically you also have to specify a printer name, but this isn't necessary as long as you use your default printer in the department.

Example % lpr project Print the content of file name project in the printer. Help (UNIX On-Line Documentation) 

man commandname --- shows you the manual page for the command

You can find out more about these commands by looking up their manual pages Example % man ls displays the on-line manual pages for the ls command. Ending a Session 

logout --- Close your Unix session

Example % logout ends your UNIX session. Operating System Information 

finger --- To known your Operating system details

Example % finger smith displays the shell you are using (where "smith" is your username), as well as other information. Searching File for Text Patterns 



ff --- find files anywhere on the system. This can be extremely useful if you've forgotten in which directory you put a file, but do remember the name. In fact, if you use ff -p you don't even need the full name, just the beginning. This can also be useful for finding other things on the system, e.g. documentation. grep string filename(s) --- looks for the string in the files. This can be useful a lot of purposes, e.g. finding the right file among many, figuring out which is the right version of something, and even doing serious corpus work. grep comes in several varieties (grep, egrep, and fgrep) and has a lot of very flexible options. Check out the man pages if this sounds good to you.

Example % grep smithies gamma.seq looks through gamma.seq for the "smithies" character pattern and displays the lines that match the pattern.

Miscellaneous tools   

webster word --- looks up the word in an electronic version of Webster's dictionary and returns the definition(s) date --- shows the current date and time. cal --- shows a calendar of the current month. Use e.g., 'cal 10 1995' to get that for October 95, or 'cal 1995' to get the whole year.

Example % cal 2009 It displays entire 12 months of 2009 calendar About other people 







w --- tells you who's logged in, and what they're doing. Especially useful: the 'idle' part. This allows you to see whether they're actually sitting there typing away at their keyboards right at the moment. who --- tells you who's logged on, and where they're coming from. Useful if you're looking for someone who's actually physically in the same building as you, or in some other particular location. finger username --- gives you lots of information about that user, e.g. when they last read their mail and whether they're logged in. Often people put other practical information, such as phone numbers and addresses, in a file called .plan. This information is also displayed by 'finger'. last -1 username --- tells you when the user last logged on and off and from where. Without any options, last will give you a list of everyone's logins.

Example % finger smith displays the shell you are using (where "smith" is your username), as well as other information.

About your (electronic) self  

 



whoami --- returns your username. Sounds useless, but isn't. You may need to find out who it is who forgot to log out somewhere, and make sure *you* have logged out. finger & .plan files of course you can finger yourself, too. That can be useful e.g. as a quick check whether you got new mail. Try to create a useful .plan file soon. Look at other people's .plan files for ideas. The file needs to be readable for everyone in order to be visible through 'finger'. passwd --- lets you change your password, which you should do regularly. ps -u yourusername --- lists your processes. Contains lots of information about them, including the process ID, which you need if you have to kill a process. Be careful not to kill your files. kill PID --- kills (ends) the processes with the ID you gave. This works only for your own processes, of course. Get the ID by using ps. If the process doesn't 'die' properly, use the option -9.



du filename --- shows the disk usage of the files and directories in filename (without argument the current directory is used). du -s gives only a total.

Example % kill 9145 It will kill 9145 process ID file. % du agni It display the sizes of files and directories. Connecting to the outside world 

  

nn --- allows you to read news. It will first let you read the news local to turing, and then the remote news. If you want to read only the local or remote news, you can use nnl or nnr, respectively. rlogin hostname --- lets you connect to a remote host telnet hostname --- also lets you connect to a remote host. Use rlogin whenever possible. ftp hostname --- lets you download files from a remote host which is set up as an ftp-server. This is a common method for exchanging academic papers and drafts

VI EDITOR COMMANDS vi filename (press Enter/Return key) Opens the named file if it exists, or creates new file and opens it. Cursor Motion Commands Esc h- move back 1 character Esc l- move forward 1 character Esc j- move down 1 line Esc k- move up 1 line G(uppercase) - Moves cursor to the last line in the file. 1G (number one and uppercase G) - Moves cursor to the first line in the file. Commands to put you in text insert mode Note: After inserting text, press Esc (escape) key to return to command mode. Esc a (lowercase) - Use to insert (append) text to the right of cursor. Esc A (uppercase) - Use to insert (append) text at the end of the current line. Esc i (letter i (eye) lowercase) - Use to insert text to the left of cursor. Esc I (letter I (eye) uppercase) - Use to insert text at the beginning of the current line. Esc o (letter o lowercase) - Use to open a new line for inserting text below cursor line. Esc O (letter O uppercase) - Use to open a new line for inserting text above cursor line Commands to delete text x (lowercase) - Removes the character under the cursor. dw (lowercase) - Deletes word under and to right of cursor. dd (lowercase) - Deletes the cursor line. D (uppercase) - Deletes from cursor to end of line.

Commands to change text Note: After making change, press Esc (escape) key to return to command mode. Esc cw (lowercase) Changes word under and to right of cursor to new text you enter. Esc cc (lowercase) Changes current line to new text you enter. Esc s (lowercase) Substitutes character under cursor with new text you enter. Esc S (uppercase) Substitutes current line with new text you enter. Esc C (uppercase) Changes text from cursor to end of line to text you enter. Esc r (lowercase) Replaces character under cursor with new character. Esc ~ (tilde) Changes case of character under cursor

Commands to delete text Esc x (lowercase) Removes the character under the cursor. Esc dw (lowercase) Deletes word under and to right of cursor. Esc dd (lowercase) Deletes the cursor line. Esc D (uppercase) Deletes from cursor to end of line

Command to join two lines Esc J (uppercase) Joins next line to the end of the cursor line. Commands to copy lines and put them in another location Esc yw (lowercase) - Yanks (copies) word under and to right of cursor. Esc Y (uppercase) Yanks (copies) line the cursor is on. Esc p (lowercase) Puts last word or line yanked or deleted after cursor. Esc P (uppercase) Puts last word or line yanked or deleted before cursor. Commands to save changes or leave vi editing session Esc – Shift :w (colon and lowercase w) Saves changes, but does not leave vi editing session. Esc – Shift :wq (colon and lowercase wq) Saves changes, and leaves vi editing session. Esc – Shift ZZ (uppercase) Saves changes, and leaves vi editing session. Esc – Shift :q! (colon, lowercase q, and exclamation) Does not save changes made. Just leaves vi editing session.

DIRECTORY CREATION Command Syntax mkdir [OPTION] DIRECTORY $mkdir <path>/ $mkdir –m $mkdir –p // Example: – mkdir project1 This creates a directory project1 under current directory Note: Write and execute permissions are needed for the user to create a directory Create the DIRECTORY(ies), if they do not already exist. -m, --mode=MODE set permission mode -p, --parents no error if existing, make parent directories as needed --verbose print a message for each created directory Making a directory – The mkdir command Directories can be created by using the mkdir (make directory) command. $ mkdir mydir A directory mydir is created under the current directory. A number or directories can be created as follows $ mkdir mydir mydir/progs mydir/data $mkdir –p mydir/mydir1/mydir2/mydir3 Creates directories hierarchically below each directory $mkdir –m 755 project Creates directory “project” and sets read, write, execute permissions for owner and read & execute permissions for group and others. Directory removal rmdir command removes directory Syntax – rmdir Example Removes project1 directory in the current directory – rmdir project1 Remove multiple directories rmdir pos1 pos2 Remove the directory recursively

rmdir –p dir1/dir2/dir2 Rule: rmdir can be executed to remove a directory if it is empty and not the current directory

Command - cd cd command is used to change the directory • • • • < file

cd cd .. cd /

- take to the home directory - takes to the parent directory - takes to the root directory

I/O redirection redirect standard input from file > file redirect standard output to file 2> file redirect standard error to file 2>&1 merge standard error with standard output $ cat > abc $ ls –l > outfile $ cat xyz abc > outfile 2> errfile $ cat xyz abc > outfile 2>&1

Eg: $ cat file2 This command is used to create file2, if it does not exist or to overwrite file2. In either case cat takes input from keyboard and writes the output to standard output, which is redirected to file2 by shell. $ cat file3 cat: file3: No such file or directory In this case cat sends error messages to standard error. To redirect this error to a file, we can use 2> $cat file3 2>error This command will create a file error and will store the standard error in it. Merging standard output and standard error $cat file2 >file4 2>&1 Both standard output and standard error are sent to file4. -c

grep options displays count of the number of occurrences

-n

displays line numbers along with the lines

-v

displays all lines except lines matching pattern

-i

Ignores case for matching

To find the number of files that are soft links in the current directory $ ls –l | grep –c “^l” To get long list of all files, excluding directories $ ls –l | grep –v “^d”

PIPE AND FILTERS The purpose of Pipes and Filters is to construct powerful Unix command lines by combining Unix commands. Unix commands alone are powerful, but when you combine them together, you can accomplish complex tasks with ease. The way you combine Unix commands is through using pipes and filters Using a Pipe The symbol | is the Unix pipe symbol that is used on the command line. What it means is that the standard output of the command to the left of the pipe gets sent as standard input of the command to the right of the pipe. Here is an example $ cat apple.txt My college Name is Crescent Engg college $ cat apple.txt | wc 3 7 40 $ In this example, at the first shell prompt, I show the contents of the file apple.txt to you. In the next shell prompt, I use the cat command to display the contents of the apple.txt file, but I sent the display not to the screen, but through a pipe to the wc (word count) command. The wc command then does its job and counts the lines, words, and characters of what it got as input.

You can combine many commands with pipes on a single command line. Here's an example where I count the characters, words, and lines of the apple.txt file, then mail the results to [email protected] with the subject line "The count." $ cat apple.txt | wc | mail -s "The count" [email protected]

Using a Filter A filter is a Unix command that does some manipulation of the text of a file. Two of the most powerful and popular Unix filters are the sed and awk commands. Both of these commands are extremely powerful and complex.

$ cat apple.txt core worm seed jewel $ cat apple.txt | sed -e "s/e/WWW/" corWWW worm sWWWed jWWWwel $ cat apple.txt | sed -e "s/e/J/g" corJ worm sJJd jJwJl $ In this example, at the first shell prompt, I showed you the contents of the apple.txt file. At the second shell prompt, I used the cat command to display the contents of the apple.txt file, and send that display through a pipe to the sed command. The sed command I created changed the first occurrence of the letter "e" on each line to "WWW." The sed took as input the information it got through the pipe. The sed command displayed its output to the screen. I then used the output of the cat command on the apple.txt file and sent it by a pipe to the sed command to change all the occurrences of an e on each line with J. Note that every occurence of e, even where there were more than one on a line, changed to J. This is because of the "g" on the end of the sed option value string. This "g" stands for global replace. It is important to note that, in this example, the contents of the apple.txt file itself were not changed in the file. Only the display of its contents changed. grep The Unix grep command helps you search for strings in a file. Here is how I can find the lines that contain the string "jewel" and display those lines to the standard output: $ cat apple.txt core worm seed jewel

$ grep jewel apple.txt jewel $

How is pipe different from a filter? A pipe (or "|") is used to redirect output to another command. For example, ls -la | more Will pipe output to the more command letting view the directory listing one page at a time. A redirector (">" or ">>" or "<" or "<<" normally tells where to output or take input from another file. ls -la > filename This will write the output of the ls command to a file called "filename”.

FILE OPERATIONS

1. Type To type out a file use the cat (short for concatenate) command Eg.:- cat myfile cat can be used to combine (concatenate) files . For example:- cat f1 f2 >f3 ,concatenates files f1 and f2 into the file f3. The command more types out a file one screenful at a time Eg.:- more myfile The commands head and tail can be used to type the first or last few lines of a file, eg:tail -3 myfile , types out the last three lines of myfile. 2. Edit A basic full screen editor is vi (visual editor). To edit a file called myfile, vi myfile vi operates in two modes: passive mode and insert mode. When you first enter vi it is in passive mode. Passive mode In passive mode the user can move round the file using the arrow keys. To delete the character pointed to by the cursor press x and to delete the character to the left of the cursor press X. To switch to insert mode press i. Insert mode In insert mode new material can be typed in. The delete key can be used to remove text added since insert mode was last selected although the text won't disappear from the screen until further text is typed in or passive mode is selected. To leave insert mode and return to passive mode press the escape key (Esc). 3. Print To print a file use the lpr command Eg.:- lpr myfile ,This goes to the default printer.

4. Copy To copy a file use cp Eg.:- cp f1 f2 copies file f1 into f2. If f2 already exists it will be replaced, use the -i option to get UNIX to ask for confirmation before replacing existing files. If f2 is a directory rather than a file then f1 is copied to that directory preserving the file name f1. 5. Rename To rename a file use the mv (short for move) command, Eg.:- mv f1 f2 , renames f1 to f2 6. Delete To delete a file use the rm (short for remove) command, Eg.:- rm f1 ,deletes the file f1. 7. chmod - Change protection on file Change r - read, w - write and x - execute access to u - user, g - group and o - others. Egs:chmod u+w file1 Give user write-access to file1. chmod g+r file2 Give group read-access to file2. chmod o-w file3 file 4 Withdraw write-access to file3 and file 4 from others. chmod -x file5 Give everyone execute-access to file5. 8. diff - Show differences between files Compare two (or three) files. Eg:- diff file1 file2 ,Compare file1 to file2

9. find - Find files in a directory tree Eg:- find ./ -name "*.cxx" Print all files named *.cxx within directory tree starting at ./ 10. grep - Search file for character string Search for one or more strings in one or more files. Egs:grep that myfile.txt Look for the string ``that'' in the file called ``myfile.txt'' and print out each line that matches. egrep -in "this|that" *.dat Extended grep search *.dat files for ``this'' or ``that'' case insensitive (-i) and where found print line number (-n) along with the line contents. 11. locate - Find file anywhere in the system Eg:-locate myfile List any file on the system that contains the name ``myfile''. 12. more - List file one screen at a time Eg:-more big.file Lists the file ``big.file'' on screenful at a time. To move to the next screen press space and to move to the next line press the RETURN key. To exit press ``q''. 13. wc - Count characters, words or lines is a file Eg:-cat myfile | wc -l List the number of lines in the file ``myfile''.