Linux Introduction: E-bridge Technologies

Linux Introduction E-Bridge Technologies

What is OS…?  An interface between users and hardware - an environment "architecture”,  Allows convenient usage; hides the tedious stuff,  Allows efficient usage; parallel activity, avoids wasted cycles,  Provides information protection,  Gives each user a slice of the resources,  Acts as a control program.

E-Bridge Technology

OS Overview

E-Bridge Technology

History of LINUX  In 80’s, Microsoft’s DOS was the dominated OS for PC.  Apple MAC was better, but expensive.  UNIX was much better, but much, much more expensive. Only for minicomputer for commercial applications.  People was looking for a UNIX based system, which is cheaper and can run on PC.  Both DOS, MAC and UNIX were proprietary, i.e., the source code of their kernel is protected.  No modification is possible without paying high license fees.

E-Bridge Technology

History Multics – 1964 Unics – 1969 Minix – 1990 Linux – 1991

Multics  Multiplexed Information and Computing Service  Written in 1964  Timesharing OS  Last version was shut down on October 30, 2008  Monolithic kernel.

Unics  Uniplexed Information and Computing System  Later renamed as UNIX  Written in 1969  Ken Thompson, Dennis Ritchie were among the developers  Multi user, Multi tasking and timesharing  Monolithic kernel

Minix  Minimal Unix  Tanenbaum developed this OS  Mainly for educational purpose  Unix like OS, implemented with Micro kernel. So the name Minix.

Linux  Developed in 1991 by Linus Torvalds  Used in most of the computers, ranging from super computers to embedded system  Multi user  Multi tasking  Time sharing  Monolithic kernel  Latest stable version of linux kernel – 2.6.28, released on 24Dec-2008

Beginning of Linux A Famous professor Andrew Tanenbaum developed Minix, a simplified version of UNIX that runs on PC. Minix is for class teaching only. No intention for commercial use. In Sept 1991, Linus Torvalds, a second year student of Computer Science at the University of Helsinki, developed the preliminary kernel of Linux, known as Linux version 0.0.1.

E-Bridge Technology

Growing and growing…

 In order to encourage wide dissemination of his OS, Linus made the source code open to public. At the end of 1992 there were about a hundred Linux developers. Next year there were 1000. And the numbers multiplied every year.  Recent estimates say about 29 million people use Linux worldwide. E-Bridge Technology

Linux Free Software Free software, as defined by the FSF (Free Software Foundation), is a "matter of liberty, not price." To qualify as free software by FSF standards, you must be able to: Run the program for any purpose you want to, rather than be restricted in what you can use it for. View the program's source code. Study the program's source code and modify it if you need to. Share the program with others. Improve the program and release those improvements so that others can use them.

E-Bridge Technology

Linux Distributions  Red Hat Linux : One of the original Linux distribution. The commercial, non free version is Red Hat Enterprise Linux, Free version: Fedora Project.  Debian GNU/Linux : A free software distribution. Popular for use on servers. However, Debian is not what many would consider a distribution for beginners, as it's not designed with ease of use in mind.  SuSE Linux : SuSE was recently purchased by Novell. This distribution is primarily available for pay because it contains many commercial programs, although there's a stripped-down free version that you can download.  Mandrake Linux  Gentoo Linux E-Bridge Technology

Linux Concepts

1.OS Features:     

Process Management Memory Management I/O Management File Management Network Management

E-Bridge Technology

Operating System

E-Bridge Technology

2.Kernel  Core or nucleus of an operating system  Interacts with the hardware  First program to get loaded when the system starts and runs till the session gets terminated  Different from BIOS which is hardware dependent.  Kernel is software dependent

E-Bridge Technology

Kernel Types  Monolithic  All OS related code are stuffed in a single module  Available as a single file  Advantage : Faster functioning  Micro  OS components are isolated and run in their own address space  Device drivers, programs and system services run outside kernel memory space  Supports modularity  Lesser in size E-Bridge Technology

3.Shell    

Program that interacts with kernel Bridge between kernel and the user Command interpreter User can type command and the command is conveyed to the kernel and it will be executed  Bash is the default for Linux

E-Bridge Technology

Shell Types      

Sh – simple shell BASH – Bourne Again Shell KSH – Korne Shell CSH – C Shell SSH – Secure Shell To use a particular shell type the shell name at the command prompt.  Eg $csh – will switch the current shell to c shell  To view the available shells in the system, type cat /etc/shells at the command prompt  To view the current shell that is being used, type echo $SHELL at the command prompt E-Bridge Technology

Text editors     

Vi Emacs gEdit kWrite TextPad

 And more…

E-Bridge Technology

C Program in Linux  Open a file with extension .c from the command prompt using vi editor  vi hello.c  Type the contents and save (Esc : wq!)  Compile the file  gcc hello.c  Run the executable  ./a.out

E-Bridge Technology

Commands: man pages  The man command allows you to access the MANual pages for a UNIX command.  To get additional help on any of the commands listed below, you can always type man name_of_command at the command prompt.  Examples:  man ssh  man passwd

E-Bridge Technology

Commands 

 

 

ls : lists the contents of a directory  l : long directory listing  a : lists all files, including files which are normally hidden  F : distinguishes between directories and regular files  h : ? Look it up using man pwd : prints the current working directory cd : changes directories  The difference between relative and absolute paths.  Special characters ., .., and ~. mkdir : creates a directory rmdir : removes a directory (assuming it is empty)  If you get an error that the directory isn’t empty even though it looks empty, check for hidden files. E-Bridge Technology

Commands  touch : creates an empty file with the specified name, or if the file already exists it modifies the timestamp.  rm : removes a file.  f : force deletion  r : recursive deletion  mv - moves a file, or renames a file  f : forces overwrite, if the destination file exists  cp - copies a file, leaving the original intact  f : forces overwrite, if the destination file exists  r : recursive copying of directories

E-Bridge Technology

Commands     

cat : shows the contents of a file, all at once more : shows the contents of a file, screen by screen less : also shows the contents of a file, screen by screen head : used to show so many lines form the top of a file tail : used to show so many lines form the bottom of a file

E-Bridge Technology

Piping and Redirection  The pipe (|) creates a channel from one command to another. Think of the pipe as a way of connecting the output from one command to the input of another command.  The pipe can be used to link commands together to perform more complex tasks that would otherwise take multiple steps (and possibly writing information to disk).  Examples:  Count the number of users logged onto the current system. The who command will give us line by line output of all the current users. We could then use the wc -l to count the number of lines... who | wc –l

E-Bridge Technology

Redirection Operating system gets information from or sends information to:  Standard input  Standard output  Standard error  Example: ls > file.txt

E-Bridge Technology

Redirection and Pipes  File redirection and pipes < # redirect input from specified source  > # redirect output to specified source  >> # redirect output and append to specified source  | # pipe the output from one command to the input to the next  Examples  grep word < /usr/dict/words  ls > listing  ls >> listing  ls -l| wc -l 

Command Line Arguments  The C language provides a method to pass parameters to the main() function. This is typically accomplished by specifying arguments on the operating system command line (console). The prototype for main() looks like: int main(int argc, char *argv[]) { … }

E-Bridge Technology

Command Line Arguments There are two parameters passed to main():  The first parameter is the number of items on the command line (int argc).

 The second parameter passed to main() is an array of pointers to the character strings containing each argument (char *argv[]).

E-Bridge Technology

Command Line Arguments Notes :  The main() routine can check argc to see how many arguments the user specified.  The minimum count for argc is 1: the command line just contained the name of the invoked program with no arguments.  The program can find out its own name as it was invoked: it is stored in the argv[0] string! Some operating systems don't provide this feature, however.  The arguments from the command line are not automatically converted: the characters are just copied into the argv strings.

E-Bridge Technology

Command Line Arguments #include<stdio.h> int main(int argc,char *argv[]) { int i; for(i=1;i<argc;i++) { Printf(“%s”,argv[i]); }

}  The string functions atoi(), atol(), atof(), etc., will also work. E-Bridge Technology

Shell Script  A shell usually interprets a single line of input, but we can also create a file containing a number of lines of commands to be interpreted  This file is a program known as a shell script  The program can also contain control structures (if-then, loops)  Shell scripts allow a sequence of commands to be executed automatically (e.g. installation of a program - see /user_client/sybase/install)

E-Bridge Technology

Shell Script

#!/bin/bash

Operators    

 

-lt -gt -le -ge -eq -ne

 if [ 2 –le 3 ] ; then ;echo "cool!" ; fi

if #!/bin/bash T1="foo" T2="bar" if [ "$T1" == "$T2" ] ; then echo “expression evaluated as true else” echo “expression evaluated as false” fi

E-Bridge Technology

for #!/bin/bash for (( i=0; i<10; ++i )) ; do echo item: $i done

E-Bridge Technology

while COUNTER=0 while (( COUNTER < 10 )) ; do echo The counter is $COUNTER (( COUNTER = COUNTER+1 )) done

E-Bridge Technology

Until #!/bin/bash COUNTER=20 until [ $COUNTER -lt 10 ]

do echo COUNTER $COUNTER let COUNTER-=1

done

E-Bridge Technology

Special Variables  $# the number of arguments  $* all arguments  $@ all arguments (quoted individually)  $? return value of last command executed  $$ process id of shell  $HOME, $IFS, $PATH, $PS1, $PS2

E-Bridge Technology

Generic Architecture of Embedded Linux

E-Bridge Technology

Monolithic and Micro Kernels

E-Bridge Technology

Monolithic Kernel  Kernel has simple design. Monolithic kernel is a single large processes running entirely in a single address space. It is a single static binary file.  All kernel services exist and execute in kernel address space. The kernel can invoke functions directly.  The examples of monolithic kernel based OSs are Linux, Unix.

E-Bridge Technology

Micro Kernel  In Microkernels, the kernel is broken down into separate processes, known as servers. Some of the servers run in kernel space and some run in user-space.  All servers are kept separate and run in different address spaces.The communication in microkernels is done via message passing.  The servers communicate through IPC (Interprocess Communication). Servers invoke "services" from each other by sending messages. The separation has advantage that if one server fails other server can still work efficiently.  The example of microkernel based OS are Mac OS X and Windows NT. E-Bridge Technology

Boot Loaders  Primary function: load Linux kernel into memory  Other functions:  Passing information to kernel during startup  Booting another OS: known as dual booting  Two most common boot loaders:  GRand Unified Boot loader (GRUB)  Linux Loader (LILO)

E-Bridge Technology

GRUB Loader  More common boot loader for modern Linux.  Stage1: first major part of GRUB  Typically resides on MBR  Points to Stage1.5  Stage1.5: loads file system support and Stage2 Resides in /boot/grub  Stage2: performs boot loader functions  Displays graphical boot loader screen  Resides in /boot/grub

E-Bridge Technology

LILO Loader Stands for Linux Loader Traditional Linux boot loader No longer supported by Fedora

Typically located on MBR. Lilo boot: prompt appears following BIOS POST Allows choice of OS to load at startup To configure, edit /etc/lilo.conf file

E-Bridge Technology

Root File System Structure(RootFS)

E-Bridge Technology

1. /bin  Contains several useful commands that are of use to both the system administrator as well as nonprivileged users.  Usually contains the shells like bash, csh, etc.... and commonly used commands like cp, mv, rm, cat, ls.  Also contains programs which boot scripts may depend on There are no (real) subdirectories in /bin.

E-Bridge Technology

2. /boot  Contains everything required for the boot process except for configuration files not needed at boot time and the map installer .  Stores data that is used before the kernel begins executing user-mode programs.  May include the system kernel (under symbolically linked).

E-Bridge Technology

3. /dev    

Usually is the location of device files. A device and a file both can be read from and written to. So config a device is same with edit a file. EX: sending data to /dev/ttyS0 that means you are sending data to a communication device, such as a modem.

 Block devices are devices that store or hold data  Character devices can be thought of as devices that transmit or transfer data.

E-Bridge Technology

4. /etc and 5. /home /etc:  Contains all system related configuration files .  Local file used to control the operation of a program  Those files must be static and cannot be an executable binary /home:  The user home directories  Accessible only to its owner and the system administrator  Contains the user’s personal configuration files  Quite large to be used as User’s Documents Space E-Bridge Technology

6. /lib and 7. /mnt /lib:  Contains kernel modules and those shared library images (the C programming code library) needed to boot the system and run the commands in the root file system, i.e. by binaries in /bin and /sbin.  Windows equivalent to a shared library would be a DLL (dynamically linked library) file. /mnt:  This is a generic mount point under mounted (mount is to make a filesystem available to the system) the filesystems or devices.  When a filesystem no longer needs to be mounted, it can be unmounted with umount E-Bridge Technology

8. /proc and 9. /sbin /proc:  Virtual file system, runtime system information (e.g. system memory, devices mounted, hardware configuration, etc).  The most of them have a file size of 0.  To view, use “cat”. Use “vi” to edit. /sbin:  Like /bin bet less important.  /sbin should contain only binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin. E-Bridge Technology

10. /root and 11. /var /root:  The home directory of the System Administrator, 'root'  Why not in '/home'? Because '/home' is often located on a different partition or even on another system and would thus be inaccessible to 'root' when - for some reason - only '/‘ is mounted. /var:  Contains variable data, files and directories the system must be able to write to during operation, like system logging files, mail and printer spool directories, and transient and temporary files. E-Bridge Technology

12. /usr  The largest share of data on a system.  The most important directories in the system as it contains all the user binaries, their documentation, libraries, header files, etc....  And its supporting libraries, and User programs like telnet, ftp, etc.... as well, can be found here.

E-Bridge Technology

Process 

The process is the OS’s abstraction for execution  the unit of execution  

 





a unit of scheduling the dynamic execution context  A program is static item – one or more files.

Process is often called a job, task, or sequential process. A process requires resources, which are managed by the operating system. The OS interleaves the execution of several processes to maximize processor utilization . OS supports Inter Process Communication (IPC) . E-Bridge Technology

Process State Diagram

E-Bridge Technology

Process Context  A process consists of :  an address space – usually protected and virtual – mapped into memory  the code for the running program  the data for the running program  an execution stack and stack pointer (SP)  the program counter (PC)  a set of processor registers – general purpose and status  a set of system resources

E-Bridge Technology

Process Address Space 0xFFFFFFFF Stack (dynamically allocated) SP

Virtual address space

Heap (dynamically allocated) static data

0x00000000

Code (text)

E-Bridge Technology

PC

Process in OS Representation  To users (and other processes) a process is identified by its Process ID (PID)  In the OS, processes are represented by entries in a Process Table (PT)  PID “points to” a PT entry  PT entry = Process Control Block (PCB)  PCB is a large data structure that contains or points to all info about the process.

E-Bridge Technology

Process Contains  Typical PCB contains:  execution state  PC, SP & processor registers – stored when process is made inactive  memory management info  Privileges and owner info  scheduling priority  resource info

E-Bridge Technology

Process Control Block (PCB) Pointer

Process state

Process number(PID) Program counter

Registers Memory limits List of open files

... E-Bridge Technology

Process Manipulation 







Basic process manipulation: creation, program loading, exiting:  fork(), exec(), wait(), exit() Process signaling:  kill() Process control:  ptrace(), nice(), sleep() An example application: The UNIX shell

E-Bridge Technology

Example: UNIX’s fork()  Creates a child process such that it inherits:  identical copies of all parent’s variables & memory  identical copies of all parent’s CPU registers  Both parent and child execute at the same point after fork() returns:  for the child, fork() returns 0  for the parent, fork() returns the process identifier of the child  Simple implementation of fork():  allocate memory for the child process  copy parent’s memory and CPU registers to child’s  expensive! E-Bridge Technology

fork: In the parent process: main() … int pid = fork(); // create a child if(pid == 0) { // child continues here … } else { // parent continues here … }

E-Bridge Technology

Exec: In the parent process: main() … int pid = fork(); // create a child if(pid == 0) { // child continues here exec(“program”, argc, argv0, argv1, …); } else { // parent continues here … }

E-Bridge Technology

Properties fork and exec  In 99% of the time, we call exec() after calling fork()  the memory copying during fork() operation is useless  the child process will likely close the open files & connections  overhead is therefore high  might as well combine them in one call (OS/2)  vfork()  a system call that creates a process “without” creating an identical memory image  sometimes called lightweight fork()  child process is understood to call exec() almost immediately E-Bridge Technology

wait system call  A child program returns a value to the parent, so the parent must arrange to receive that value  The wait() system call serves this purpose  it puts the parent to sleep waiting for a child’s result  when a child calls exit(), the OS unblocks the parent and returns the value passed by exit() as a result of the wait call (along with the pid of the child)  if there are no children alive, wait() returns immediately  also, if there are zombies waiting for their parents, wait() returns one of the values immediately (and deallocates the zombie). E-Bridge Technology

Threads  Threads is a light weight process.  threads in the same process share:  * Process instructions  * Most data  * open files (descriptors)  * signals and signal handlers  * current working directory  * User and group id

E-Bridge Technology

Thread Introduction         

Each thread has a unique: Thread ID set of registers stack pointer stack for local variables return addresses signal mask priority Return value: errno

E-Bridge Technology

Single and Multi threads

E-Bridge Technology

Thread and Process  How threads and processes are similar:  Each has its own logical control flow  Each can run concurrently  Each is context switched  How threads and processes are different:  Threads share code and data, processes (typically) do not  Threads are somewhat less expensive than processes Process control (creating and reaping) is twice as expensive as thread control Linux/Pentium III numbers: ~20K cycles to create and reap a process ~10K cycles to create and reap a thread E-Bridge Technology

Thread create and join • • • •

/* * hello.c - Pthreads "hello, world" program */ #include "csapp.h"

•

void *howdy(void *vargp);

• •

int main() { pthread_t tid;

h

• • • •

Pthread_create(&tid, NULL, howdy, NULL); Pthread_join(tid, NULL); exit(0); }

• • • • •

/* thread routine */ void *howdy(void *vargp) { printf("Hello, world!\n"); return NULL; } E-Bridge Technology

IPC (InterProcess Communication) IPC methods:        

Signals Mutex (MUTual EXclusion) Semaphores Shared memory Memory mapped files Pipes & named pipes Sockets Message queues

E-Bridge Technology

1. Signals    

Software interrupts. Asynchronous. Can be recognized or ignored. Examples: 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12)SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 17)SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP

 Also see man 7 signal for a lengthier description.

E-Bridge Technology

2. Mutex (Mutual Exclusion)  Used to control access to shared memory and other resources, in general.  Method to exclude other processes from using a shared variable until our process is finished with it.  Simpler than semaphore.  Two states: locked or unlocked.  Functions:  Declare mutex variable  Initialize mutex variable (just once)  Lock -> C.S. -> unlock

E-Bridge Technology

3. Semaphore 

Two basic operations:  Up – increment the value of the semaphore  Down – decrement the value of the semaphore  Binary semaphores = mutex  Create semaphore and initialize it to 1  1 = unlocked  0 = locked  Then to use this as a mutex:  Down

c.s.  up E-Bridge Technology

4. Shared Memory  Shared memory permits processes to communicate by simply reading and writing to a specified memory location.  Shared memory is the fastest form of inter process communication because,  All processes share the same piece of memory. Access to this shared memory is as fast as accessing a process’s non shared memory,  And it does not require a system call or entry to the kernel.  It also avoids copying data unnecessarily.

E-Bridge Technology

5. Mapped Memory  Mapped memory is similar to shared memory, except that it is associated with a file in the filesystem.  Mapped memory forms an association between a file and a process’s memory.  Linux splits the file into page-sized chunks and then copies them into virtual memory pages so that they can be made available in a process’s address space.Thus,  The process can read the file’s contents with ordinary memory access.  It can also modify the file’s contents by writing to memory. This permits fast access to files.

E-Bridge Technology

6. Pipe  A pipe is a communication device that permits unidirectional communication.  Data written to the “write end” of the pipe is read back from the “read end.” Pipes are serial devices; the data is always read from the pipe in the same order it was written.  Typically, a pipe is used to communicate between two threads in a single process or between parent and child processes.  In a shell, the symbol | creates a pipe. For example, this shell command causes the shell to produce two child processes, one for ls and one for less:  % ls | less

E-Bridge Technology

7. Sockets

E-Bridge Technology

Sockets  A socket is a bidirectional communication device that can be used to communicate with another process on the same machine or with a process running on other machines.  Internet programs:  such as Telnet, rlogin, FTP, talk, and the World Wide Web use sockets.  For Example:  You can obtain the WWW page from a Web server using the Telnet program because they both use sockets for network communications.4

E-Bridge Technology

8. Message Queue  It is like a pipe and is used to transfer messages between processes in Unix system.  A message queue is a queue onto which messages can be placed. A message is composed of message type and message data.

 The maximum size of a message in a queue is limited by the operating system and for fedora it is typically 65535 bytes.

E-Bridge Technology

FILE System  A named collection of related information recorded on secondary storage (e.g., disks)  File attributes  Name – only information kept in human-readable form  Identifier – unique tag (number) identifies file within file system  Type – needed for systems that support different types  Location – pointer to file location on device  Size – current file size  Protection – controls who can do reading, writing, executing  Time, date, and user identification – data for protection, security, and usage monitoring E-Bridge Technology

File Control Block  File operations  Create, Open, Read, Write, Seek, Delete, …

E-Bridge Technology

Thank you

E-Bridge Technology