1.parallel Processing

Parallel Processing

Parallel Processing • It is an efficient form of information processing which emphasizes the exploitation of concurrent events in the computing process. • Purpose of parallel processing – To speed up the computer processing capability – To increase its throughput

• Advantages over sequential processing – Cost effective – Improved performance ,i.e., faster execution time

Concurrency implies parallelism, simultaneity, and pipelining – Parallel events occur in multiple resources during the same time interval – Simultaneous events may occur at the same time instant – Pipelined events may occur in overlapped time spans

• Note that parallel processing differs from multitasking, in which a single CPU executes several programs at once. • Parallel processing is also called parallel computing.

4 levels of parallel processing • • • •

Job or Program level Task or procedure level Interinstruction level Intrainstruction level

Distributed Processing is a form of parallel processing in a special environment.

Parallelism in Uniprocessor System Basic Uniprocessor System • A typical Uniprocessor (super minicomputer) consists of 3 major components: – Main memory – CPU – I/O subsystem

• Uniprocessor : Mainframe IBM – Main memory: 4 LSUs, storage controller – Peripherals are connected to the I/O channels – Operation with the CPU is asynchronous

Parallel Processing Mechanisms • • • • • •

Multiplicity of functional units Parallelism and pipelining within the CPU Overlapped CPU and I/O operations Use of hierarchical memory system Balancing of subsystem bandwidth Multiprogramming and Time-sharing

1.Multiplicity of functional units • Use of multiple processing elements under one controller • Many of the ALU functions can be distributed to multiple specialized units • These multiple Functional Units are independent of each other

Example: • IBM 360/91 – 2 parallel execution units • Fixed point arithmetic • Floating point arithmetic(2 Functional units) – Floating point add-sub – Floating point multiply-div

2.Parallelism & pipelining within the CPU • Parallelism is provided by building parallel adders in almost all ALUs • Pipelining – Each task is divided into subtasks which can be executed in parallel

3.Overlapped CPU and I/O operations • I/O operations can be performed simultaneously with the CPU computations by using – separate I/O controllers – I/O channels – I/O processors

DMA-Cycle-stealing • Perform I/O transaction between the device and the main memory without the intervention of the CPU – I/O controller will generate a sequence of memory addresses – CPU is responsible for initiating the block transfer – CPU is notified, when the transfer is complete

Use of I/O Processor : achieves maximum concurrency • I/O subsys is facilitated by an IOP • CPU is free to proceed with its primary task • IOP consists of: – I/O processor – I/O channels

• I/O processor is attached directly to system bus • I/O processor is capable of executing I/O requests

4.Use of hierarchical memory system • Speed of CPU = 1000 times speed of Main memory

• hierarchical memory structure is used to close up the speed gap – Cache memory – Virtual memory – Parallel memories for array processors

5.Balancing of subsystem bandwidth • Balancing bandwidth between main memory and CPU • Balancing bandwidth between main memory and I/O

6.Multiprogramming and Time-sharing • Multiprogramming – Mix the execution of various types of programs (I/o bound, CPU bound) – The interleaving of CPU and I/O operations across several programs

• Time-sharing – Time-sharing OS is used to avoid high-priority programs occupying the CPU for long – Fixed or variable time-slices are used – Creates a concept of virtual processors

Parallel Computer Models • Pipeline Computers • Array Computers • Multiprocessor Computers

Performance of parallel computers • Parallel computers with n identical processors do not have speeds n times faster than a single processor computer, but less than n • Speeds range from log2n to n/ln n