8.3
Theory of Parallel Computing
Traditionally, computer software has been written for the serial computation and time
sharing computation. Therefore to solve a problem, an algorithm is constructed which
produces a serial stream of instructions. These produced instructions are executed
sequentially one after the other on the CPU of the computer.
87
Parallel computing on the other hand uses multiple processing elements simultaneously on
a problem. The problem is broken into parts which are independent so that each
processing element can execute its part of the algorithm simultaneously with others. The
processing elements can be diverse and include resources such as a single computer with
multiple processors, a number of networked computers, specialized hardware or any
combination of any of the above. The software speedup was achieved by using a CPU with
higher clock rates, which significantly increased each passing year. However, when clock
speed reached 4GHz, the increase in power consumption and heat dissipation formed what
is known as the “Power Wall”
which effectively caused the CPU clock rate to level off [136].
Along with this, many applications today require more computing power than a traditional
sequential computer can offer. All these things made the vendors search for an alternative
to make increase of available cores with in a processor instead of increasing the clock
rates. The increase of cores on the processor made the CPU clock rates to remain same or
even reduced to economize the power usage. The old software design used for the
sequential process will not get increased directly with the increase in the processor cores.
To get the benefit form the current developed multi core processor, new software has to be
designed to take the advantages of the new architecture. This makes the use of all available
multi cores and performs process in parallel. Parallel computing is defi
ned as “a form of
computation in which many calculations are carried out simultaneously, operating on the
principle that large problems can often be divided into smaller ones, which are then solved
concurrently (in parallel)".
Many different hardware architectures exist today to perform a single task using multiple
processors. Some examples are: Grid computing
–
a combination of computer resources
from multiple administrative domains applied to a common task.
Massively Parallel
Processor
(MPP) systems
–
known as the supercomputer architecture Cluster server
system
–
a network of general-purpose computers.
Symmetric Multiprocessing
(SMP)
system
–
identical processors (in powers of 2) connected together to act as one unit. Multi-
core processor
–
a single chip with numerous computing cores [136]. Heterogeneous
computing systems also provide an opportunity to dramatically increase the performance
of parallel and HPC applications on clusters with CPU and GPU architecture [137]. This
concept can be achieved by combining the GPU and multicore CPUs.
|
