1What is the primary advantage of parallel computing over serial computing?
Faster execution for certain tasks
2What is scalability in parallel computing?
It is a measure of how the algorithm reacts as the size of the problem grows with relation to the number of processors.
3What is one of the biggest performance bottlenecks in a distributed memory system?
Communication
4How does pipelining speed up processing?
By parallelizing the steps in executing a program so that different steps can be executed for different instructions simultaneously.
5What are the two main types of parallelism?
Task parallelism and data parallelism
6What does load balancing in parallel computing refer to?
Sharing the work evenly among the cores so that no single core is heavily loaded.
7What are the four steps of Foster’s Methodology?
Partitioning, Communication, Aggregation, Mapping
8What is a race condition?
A situation where multiple cores access shared resources in an unpredictable manner leading to incorrect results.
9What does MPI stand for?
Message-Passing Interface
10What is the purpose of MPI_Init?
It tells MPI to do all the necessary setup before running a parallel program.
11What function in MPI is used to synchronize all processes?
MPI_Barrier
12What is the purpose of MPI_Sendrecv?
It allows sending and receiving data simultaneously between processes.
13What is the main concept of Amdahl’s Law?
The speedup of a program is limited by the fraction of the program that cannot be parallelized.
14What is instruction-level parallelism (ILP)?
A technique that allows a single processor to execute multiple instructions simultaneously.
15What are the types of parallel hardware according to Flynn’s Taxonomy?
SISD, SIMD, MISD, MIMD
16What is the von Neumann bottleneck?
The separation of CPU and memory, which limits the speed of data transfer.
17What are the two types of locality in caching?
Spatial locality and temporal locality
18What is strong scalability?
When efficiency remains constant while increasing the number of processors without increasing problem size.
19What is weak scalability?
When efficiency remains constant by increasing both problem size and number of processors at the same rate.
20What is the equation for speedup in parallel computing?
Speedup (S) = Serial Time (Tserial) / Parallel Time (Tparallel)
21What does MPI_Gather do?
Collects data from all processes and combines it on a single process.
22What does MPI_Scatter do?
Distributes data from one process to all other processes.
23What is the purpose of mutual exclusion (mutex) in parallel computing?
To prevent multiple threads from accessing shared resources simultaneously and causing race conditions.
24What does Tparallel = Tserial / p + Toverhead represent?
The equation for parallel execution time, including overhead.
25What is the main difference between shared memory and distributed memory systems?
Shared memory systems use a single address space accessible by all processors, while distributed memory systems use separate address spaces for each processor.
26What does MPI_Bcast do?
Broadcasts data from one process to all other processes in the communicator.
27What is the purpose of a cache in a processor?
To store frequently accessed data closer to the CPU to improve performance.
28What is the principle of temporal locality?
If a memory location is accessed once, it is likely to be accessed again soon.
29What is the principle of spatial locality?
If a memory location is accessed, nearby locations are likely to be accessed soon.