Amdahl’s law

Amdahl's Law: Understanding the Limits of Parallel Processing

In the world of electrical engineering, achieving faster processing speeds is a constant pursuit. Multiprocessor systems, with their ability to split tasks across multiple cores, seem like the perfect solution. However, a fundamental principle known as Amdahl's Law highlights the inherent limitations of parallel processing.

Amdahl's Law, formulated by Gene Amdahl in 1967, states that the speedup factor of a multiprocessor system is given by:

\(S(n) = {n \over 1 + (n - 1)f}\)

where:

• n is the number of processors
• f is the fraction of the computation that must be performed sequentially (by one processor alone)

The remaining portion of the computation, \((1-f)\), is assumed to be perfectly parallelizable, meaning it can be divided into n equal parts, each executed simultaneously by a separate processor.

What does this mean?

Amdahl's Law tells us that even with an infinite number of processors, the speedup of a program is limited by the portion that cannot be parallelized. As the number of processors (n) approaches infinity \(n → ∞\) , the speedup factor tends towards \(1 \over f\),  highlighting the crucial role of the serial fraction.

For example:

Imagine a program where 20% of the code must be executed sequentially (f = 0.2). Even with an infinite number of processors, the maximum speedup attainable is \({1 \over 0.2} = 5.\) This means that the program can at most run 5 times faster than on a single processor, no matter how many more cores are added.

Implications of Amdahl's Law:

• Serial fraction is critical: A small percentage of sequential code can significantly limit the potential speedup.
• Focus on parallelization: Optimizing the code to reduce the serial fraction is crucial for achieving significant performance gains with parallel processing.
• Practical limits: Amdahl's Law highlights the practical limitations of parallel processing, reminding us that achieving unlimited speedup is impossible.

Beyond the limitations:

While Amdahl's Law outlines important limitations, it's not the end of the story. Modern techniques like vector processing, GPU computing, and specialized hardware can effectively tackle some of the bottlenecks associated with serial computations.

In conclusion:

Amdahl's Law is a fundamental principle in electrical engineering, providing a realistic view of the potential speedup achievable with parallel processing. By understanding the impact of the serial fraction, engineers can focus on optimizing code and designing systems that maximize the benefits of parallel processing. While achieving infinite speedup may not be possible, Amdahl's Law empowers us to make informed decisions and unlock the true potential of parallel computing.