Amdahl's Law is a principle used in computer architecture and parallel computing to predict the theoretical maximum improvement in system performance when only part of the system is improved. Named after Gene Amdahl, a pioneer in computer architecture, this law highlights the limitations of parallel processing and helps in understanding the impact of enhancements on overall system performance.
Definition and Formula
Amdahl's Law states that the overall performance improvement of a system using parallel processing is limited by the portion of the system that must operate sequentially. The law is mathematically expressed by the formula:
$$ S_{\text{max}} $$ is the maximum possible improvement in execution time of the entire task.
$$ p $$ is the proportion of the execution time that the part of the task being improved occupies under non-parallel circumstances.
$$ s $$ is the speedup of the part of the task that benefits from improved performance (e.g., parallel processing).
Practical Implications
The key takeaway from Amdahl's Law is that improving a part of the system that constitutes a small fraction of the total execution time yields diminishing returns on overall system performance. This is because the sequential portion of the task, which cannot be parallelized or improved, becomes a bottleneck. As a result, even with significant improvements in parallelizable components, the overall speedup is capped by the sequential elements[1][4].
Example
Consider a task that takes 100 units of time, with 90 units of time being parallelizable. If the parallelizable portion is sped up by 10 times, according to Amdahl's Law, the maximum speedup of the entire system would be:
