For the past four decades explosive gains in computing power have contributed to unprecedented progress in innovation, productivity and human welfare. But that progress is now threatened by the unthinkable: an end to the gains in computing power.
We have reached the limit of what is possible with one or more traditional, serial central processing units, or CPUs. It is past time for the computing industry--and everyone who relies on it for continued improvements in productivity, economic growth and social progress--to take the leap into parallel processing.
Reading this essay is a serial process--you read one word after another. But counting the number of words, for example, is a problem best solved using parallelism. Give each paragraph to a different person, and the work gets done far more quickly. So it is with computing--an industry that grew up with serial processing--and which now faces a serious choice between innovation and stagnation.
The backdrop to this issue is a paper written by Gordon Moore, the co-founder of
This bold prediction became known as Moore's Law. And it held true through the 1980s and '90s--fueling productivity growth throughout the economy, transforming manufacturing, services, and media industries, and enabling entirely new businesses such as e-commerce, social networking and mobile devices.
Moore's paper also contained another prediction that has received far less attention over the years. He projected that the amount of energy consumed by each unit of computing would decrease as the number of transistors increased. This enabled computing performance to scale up while the electrical power consumed remained constant. This power scaling, in addition to transistor scaling, is needed to scale CPU performance.
But in a development that's been largely overlooked, this power scaling has ended. And as a result, the CPU scaling predicted by Moore's Law is now dead. CPU performance no longer doubles every 18 months. And that poses a grave threat to the many industries that rely on the historic growth in computing performance.
Consider just a few examples, with significant social consequences.
Public agencies need more computing capacity to forecast dangerous weather events and analyze long-term climate change. Energy firms need to assess massive quantities of seismic and geological data to find new ways to safely extract oil and gas from existing reserves. Pharmaceutical researchers need increased computing power to design drug molecules that bind to specific cell receptors. Clinical oncologists need better and faster medical imaging to diagnose cancers and determine treatments. Cardiac surgeons want to assess damaged tissues visually in real time to ensure their procedures will be effective.
But these needs will not be met unless there is a fundamental change in our approach to computing.
The good news is that there is a way out of this crisis. Parallel computing can resurrect Moore's Law and provide a platform for future economic growth and commercial innovation. The challenge is for the computing industry to drop practices that have been in use for decades and adapt to this new platform.
Going forward, the critical need is to build energy-efficient parallel computers, sometimes called throughput computers, in which many processing cores, each optimized for efficiency, not serial speed, work together on the solution of a problem. A fundamental advantage of parallel computers is that they efficiently turn more transistors into more performance. Doubling the number of processors causes many programs to go twice as fast. In contrast, doubling the number of transistors in a serial CPU results in a very modest increase in performance--at a tremendous expense in energy.
More importantly, parallel computers, such as graphics processing units, or GPUs, enable continued scaling of computing performance in today's energy-constrained environment. Every three years we can increase the number of transistors (and cores) by a factor of four. By running each core slightly slower, and hence more efficiently, we can more than triple performance at the same total power. This approach returns us to near historical scaling of computing performance.
To continue scaling computer performance, it is essential that we build parallel machines using cores optimized for energy efficiency, not serial performance. Building a parallel computer by connecting two to 12 conventional CPUs optimized for serial performance, an approach often called multi-core, will not work. This approach is analogous to trying to build an airplane by putting wings on a train. Conventional serial CPUs are simply too heavy (consume too much energy per instruction) to fly on parallel programs and to continue historic scaling of performance.
The path toward parallel computing will not be easy. After 40 years of serial programming, there is enormous resistance to change, since it requires a break with longstanding practices. Converting the enormous volume of existing serial programs to run in parallel is a formidable task, and one that is made even more difficult by the scarcity of programmers trained in parallel programming.
Parallel computing, however, is the only way to maintain the growth in computing performance that has transformed industries, economies, and human welfare throughout the world. The computing industry must seize this opportunity and avoid stagnation, by focusing software development and training on throughput computers - not on multi-core CPUs.
Let's enable the future of computing to fly--not rumble along on trains with wings.
Bill Dally is the chief scientist and senior vice president of research at NVIDIA and the Willard R. and Inez Kerr Bell Professor of Engineering at Stanford University.
Read more Forbes opinions here.