Why I love parallel programming
To explain a little more about the intentions of this blog, I will try to make it clear in this article, why parallel programming is a great thing to do and why I love doing it.
When I first started with parallel programming while at the university, I mainly did it because it was a challenge. Wrapping your mind around the flow of control is not always easy in a sequential program, and it is an order of magnitude harder in a parallel one, since there are multiple threads of control at the same time. But since it is hard, it is also an area with a great feeling of accomplishment, when you get that program of yours running in parallel! Of course, you can also look at it from the other way around: once you master parallel programming, sequential programming feels almost like a vacation (OK, I am exaggerating a tiny bit).
A second reason why parallel programming is fun is, that you have a really good measure for success: the speedup. You always know, how good your parallelization strategy is, just by measuring the sequential and parallel execution times, dividing them and comparing that with the number of processors you used. You always know, when you did a good job and when you did not (unfortunately, it is not quite as easy as it sounds here, since some programs are easy to parallelize, while with others you will rather win the lottery than see any speedups at all).
The third reason is relatively new. When I started with parallel programming, it was a discipline mostly practiced in high performance computing. A lot of the parallel codes then were numerical calculations, done by natural scientists. To be honest, I do not like numerics very much. I will even go as far as to say, I hated the class at the university. But the times are changing. With the arrival of multi-core CPUs in commodity PCs, the whole computing world is going parallel. It starts with computer games (as always) today, but I am sure all that parallel computing power of todays and tomorrows dual- and quad-core CPUs is not going to be sitting there forever running anti-virus software or the like. Herb Sutter has a great article on the subject titled The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software, which shows better than I could ever do it here that we are in the middle of a revolution right now! It is a parallel revolution, and this time it is for real. Programming parallel applications allows me to be right in the middle of that revolution, which is an exciting place to be!
The last reason I want to mention why I love parallel programming is quite personal: I am an optimizer. I hate inefficiencies, I hate to waste time or resources and parallel programming is a very nice way to express my quest for efficency. Making programs run faster is a deeply satisfying thing to do for a person with that mindset, so if you happen to feel the same, you might want to consider this career path (or maybe you already have and I am preaching to the choir – would not be the first time this happened to me).
To sum up the points I made in this article: I love parallel programming because it is hard, rewarding and makes sequential programming easier. I love the way I can measure my success using a simple number (speedup). There is a revolution happening right now, and by programming in parallel I can feel like living on the edge of technology – which is a fun thing to do. And last but not least, I love parallel programming because it supports my quest for efficiency. Pick any one you like, four reasons is more than enough for me!
Of course, not everything is as enjoyable as I made it sound like in this article – and therefore someday I will also be writing about the things I (sometimes) hate about parallel programming.