Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

Why I love parallel programming

To explain a little more about the intentions of this blog, I will try to make it clear in this article, why parallel programming is a great thing to do and why I love doing it.

When I first started with parallel programming while at the university, I mainly did it because it was a challenge. Wrapping your mind around the flow of control is not always easy in a sequential program, and it is an order of magnitude harder in a parallel one, since there are multiple threads of control at the same time. But since it is hard, it is also an area with a great feeling of accomplishment, when you get that program of yours running in parallel! Of course, you can also look at it from the other way around: once you master parallel programming, sequential programming feels almost like a vacation (OK, I am exaggerating a tiny bit).

A second reason why parallel programming is fun is, that you have a really good measure for success: the speedup. You always know, how good your parallelization strategy is, just by measuring the sequential and parallel execution times, dividing them and comparing that with the number of processors you used. You always know, when you did a good job and when you did not (unfortunately, it is not quite as easy as it sounds here, since some programs are easy to parallelize, while with others you will rather win the lottery than see any speedups at all).

The third reason is relatively new. When I started with parallel programming, it was a discipline mostly practiced in high performance computing. A lot of the parallel codes then were numerical calculations, done by natural scientists. To be honest, I do not like numerics very much. I will even go as far as to say, I hated the class at the university. But the times are changing. With the arrival of multi-core CPUs in commodity PCs, the whole computing world is going parallel. It starts with computer games (as always) today, but I am sure all that parallel computing power of todays and tomorrows dual- and quad-core CPUs is not going to be sitting there forever running anti-virus software or the like. Herb Sutter has a great article on the subject titled The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software, which shows better than I could ever do it here that we are in the middle of a revolution right now! It is a parallel revolution, and this time it is for real. Programming parallel applications allows me to be right in the middle of that revolution, which is an exciting place to be!

The last reason I want to mention why I love parallel programming is quite personal: I am an optimizer. I hate inefficiencies, I hate to waste time or resources and parallel programming is a very nice way to express my quest for efficency. Making programs run faster is a deeply satisfying thing to do for a person with that mindset, so if you happen to feel the same, you might want to consider this career path (or maybe you already have and I am preaching to the choir – would not be the first time this happened to me).

To sum up the points I made in this article: I love parallel programming because it is hard, rewarding and makes sequential programming easier. I love the way I can measure my success using a simple number (speedup). There is a revolution happening right now, and by programming in parallel I can feel like living on the edge of technology – which is a fun thing to do. And last but not least, I love parallel programming because it supports my quest for efficiency. Pick any one you like, four reasons is more than enough for me!

Of course, not everything is as enjoyable as I made it sound like in this article – and therefore someday I will also be writing about the things I (sometimes) hate about parallel programming.

One Response to Why I love parallel programming


  1. Comment by Refik | 2007/05/17 at 02:50:10

    I completely agree with you Michael on parallel programming! I am not sure if we can put it in the same bucket but I am used to developing various electronic applications on FPGA devices which process everything on the parallel basis, sometimes it is really pain in the a** to synchronize everything however it is one of the rare good things when you work with DSP and need to process data fast, normal sequential processors just won’t work out for you.

    Btw. I enjoy reading your articles which are even understandable for my undergraduate level. Keep up pal!

    Refik 🙂

Comments are closed