Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

Expanding on my earlier article about Why OpenMP is the way to go for parallel programming, I would like to point out a couple more strengths of OpenMP. And the best thing about this is: I do not have to do it myself this time, because OpenMP evangelist Ruud van der Pas has already done the work in his nice article on Using OpenMP to parallelize for CMT. Therefore I will just tease you with the main advantages he has come up with (shamelessly copied from his article): (more…)

I guess it’s time once and for all to explain the reasons why I think OpenMP is the future of parallel programming, at least in the short term. I will start with the short version:

  • relatively high level of abstraction
  • performance
  • maturity

(more…)

This article is about high-level-optimization, i.e. I will explain how I usually approach optimizing a program without going into the gory details. There are a million books and web pages about low-level-optimizations and the tricks involved out there, therefore I will not dive into that (could not possibly hope to cover this in a single blog-entry anyways).

Many of the points made here may sound like a matter of course, yet I have not obeyed them myself enough times in the past to know, that sometimes the temptation to just start changing stuff around in my programs is bigger than my rationality. All of the points presented here can be found in any reasonably good book about optimization, but I know enough programmers that have never read any of them (to be honest: I have not read too many of them either). A little repetition of some obvious points will therefore not hurt. (more…)

We have a dual-processor, dual-core Opteron system at work (that makes 4 cores in total 😉 ). I often use it for performance-measurements of my parallel codes. I expected performance variations in my benchmarks when only two threads are used. The explanation of course would be simple: the Opteron-system is actually a NUMA-system. Half of the memory belongs to processor one, the other half belongs to processor two. Both processors are connected through the Hypertransport-bus (running at 1000MHz if I am not mistaken). If processor one wants to access memory belonging to processor two, it can do so. It will take longer than usual though, because it cannot do so directly, but only via processor two. That gives us two different memory access times. It gets even more complicated with four processors (does not matter how many cores they have, as the two cores on the dual-core processors share a single memory controller). On many boards, they are connected like a ring. And that gives us three different memory access times, as a memory access may have to travel across zero, one or in the worst case two hypertransport-connections. OK, enough of the NUMA-basics already, lets return to my main thread. (more…)

To explain a little more about the intentions of this blog, I will try to make it clear in this article, why parallel programming is a great thing to do and why I love doing it.

When I first started with parallel programming while at the university, I mainly did it because it was a challenge. Wrapping your mind around the flow of control is not always easy in a sequential program, and it is an order of magnitude harder in a parallel one, since there are multiple threads of control at the same time. But since it is hard, it is also an area with a great feeling of accomplishment, when you get that program of yours running in parallel! Of course, you can also look at it from the other way around: once you master parallel programming, sequential programming feels almost like a vacation (OK, I am exaggerating a tiny bit). (more…)

Hello world,

my name is Michael Suess and this is my blog (you already guessed that, right?). Since this is my first post, I will introduce myself, what I am doing and why this may be interesting to you. And since I do not want this to be boring, I will use a Q&A-session to do so.

Who are you?
My name is Michael Suess, I am currently situated in Kassel, Germany, doing my PhD in a topic related to parallel programming. The topic of my thesis can be summarized shortly as: “exploring ways to make parallel programming easier”.

There must be more to you than just a location and a thesis, right?
Of course there is. I am married and have an nine month old son. I have hobbies (e.g. roleplaying, jogging, soccer, books), friends and even a life. But since this blog is about neither of these, I will stop telling you about them now.

(more…)