Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

Why OpenMP is the way to go for parallel programming

I guess it’s time once and for all to explain the reasons why I think OpenMP is the future of parallel programming, at least in the short term. I will start with the short version:

  • relatively high level of abstraction
  • performance
  • maturity


Are you still with me? OK, then let’s hear the long version – actually you will hear the really long version now ;-). When I started my PhD., my supervisor and me relatively quickly agreed that we wanted to do something to make parallel programming easier. Therefore I started looking around and tried to figure out, what the present state of the art regarding parallel programming looked like (you can still see the project page for that subproject here, some more results will be published shortly). I found that the dominant languages for parallel programming were still C/C++ and Fortran. I tried several others (e.g. Java, Haskell, Ocaml, Python), but each had some shortcoming making it inferior for parallel programming. I did not want to invent a new language from scratch, simply because I did not have the experience required to do a good job with it and also because I do not like the idea of investing four years of my life into building something that no one except me will ever use – which might sound overly pessimistic but judging from all the abandoned webpages I encountered during my search this seems to be what happens to the majority of newly invented languages.

Since I also do not speak Fortran too well and do not like it too much either, I had to stick with C or C++ and all the existing parallel programming systems for it (which eliminated High Performance Fortran from the equation). At that point, OpenMP was the logical choice, because it already was designed for ease of use. Other advantages are the reasons stated at the beginning of this article. To come back to the points made there, first of all it has a higher level of abstraction than most of the other parallel programming systems I know. Which is great. Consider the following program snippet as an example:

const int N=120000;
int arr[N];

#pragma omp parallel for
for (int i=0; i

  • you do not see how each and every thread is created and initialized
  • you also do not see a function declaration for the code each thread executes
  • you also do not see exactly how the array is divided between the threads
  • In most other parallel programming systems, you will have to write code for all this and a lot more, as soon as it gets more complicated. Or maybe you have to write code for explicitly sending messages to different processes if you are using a message passing system such as MPI or PVM. Not with OpenMP, and that is the very first reason I like it – it makes my life as a programmer easier, because I have to do less. Higher level of abstraction, as I said.

    The other two reasons I mentioned earlier are quickly explained: the performance of OpenMP is on par with the performance of other, lower-level parallel programming systems. Since performance is an important factor (after all, if I did not care about performance, I would not parallelize my application at all, simple as that), this point is of critical importance.

    The last advantage OpenMP has over a lot of parallel programming systems is its maturity. Version 2.5 of the specification is out presently, and work on 3.0 is progressing. Since OpenMP has been around since 1997, the compilers are relatively advanced by now (at least for C and Fortran, there are still some problems with C++, which I might describe in more detail in a future article). There is also more than one compiler available (e.g. the Intel, Sun and Portland Group Compilers, Visual Studio recently added support and GCC will have support with the upcoming version 4.2, to name just a few), which is way more than I can say for most research parallel programming systems.

    Because of these three reasons, I chose OpenMP as the basis of my research and have been active since this time to find problems with it, come up with solutions for these problems and generally try to improve the system. You can see part of my work at this project description page, but since I have joined the very kind folks at the OpenMP language committee recently, I have done some work there as well, which is not publically visible.

    I guess now you know, why in my humble opinion OpenMP is the way to go for parallel programming at this point in time, what I am presently doing for my PhD. and why this blog will be heavily biased towards OpenMP.

    If you still want to read it, please consider subscribing to it using either the RSS-feeds or the newly added email subscription option, both conveniently located right at the top of the sidebar. The email subscription service is powered by Feedburner and their privacy policy assures that no email address will ever be sold or misused by them and I have no reason not to believe them in this point – and of course I will not do so either.

    8 Responses to Why OpenMP is the way to go for parallel programming


    Comments

    1. Comment by Cobo | 2006/08/14 at 12:45:12

      Great article. Now I guess why you’re so interested in OpenMP instead of PThreads or any other systems. I have very little background with PThreads, but if OpenMP gives that level of abstraction and is close to the efficiency PThreads can give… I should try it out.
      Could you recommend any good book/documentation related to OpenMP (theory and programming, better with C)?
      Also I’m having problems to find if there’s any architecture OpenMP doesn’t support (x86, x86-64, PowerPC, Alpha…). Anything useful?

      Willing to read more and more of this stuff.

      Thanks!

    2. Comment by Michael Suess | 2006/08/14 at 13:37:56

      Glad you liked the article.

      For resources on OpenMP, the best place to check would be the Compunity. Look under Resources for a list of compilers and tools. Under Training you will find a list of tutorials which are good starting points.

      There is a book on OpenMP called “Parallel Programming in OpenMP”, but I usually do not recommend it because it is outdated and OpenMP is not that difficult as to warrant a whole book on it (just my opinion). If you really like to have a book, I would recommend bying one on the more general subject of parallel programming, e.g. “Introduction to Parallel Computing” by Grama et. al. It has a section on OpenMP in it, as well as sections on Threads, MPI and many useful parallel algorithms. I bet there are good books on parallel programming in Spanish as well, if you prefer that and I bet some of them have sections on OpenMP in them as well.

      OpenMP-support has not much todo with the architecture, but rather with the compiler and its runtime library. If the compiler supports OpenMP on a specific architecture, it will map the parallelism described by the OpenMP-directives to the native thread library provided by the operating system (e.g. POSIX threads or win32 threads). So if you have a compiler for a specific architecture that supports OpenMP, you are set to compile and run OpenMP programs there. For a list of compilers supporting OpenMP see the Compunity page of compilers.

      As soon as GCC 4.2 with OpenMP-support is released, I do not know of any major architecture that OpenMP does not support (since GCC basically runs everywhere 🙂 ).

      Hope that helps!

    3. Comment by Cobo | 2006/08/15 at 07:45:35

      Thanks for all the doc and links.

      I have already been looking around Compunity, and it looks quite good.
      As I was reading your comment another programmer was recommending me the same book on parallel computing, so it seems I’ll have to read it :).

      I’ll start any time soon with OpenMP, maybe October after my September exams, and a little rest…

      Thanksa lot for all the interest and help!

    4. Comment by Cem DEMiRKIR | 2008/09/08 at 20:17:00

      Hi,
      I’m new in this multithreaded programming world. I guess OpenMP is one of best API for multithreaded programming to learn and save your time for learn it… In my opinion the most attractive property of OpenMP it is simply compiler switches which can enable the switching between sequential and parallel runs and it is easy to learn and most of the compiler supports it. But I’d really like to learn How I can debug an OpenMP build code. I’m generally in Windows world and prefer to write code in Visual Studio. But I don’t know How to debug an OpenMP compiled code using the nice thread showing properties of 2008. Is there any tutorial or a documentation about this subject ?

      Regards
      Cem

    5. Comment by Andrey | 2008/09/14 at 18:00:51

      32 OpenMP traps for C++ developers :mrgreen:
      http://www.codeproject.com/KB/cpp/32_OpenMP_traps.aspx

    6. Comment by jkriegshauser | 2008/10/01 at 20:03:18

      Very interesting.

      My personal experience with OpenMP was that it performs well if you have very CPU-bound repetitive tasks. If you read my latest blog post I talk about how the first attempts at using [something similar to] OpenMP in EverQuest II didn’t net us any significant gains on the largest repetitive tasks we have: finding degenerate triangles, animation vertex transformations, particle system init, etc. I imagine this is true in larger projects that are doing a lot more than calculations.

      I’m rather interested in Intel’s Threading Building Blocks over OpenMP for a few reasons:
      – No specific compiler support/integration required
      – Greater flexibility of task concepts
      – Supports simple functionality of OpenMP and more
      – Cross-platform
      – Now Open-Source

      Applying some of the principals from the TBB to the EverQuest II codebase did get us a framerate improvement of 10-15% on dual-core hardware.

    7. Comment by Cath | 2009/01/30 at 14:51:29

      “yes, I know, this code is a toy-example and is probably slower on most platforms than the sequential version,”

      You know this is a bit of a contra to what OpenMP books say – that it is best at “simple” loops. So how come this piece is slower (well with gcc it is really and i don’t know what loop it takes to b faster, for particle loops seems to b also clumsy). Any info from ur experience abt particle loops (calculating forces between particles) usin’ gcc 4.2?

    8. Comment by saru | 2011/01/15 at 10:25:51

      its very nice, but i came across with another similar blog

      http://openmp.blogspot.com/2011/01/home.html

      so i just thought of sharing it here,.. it was very helpful for me,..


    Comments are closed