Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

Top 5 Reasons why I Hate Parallel Programming (Sometimes)

AngryMy regular readers know that I am doing a lot of parallel programming for work. I am writing my PhD-thesis on it. I am maintaining this blog on the very topic and have even stated in the past that I love parallel programming and why. Yet, there are still a few days when I wish things would just go a little smother. And on even fewer of these days, I hate parallel programming. This post lists and explains the top five reasons why this is the case sometimes. It is also my submission for Darren Rowses (problogger) new group writing project.

Here they are:

  1. It’s the platform that matters, and parallel programming platforms are not on par with their sequential counterparts. This hits me every time I search for a good parallel profiler or debugger. Or when I am looking for a particular library only to realize that it is not thread-safe and may therefore not be usable for me. I feel the pain when I am looking at an IDE and it does not understand the parallel constructs I am working with every day. Thats when I know that parallel programming still has a long way to go before mainstream adaption.
  2. It’s way too easy to make mistakes (I have a Top 10 List pending for that topic alone), therefore I will not elaborate it here.
  3. The programming languages that many parallel programming systems are based on are either not powerful enough (C, C++, Fortran), or their performance is suboptimal (all the scripting languages – yes, you can do parallel programming in some of them – I have even tried it in the past). If you have programmed in a scripting language before, you will know what I mean (feature- and performance-wise). I don’t want to care about how many elements fit into my arrays, I have come to expect that the language takes care of that for me, just to name an example. I guess I am spoiled by programming for the web. *sigh*
  4. Parallel Programming is still mostly a manual process – one needs to think about data- and task-distribution without much help from any tools – which makes the process error-prone and one needs a lot of experience to do a good job at it. And even with a lot of experience, some of it just boils down to trial-and-error. And I thought this was supposed to be an engineering process :sad:.
  5. Compilers are buggier when writing parallel code than with sequential code – simply because fewer people write parallel code and therefore the compilers are less well tested in this regard. I have blogged about this before and unfortunately it is still true for many systems. I hope this changes soon as more people explore the realm of parallel programming.

There, now you have it – my personal hate-list about the very topic I usually love. I do realize that it is subjective. There are e.g. languages suitable for parallel programming that provide a very powerful platform – Java comes to my mind, which I am looking into right now and a lot has changed since the last time I have worked with it (especially with the concurrency features introduced in Java 1.5) – will post about this later. Whether or not its performance has improved since I last played with it, remains to be seen and is the topic of another post ;-). Anyways, feel free to add your very own reasons, what is it that bothers you most about parallel programming? I am very anxious to hear your comments!

53 Responses to Top 5 Reasons why I Hate Parallel Programming (Sometimes) »»


Comments

  1. Comment by Keith | 2007/05/08 at 18:22:16

    Good points. I’ve recently become more interested in parallel programming myself, and so have been looking at Erlang. You did not mention it here (although I see from other posts that you have spoken with Joe Armstrong. Have you used it in any of your research, and what are your thoughts on it?

  2. Comment by Adam | 2007/05/08 at 19:24:46

    Ditto to Keith’s question – what about Erlang?

  3. Comment by Steve | 2007/05/08 at 19:30:04

    If you’re thinking about looking back into Java, you might be interested in Scala as well.

    http://www.scala-lang.org/

    For some motivation, perhaps a writeup on Actors in scala might be more appropriate:

    http://debasishg.blogspot.com/2006/11/threadless-concurrency-on-jvm-aka-scala.html

    It would be interesting to hear your take on both when you get a chance to.

  4. Comment by Robert Brewer | 2007/05/08 at 19:31:25

    6. OOP makes parallel programming harder than it needs to be–you typically want to share all class definitions, but only share all instances of some classes. Unfortunately, the boundary between class attributes and instance attributes is purposefully muddy in most OO languages.

  5. Comment by Joseph Huangn | 2007/05/08 at 20:37:24

    The Mercury compiler can automatically parallelize code. http://www.cs.mu.oz.au/research/mercury/information/papers.html#wangp-hons

  6. Comment by John David Eriksen | 2007/05/08 at 21:27:34

    Profiling tool for UPC and SHMEM: http://ppw.hcs.ufl.edu/

  7. Comment by Michael Suess | 2007/05/08 at 22:22:59

    Keith and Adam: So many things to look at. I have started looking at Erlang in the past, but have only made it half way through the tutorial. I will get to spend some more time with it eventually and make sure to post about it then.

    Steve: Scala is on my list as well, and is in good company with e.g. the Intel Threading Building Blocks or the new languages like Fortress or Chapel. Unfortunately, nobody pays me to try out new languages, but my employer would rather see me do some research on my own ;-).

    Robert: I have answered an email from a colleague regarding OOP and parallel programming recently. My feelings are mixed in this regard and I will make sure to post about this as well soon.

    Joseph: To be honest, I have not had the best experiences with auto-parallelizing compilers. I have looked into them and the research that accompanies them at the start of my dissertation (3 and a half years ago) and have come to the conclusion that not much is happening in this field and that most compilers cannot do much more than parallelize the most simple loops. Maybe that has changed lately, but somehow I doubt it…

    John: I am not saying there are no parallel profiling tools. But in my experience, they are not as good as their sequential counterparts (yet).

  8. Comment by denis bider | 2007/05/09 at 01:54:50

    Michael – I am surprised to learn that, as an author of a blog on parallel programming, you remain uneducated about such a unique language as Erlang. I’ve never seriously used Erlang, however I think that if you are serious about making a PhD on parallel programming, Erlang is something you absolutely cannot skip. It would be like trying to get a driver’s license without knowing about how to drive on highways. It may take you a bit to really deeply appreciate Erlang and understand all the tutorial examples, because it’s different from the standard paradigm in so many different ways; but it’s really important to understand the approach this language takes because it’s so original. And it works.

  9. Comment by Vincent McBurney | 2007/05/09 at 12:36:40

    I will be subscribing to your blog to follow your progress. I blog about parallel ETL products like DataStage that try to utilize a parallel processing architecture while hiding the complexity of component instances and data partitioning, but I’d certainly like to know what is going on under the covers.

  10. Comment by Kent Boogaart | 2007/05/11 at 06:51:35

    My #1 hate is not being able to easily reproduce problems that occur in parallel environments. “I think I fixed the problem, but I’m not entirely sure”.

  11. Comment by Basil Vandegriend | 2007/05/13 at 16:26:51

    Interesting article! I’ve dabbled in concurrent programming in the past just enough to want to avoid it if at all possible, so I can relate to some of your points – #2 and #4 in particular.

    Check out my Problogger submission, which I think you’ll find relevant: Top Five Essential Practices for Developing Software

  12. Comment by KJK::Hyperion | 2007/05/14 at 11:00:46

    Personally, I have a thing for lock-free structures and transactional memory. Unfortunately, I see an overabundance of (fanta)scientific papers and a distinct lack of usable libraries, and the scientific work is sometimes so unrealistic I can hardly believe any of it could be implemented (“it’s very easy, as the garbage collector will take care of the hard part… no garbage collector? sucks to be you”, “we don’t need a full garbage collector for that, we *only* need an ad-hoc garbage collector with per-thread data that all other threads can read”, etc.). I’d prefer a straight answer, even if that is “can’t be done, just let it die”

  13. Comment by Martin Dowie | 2007/05/16 at 20:41:24

    Have you tried Ada (specifically Ada2005)? It’s a language that can perform as well as C or C++ and has had parallelism built (aka ‘tasking model’) into the language since its first incarnation (Ada83). Of course, back in those days that made way too many demands on the OS and it got a bit of a bad rep. but not a problem today.

    It should certainly knock points 3 and 5 on the head.

    The latest language revision added OO support to the tasking model, so you now derive from an active class.

    Check out https://libre.adacore.com/ for a free (beer & speech) compiler, IDE and add ons (XML, CORBA, Gtk, etc).

    And its not just the military that are using it. Digital TV, SatNav and Processor Fabrication are some of the more ‘unheard’ of users.

    Cheers
    — Martin

  14. Comment by Beliz Senyuz Saybasili | 2007/10/02 at 18:42:11

    Hi Michael :)

    Did you hear XMT? XMT is Professor Uzi Vishkin’s research project at the University of Maryland (http://www.umiacs.umd.edu/~vishkin/index.shtml).

    A brief description from SPAA’07:
    XMT (eXplicit Multi-Threading) on-chip general-purpose computer architecture is aimed at the classic goal of reducing single task completion time. It gives an easy general-purpose parallel programming model, and provides good performance with any amount of parallelism provided by the algorithm (up- and down-scalability).

    And this is the most striking: XMT is easy to program. There is a parallel programming course this semester for high school students.

  15. Comment by Adam K. | 2009/05/02 at 19:30:21

    Give LabVIEW a chance. It’s inherently parallel, and makes writing programs that take advantage of multicore extremely intuitive.

    Disclaimer: I work on the product. I’m still a programmer, though, and I really think it’s a cool product that more people should try.

  16. Comment by scatman | 2009/10/09 at 08:46:35

    “not powerful enough (C, C++, Fortran)”

    c and c++ mpi parallel programming are not powerful for you!!??


Trackbacks & Pingbacks »»

  1. Pingback by Top 5 - Group Writing Project Day 2 | 2007/05/09 at 09:08:28

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  2. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  3. Pingback by Top Five Group Writing Project: Day Two | 2007/05/09 at 13:59:05

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  4. Pingback by Top Five Group Writing Project: Day Two | 2007/05/09 at 13:59:05

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  5. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  6. Pingback by Group Writing Project - Top 5 day 2 | Techie Buzz | 2007/05/09 at 17:51:13

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  7. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  8. Pingback by Day Two - Top Five Posts - momeorganizer.com | 2007/05/11 at 15:46:53

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  9. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  10. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  11. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  12. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  13. Pingback by Top 5 - Group Writing Project Summary | Slaptijack | 2007/05/12 at 09:12:06

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  14. Pingback by Jestertunes » Group Writing Project Full List | 2007/05/12 at 09:33:13

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  15. Pingback by Jestertunes » Group Writing Project Full List | 2007/05/12 at 09:33:13

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  16. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  17. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  18. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  19. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  20. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  21. Pingback by Top 5 Group Writing Project Is Now Complete | 2007/05/13 at 03:14:33

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  22. Pingback by Runningmonkeys » Blog Archive | 2007/05/13 at 06:42:20

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  23. Pingback by Some link love to fellow Group Writers | Blogging Notes | 2007/05/13 at 08:14:13

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  24. Pingback by Wordpress Tips » Adsense Theme Wordpress | 2007/05/13 at 16:24:49

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  25. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  26. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  27. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  28. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  29. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  30. Pingback by A million top 5’s | 2007/05/17 at 13:29:37

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  31. […] post that struck me most recently was his “5-things” post. One of my largest complaints about parallel programming are the debuggers. In many […]

  32. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  33. Pingback by Full List for Top 5 Contest — My Life With IT | 2007/08/03 at 12:02:46

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  34. […] College 721.  Top 20 Reasons why Web Apps are Superior to Desktop Apps 722.  Top 5 Reasons why I Hate Parallel Programming (Sometimes) 723.  Top Reasons to use LinkedIn 724.  17 Tips that will save you time in […]

  35. Pingback by The Power of Linking | 2007/08/12 at 06:56:20

    […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  36. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

  37. […] Top 5 Reasons why I Hate Parallel Programming (Sometimes) by MIchael Suess […]

Leave a Reply

HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>