Thinking Parallel

From time to time I get an email asking what books I recommend for people to learn more about parallel programming in general, or about a specific system. You need to ask no more, as this is my list of recommended books. Please beware that the links below are affiliate links to your local version of Amazon, which basically means I will be awarded a small amount of money each time you purchase a book there after you clicked on any of the links - and also beware that it merely boils down to a couple of dollars each month, if at all. And of course, it has no disadvantages at all for you. But if you want to show your appreciation for my work, it’s still a nice token of reward .

General

I have to admit I cannot really recommend a single book to cover the topic of parallel programming wholeheartedly. The book on the left by Grama et. al. is the one we generally recommend to our students. It covers everything there is to know about the parallel programming basics: from architectures, algorithms and analytical models up to specific parallel programming systems (OpenMP and MPI). Unfortunately, it is not exactly what I call a joy to read. Also, at times it is a little imprecise. Yet, I have not been able to find a more readable book of this scope and until I do I have to stick to recommending it. If you can only afford one book on parallel programming, this should be it.

POSIX Threads

This one is easy. I have called this book the bible on POSIX threads before and I still vouch for it. Written by David Butenhof, one of the original creators of the standard, this book covers everything from a nice introduction to the general concept of threading to all the gory details, while still being a joy to read. Whether you are a beginner to working with threads or an old hand that is looking for a reference - if you are programming POSIX threads, this book should be on your desk. Period.

OpenMP

If you ask me there is no really good book on OpenMP. But truth be told, there is not even much of a need for one. There are two reasons for that: First, OpenMP is relatively simple, at least when compared to other threading systems. When you have understood the basic concepts of threading, OpenMP should be a peace of cake (which is the reason I like it so much). The second reason is: there is a very nice and understandable resource about it on the net to teach you every detail there is to know about it: the OpenMP specification. No, I am not kidding you, the specification is actually very readable, covers every possible detail and even has a huge appendix with examples. Everything there is to know about OpenMP available in one convenient file - and it is even free for everyone to download .

MPI

MPI has a big specification with more than 300 functions. This is the book (or better: the books, as it is really two volumes) I use to keep track of all of them. Don’t expect much introductory material in there, though, as it is really just a reference. A good one, though. If you are a beginner to MPI, you will probably do better with a copy of Using MPI or Using MPI 2.

C#

According to my own experiences and to the ones from my readers, there is no really comprehensive book on the topic of C# and threads (yet). I recommend the book on the left for beginners, as it offers a nice introduction to the topic - but beware that it is merely an introduction. There is also part of a new book by Joseph Albahari on C# 3.0 available on the net, which offers a more thorough introduction to C# as well (and lucky us, the part on threading is available for free ).

Java

I like this book so much I have written an article of praise for it here. I unconditionally recommend it for anyone involved in programming in Java. It does not really cover the absolute basics (although these can be learned on the fly while reading the book just by looking into the excellent Java documentation). Yet, all of the content is well-presented, a joy to read, includes helpful examples and is up-to-date (Java 1.6 is covered). The authors were heavily involved in the creation of the java.util.concurrent package introduced in Java 1.5., which is why they obviously know what they are writing about. If you want to program with Java threads, this is the book to have.

Erlang

There is really only one book on Erlang available (yet). This will change soon, as Joe Armstrong has written another one on the same topic. It is supposed to be even better, yet I have not read it, therefore I cannot recommend it (yet). The one shown on the left here covers everything there is to know about Erlang and is a good read - although it is not exactly new. Whether or not you want to wait for the new book is up to you (should be out at the end of July)…

So much for my recommendations. I have only given recommendations on books that I actually know and systems that I have some kind of experience with. But feel free to comment on your own favorites below, even if the system is not mentioned here (yet). Hope it helps someone to make their first steps in the vast world of parallel programming !