This is the fourth post in my Interviewing the Parallel Programming Idols-Series. I don’t think I need to introduce my interview-partner today to anyone who has done threads-programming (except maybe to the Windows-folks). His name is David Butenhof, and he has left a big footprint in the POSIX Threads standard. David presently works for Hewlett Packard and can frequently be found on comp.programming.threads (the newsgroup on everything threads-related). And let’s not forget he wrote a great book on POSIX threads: Programming with POSIX Threads. I have called this book the bible of POSIX Threads in the past, and I will repeat it here.
But since you are not reading this article to hear me praising him, but for his answers, I will start with the interview now. Five questions about parallel programming in general first:
Michael: As we are entering the many-core era, do you think parallel computing is finally going to be embraced by the mainstream? Or is this just another phase and soon the only people interested in parallel programming will be the members of the high performance community (once again) ?
ALL modern operating systems, including Mac OS X and Windows, have been “inherently multiprocessing” long before multi-core chips trickled down from the enterprise to the desktop. And even a user who only launches sequential programs benefits from having multiple cores. OS daemons and drivers, for example, can be scheduled on the other core(s). And many modern support libraries (including especially the OS bundled graphics frameworks) are inherently parallel even if the application doesn’t think it is. Pop up the Mac OS X activity viewer, or the Windows task manager, and see how many processes have a single thread. Sure, there ARE some; but a tiny minority of the processes running on any typical system.
I don’t think any of this is going to go away. After all, it started long before multi-core chips, for completely different reasons, and only ACCIDENTALLY allows trivial exploitation of them.
What will change is the methodologies and techniques by which the capability is exploited at the user/application level. We’ve seen widespread adoption of basic threading API technologies; and even though there are indeed still a lot of programmers hanging back in fear of the unknown, the front wave is pressing on beyond that simplicity into generalized lock-free techniques and alternate language metaphors.
“Always in flux, the future is.” I don’t think the genie is going back into the bottle anytime soon; but the thing about genies is that almost nothing is impossible… but a lot is “inconceivable”.
Michael: From time to time a heated discussion evolves on the net regarding the issue of whether shared-memory programming or message passing is the superior way to do parallel programming. What is your opinion on this?
Michael: From your point of view, what are the most exciting developments /innovations regarding parallel programming going on presently or during the last few years?
Transactional memory has a lot of promise to simplify synchronization and visibility issues across all models of shared memory multiprocessing; but (and?) it’s still young.
Michael: Where do you see the future of parallel programming? Are there any “silver bullets” on the horizon?
Michael: One of the most pressing issues presently appears to be that parallel programming is still harder and less productive than its sequential counterpart. Is there any way to change this in your opinion?
So much for the first part of this interview. Without further ado, here is the second part about POSIX Threads:
Michael: What are the specific strengths and weaknesses of POSIX Threads as compared to other parallel programming systems? Where would you like it to improve?
Michael: If you could start again from scratch in designing POSIX Threads, what would you do differently?
But if you go deeper, the biggest flaws in POSIX threads really aren’t in POSIX threads at all, but in the fact that we relied on existing language syntax and semantics to support the API. We did it deliberately, and had no practical choice at the time because language designers wouldn’t adapt until thread technology had been proved in the real world. THAT needs to be addressed, and still can be. A real parallel-safe memory model for C++ and C, for example, is long overdue. ANSI C needs to borrow some simplistic form of exception technology from C++ — even if it’s only based on Microsoft’s minimalist try/except syntax. For consistency and reliability, for example, POSIX cancellation needed to be an exception, as it was in the original CMA. And truly portable POSIX thread code would be a lot easier with a real parallel-safe C memory model.
If such changes happen, they’ll enable (in fact, require) the POSIX thread standard to become more like what it should have been in the first place.
Michael: Are there any specific tools you would like to recommend to people who want to program in POSIX Threads? IDEs? Editors? Debuggers? Profilers? Correctness Tools? Any others?
There are a wide range of others, but I’ve never really kept up on all of the development; and the landscape changes awfully fast.
The biggest problem people see is asynchronous memory corruption. It’s really hard (and expensive) to monitor and detect. (Though Visual Threads had hooks to help, at a big cost.) Hardware monitoring technologies to express and detect “rule violations” may be essential here. There might even be work on it, though I haven’t seen any. Transactional memory is, in a sense, an attempt to get around all of this; but if it ever gains widespread deployment, that day is a long way off.
Michael: Please share a little advice on how to get started for programmers new to POSIX Threads! How would you start? Any books? Tutorials? Resources on the internet? And where is the best place to ask questions about it?
The comp.programming.threads newsgroup is a great resource for anyone involved in threads. The discussions can be fascinating and educational. (I’ve usually followed it fairly regularly and frequently contributed… between an extremely hectic schedule and unreliability of the HP news server I’d been using, I haven’t “been there” much in the past few months.)
Grab a system and experiment, once you know anything at all. See what happens, think about why, and figure out experiments to increase your knowledge. “The Scientific Method” applies to engineering, too; and hard-won personal experience is often far more valuable than something you’ve read.
Michael: What is the worst mistake you have ever encountered in a POSIX Threads program?
Then again, as the old saying goes, “wisdom comes from experience, and experience comes from lack of wisdom”.
Michael: What a very fine quote to end an interview with, thank you very much for your answers!