Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

Ten Questions with Joe Armstrong about Parallel Programming and Erlang

Joe ArmstrongThis is the first post in my Interviewing the Parallel Programming Idols-Series. My interview partner today is Joe Armstrong, one of the founding fathers of the Erlang programming language. He presently works for Ericsson, where he designed the very first version of Erlang (that was a long time ago in 1986). He is also co-author of the book on Erlang to date: Concurrent Programming in Erlang. And soon, we will have the pleasure of having another book from him in association with the Pragmatic Programmers (this time with him as the main author) – Programming Erlang: Software for a Concurrent World. Besides writing books and inventing programming languages, he teaches courses on Erlang, pursued his PhD in computer science, founded one of the first Erlang startups and enjoys his motorcycle. I wonder where he finds the time to do all that. :D

But enough with the preface, let’s start with the interview. The first five questions are about parallel programming in general:

Michael: As we are entering the many-core era, do you think parallel computing is finally going to be embraced by the mainstream? Or is this just another phase and soon the only people interested in parallel programming will be the members of the high performance community (once again) ?

Joe: mainstream yes yes yes – we’re talking about potentially doubling efficiency on a dual-core quadrupling efficiency on a 4x core etc. I have had programs running 18 times faster on a 32 core machine. This is x 18 – we’re used to looking for 5% – 10% improvements by re-coding parts of our applications not 400% or 1800% – this effects all software.

Michael: From time to time a heated discussion evolves on the net regarding the issue of whether shared-memory programming or message passing is the superior way to do parallel programming. What is your opinion on this?

Joe: Shared memory systems are impossibly difficult to understand and get right, especially when errors occur. It’s bad enough without errors, but with errors it’s virtually impossible. If a program holds a lock and is messing around with shared memory and crashes how the heck can the other processes know what it was doing and fix up the errors?

Transaction memories and pure message passing system suffer from none of these problems.

Michael: From your point of view, what are the most exciting developments /innovations regarding parallel programming going on presently or during the last few years?

Joe: P2P architectures, distributed hash tables, planet-scale integration of systems things like planet lab etc.

Things like bit torrent are destroying traditional industries, the spinning jenny destroyed the home yarn spinning industry in the 1770′s bit torrent is doing the same thing to the record industry. It’s not a question of if it happens it’s a question of when.

Michael: Where do you see the future of parallel programming? Are there any “silver bullets” on the horizon?

Joe: Yes – pure message passing infrastructures will solve many problems (things like amazon simple queue service). This is back to Paul Morisson’s flow based programming.

Michael: One of the most pressing issues presently appears to be that parallel programming is still harder and less productive than its sequential counterpart. Is there any way to change this in your opinion?

Joe: It’s not harder – it’s hard because the dominant model is locks, threads, shared memory – now that *is* hard. Pure message passing systems are easy to program and reason about.

The world *is* parallel – we are parallel.

So much for the first part of this interview. Without further ado, here is the second part about Erlang:

Michael: What are the specific strengths and weaknesses of Erlang as compared to other parallel programming systems? Where would you like it to improve?

Joe: Strengths: It was designed for building fault-tolerant systems, so it degrades nicely in the presence of software and hardware failures. It takes failure seriously. Processes and failure handling is a defined part of the language and NOT something provided by the OS.

Weakness – distributed Erlang has all-or-nothing security. It would be better to have different levels of security and capability or trust based security models.

Michael: If you could start again from scratch in designing Erlang, what would you do differently?

Joe:

  • Make modules and code handling higher-order, add introspection.
  • Add a layer of contracts for asserting properties of protocols.
  • Make protocols into first-class citizens in the programming language.
  • Make a trust/capably layer for distributed computing.
  • Also make layers for weakly-connected distributed systems. Distributed Erlang is only for cluster-like computing with strong security.

Michael: Are there any specific tools you would like to recommend to people who want to program in Erlang? IDEs? Editors? Debuggers? Profilers? Correctness Tools? Any others?

Joe: No – 95% of all the fun can be had with the compiler alone. Then people get very religious about their IDE’s etc. Me, I use emacs and make and xterm. There are pointy clicky things which some people love but I can’t speak about these from personal experience.

Michael: Please share a little advice on how to get started for programmers new to Erlang! How would you start? Any books? Tutorials? Resources on the internet? And where is the best place to ask questions about it?

Joe: Books – (shameless plug) read my new book – it will be coming out really soon.

Tutorials: www.trapexit.org
Distribution: www.erlang.org
Questions: erlang mailing list (sign on at www.erlang.org)

Michael: What is the worst mistake you have ever encountered in a Erlang program?

Joe: Programs without documentation – I hate it when people say “read the code” – the code is the *solution* to a problem – by reading the code I have to *guess* what the problem was, and this is error prone. I like to be
told what the problem was, not guess.

So the worst mistake was when I wrote a multi-page comment to explain some particular tricky point in a program only to find that in a later version somebody had removed the entire comment.

Michael: Thank you very much for the interview!

19 Responses to Ten Questions with Joe Armstrong about Parallel Programming and Erlang »»


Comments

  1. Kig
    Comment by Kig | 2007/03/20 at 19:43:37

    I really wish he’d implement the things that he wishes he’d done differently in Erlang.

    I’d sign a petition or contribute to a pool to get him to try.

  2. Comment by Christopher C. Aycock | 2007/03/21 at 02:12:05

    Message-driven communication like what Erlang has (essentially asynchronous RPCs) are really a much better approach than either MPI or pthread. MPI is like assembly programming, whereas pthread is like writing sequential programs with all the data in global variables. There’s a reason Hoare’s CSP uses messages for communication.

  3. Comment by anonymous | 2007/03/21 at 23:32:28

    “what are the most exiting developments”

    What’s wrong with this picture?

  4. omo
    Comment by omo | 2007/03/22 at 17:45:00

    Fun to read !

    I made a Japanese translation of the interview.
    Which is available at
    http://www.hyuki.com/yukiwiki/wiki.cgi?TenQuestionsWithJoeArmstrong
    If you have any trouble with this, please contact me.

    Thank you for good article.

  5. Comment by Alexander Petrov | 2007/03/22 at 18:24:15

    And I’ve translated the interview to Russian.
    Here:
    http://www.rsdn.ru/Forum/Message.aspx?mid=2414192&only=1

    I hope you can approve it…

    Thank you for advance.

  6. Comment by Michael Suess | 2007/03/22 at 21:48:46

    I have no problem with you translating any of my posts as long as you link back to my original (which you have obviously done). I also do appreciate the notification…

  7. Comment by Andrew Chase | 2007/03/23 at 01:18:27

    I thought your questions, especially in the first half, were lacking. Having read anything at all about/by Joe previous to this interview the reader already knows how Joe would respond to at least half these questions (i.e. the argument of share memory versus message passing).

    Your efforts are certainly appreciated, but a focus on hearing new info would be great (at least for this reader :-)

  8. Comment by Michael Suess | 2007/03/23 at 14:36:58

    Hi Andrew: I appreciate your comments. I know Joe’s answers to one or two of the questions were not that difficult to guess when you already know him (e.g. when you hang out on the Erlang Mailing Lists often). But I suspect most of my readers don’t do that and don’t know Joe that well – and besides, I hope a lot of fun will come from the differences between the answers of the people I have asked – and I can assure you that they don’t all agree to the points Joe raised :-)

  9. Comment by Grant | 2007/04/03 at 22:48:11

    I don’t see a parallel language ever becoming mainstream without correctness, performance, and other development tools. I was quite disappointed with Joe’s answer to the query about development tools. Are there really no developer tools available for Erlang, or did I misinterpret his answer?

  10. Comment by Paul Morrison | 2007/04/17 at 21:28:16

    I appreciate the kind words from Joe, and of course I agree with most of what he says! Maybe you might find the FBP web site interesting, and perhaps even contribute to the FBP Wiki – that goes for anyone reading this web site, of course!

  11. Comment by A Jackson | 2007/04/21 at 18:15:36

    Grant: You missundersood what Joe said. HE doesn’t want to recommend any, he uses Emacs.
    But there are others, like additions to Eclipse etc.

  12. Comment by Ian Davis | 2007/04/25 at 14:11:26

    In response to Joe,

    Michael stated “parallel programming is still harder and less productive than its sequential counterpart.” To which Joe replied “It’s not harder – it’s hard.” To keep this short:
    1. How is debugging a parallel application easier than a serial application? How does this not make it harder?
    2. How can you verify the correctness of your parallel algorithm? Verifying serial algorithms is relatively straight forward.
    3. How does the introduction of needed fault tolerance and load balancing not increase the difficulty? What does one do to minimize the synchronization time? How do we recover when the master dies? Worker? The job cannot be allowed to stop and we have to manually figure out how to recover. How do we deal with problems whose processing pieces are not a multiple of the number of nodes we have? Along that line, what about the cache/memory/swap envelope?
    4. How does having to rewrite the serial application into a parallelizable form not increase difficulty? This includes trying to figure out which parallel algorithm will have the best performance.

    -Ian

  13. Comment by Ben St. John | 2007/04/25 at 15:27:27

    Michael, I like your idea to interview some top people, and its execution, but I do feel you soft-balled things a bit for Joe. For example, the claim that parallel programming in Erlang isn’t harder, the claim that the world is parallel (well, yes, but it’s also serial, and *so what*?), and also the claim that message passing is the way to go all would have benefitted from a follow-up question.

    Regarding the last claim, while I agree in many cases it is, and it definitely makes things simpler, it also has costs, and this should at least be recognized.

    Nonetheless, thank you!

  14. Comment by Michael Suess | 2007/04/26 at 21:35:42

    I do realize Joes answers were the most controversial of my interviews. Please also note, that I did not try to “softball” anyone, all of the interview participants were asked the same questions – and no more. But obviously Joes answers raised the most interest, therefore maybe I will ask him to answer your comments some day…


Trackbacks & Pingbacks »»

  1. Pingback by links for 2007-03-23 « Bloggitation | 2007/03/23 at 02:22:13

    [...] Ten Questions with Joe Armstrong about Parallel Programming and Erlang (tags: erlang programming web) [...]

  2. [...] 暂时没有时间翻译,先把原文发上来,原文地址在这里 。 [...]

  3. [...] 转贴注: 在前面转贴的“ThinkingParallel 对 Joe Armstrong 的访问”中,Joe Armstrong 自己也提到,如果现在有机会让他重新再来实现 Erlang,他很想引入一个全新的安全机制(可能类似于Java的,不过我认为Java的安全机制过于复杂,很难理解),以取代目前的“All or None”权限控制模式。 [...]

  4. [...] I just came across this blog post: http://www.satine.org/archives/2007/12/13/amazon-simpledb/. The author got the scoop that Amazon’s just-released SimpleDB runs on Erlang. This is actually the second Amazon Web Service that runs on Erlang — at least, if the rumors I’ve heard that Amazon SQS was built with Erlang as well are true (update: I found the reference in this article: http://www.thinkingparallel.com/2007/03/20/ten-questions-with-joe-armstrong-about-parallel-programmi…). [...]

  5. [...] is a concurrent functional programming language that is claimed to be well suited for parallel programming.Some experts however think otherwise. The author makes a good case for Erlang not being [...]

Leave a Reply

HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>