Thinking Parallel

A Blog on Parallel Programming and Concurrency by Michael Suess

How Much of the Industry Will Go Parallel?

IndustryFrom time to time, one of my readers asks a question via email that just keeps me thinking. And sometimes, when I realize the question is not only interesting for me, but maybe for you as well, I may decide to post it and make an article out of it. Just like last week, when a reader asked: How much of the software industry will have to deal with the concurrent computing problem? This article contains my thoughts on the subject and also allows me to let my mind wander in search for the applications that this kind of power will allow.

The short answer to the question: Many people in the software industry need to deal with it. How many exactly, I cannot tell, because I don’t have the big picture (TM) on the software industry as a whole.

I can see two groups that will definitely embrace parallel programming: the first one is everyone with a program that has substantial calculations involved – or in other words: all compute-bound programs. Game programming is the premier example, but stuff like image-processing, multimedia applications and I don’t know what else will surely follow (or already do).

It’s quite easy to ignore the fact that there are two cores in today’s machines, as there are probably more than enough processes on the machine to keep them at least somewhat busy. Firewalls, anti-virus-software and things like that will happily work on the second core, keeping one core free for the program I am actually working in. The same thing probably somehow applies to the quadcore machines that will be sold next year (or already are sold right now).

Things get harder when we have 8, 16 or maybe 80 cores to keep busy. As many of you know, Intel is developing a machine with that many cores on a single chip. Every program that is not spending its entire time waiting on the user cannot afford to ignore this potential, as the users are going to wonder why they have to wait for their programs, while their task manager shows a lot of idle resources.

I would call this first group the performance group. They have performance problems and the only way to solve them on today’s machines is to embrace parallelism. As I said, I have no idea how big this group is, but I bet it is growing.

The second group of software developers that will embrace parallelism is probably even smaller, but if I were you I would try to watch it closely. For them, parallelism is not a burden – many people in the first group I mentioned probably think so, because in the good old days there was no need to parallelize programs, they would just get faster in time with no effort involved. This second group views parallelism as a chance to do new things and provide value that was not possible before.

I will try to explain it with an example: Your email application most likely already uses threads to keep the user interface responsive while connecting to your mailbox in the background. It also most likely has fancy search features included. What if both of these capabilities would be combined and your email application analyzed the text you are typing as you type it, looking for similar mails you have written or received in the background and keeps them at a handy place? An application like that is possible with today’s computing power – yet I have never seen it.

Searching through your whole hard drive using one of the desktop search programs that were all the hype a few years ago is another example – I don’t want to change my application and type in a search phrase into these programs, I want them to work in the background to provide me with the information I need know – without me explicitly asking them for it.

Let’s see what else I can come up with: While you are chatting with a friend using e.g. Skype, a voice recognition software could run in the background on a different core, analyze your conversation and try to understand key phrases. Or even the main nouns would be enough. And while you are busy talking about a certain topic, in a convenient location a few websites could automatically be loaded that are related to your conversation to make you appear more knowledgeable on a subject. Or present you with a few Ebay-listings. Wikipedia. Google search results. Related books on Amazon. The possibilities are endless and the computing power to make it possible is here today or will be available shortly.

Do the ideas presented here make sense? I don’t know. Maybe none of them would actually work and provide additional value. But I have put them down here so you can see what I mean when I am talking about innovation through parallel programming. People that are possibly a lot smarter than me will come up with creative ideas to use the cores and the tremendous computing power that will be available when there are 4, 8, 16 or even 80 cores in your machine. And these are the people and ideas I would keep a close eye on, as some of them may dramatically change the way we use our computers every day.

I want to close this article with a short challenge: what would you do, when you had 80 or so cores to spare? What kind of program do you crave for that this kind of parallel programming power makes possible in the first place? Thanks for sharing!

13 Responses to How Much of the Industry Will Go Parallel?


  1. Comment by Barry Kelly | 2007/05/30 at 03:28:49

    Almost all the industry will go parallel – it already is doing so. But it won’t necessarily be language-level parallelism.

    Most software development is of in-house or consultant-led line-of-business data manipulation applications. These systems are slowly but surely moving towards using Web-based front ends, whether Internet or intranet, because of maintenance and deployment benefits.

    Server-based applications are naturally parallel. A single thread dealing with a single request / response transaction doesn’t need to do much in order to scale fairly linearly with processor core count, all things being equal.

    GUI applications are different, there’s no doubt. But if you count the applications you deal with on a daily basis, I’d be willing to bet that almost none of them are desktop-based. Most applications these days are things like content management systems (all those blogs run on something) or more distributed querying systems (Google, Amazon, etc.).

    I know this isn’t what’s going to be using the CPU cores on the client’s machines, which is what you’re talking about, but frankly, desktop development is very much niche these days. Microsoft ships greater than 50% of all shrink-wrapped software, in the form of Windows and Office. Games and virus protection eat up a good chunk of the rest, I’d guess.

    After those have been taken out, there are only narrow areas left: things like free / shareware utilities (WinZip, CD ripping, etc.), or imaging applications and similar things which don’t map well to the web.

    Desktop development is not a growth market, not until the desktop browser gets replaced or transformed into something more powerful (I’m thinking Silverlight, Apollo, etc.) – but I wouldn’t hold my breath waiting for that to happen. The Achilles heel of applications built on Flash etc. is the very lack of a text-based definition, and the consequent opaqueness to search, and its attendant ad revenue.

  2. Comment by Kent Boogaart | 2007/05/30 at 08:09:46

    I was thinking about a related take on this the other day. I was thinking about how *users* would want their cores to be utilized and how applications might (or might not) intelligently throttle their use of cores. “Core Etiquette”, if you will.

    For example, it might be possible for video encoding software to utilize all 80 cores of your machine and get the job done in 5 minutes instead of 6 hours. However, would the user *want* all their cores pegged for the 5 minutes, or would they prefer to wait a little bit longer for the sake of responsiveness? How might the application determine whether it should use all 80 cores (ie. start 80 threads), or a subset thereof?

    How will multiple CPU-intensive applications cooperatively utilize the correct number of cores? Whose responsibility is it? The applications? The O/S? The users?

    Thanks for the great post!

  3. Comment by Vagif Verdi | 2007/05/30 at 08:57:16

    Yeah right, 80 cores my foot! Hard Drive, RAM, and network connection will become a bottleneck much earlier, than you will be able to utilize all your cores.
    For massive parallel computing we will have to first retire outdated pc architecture.

  4. Comment by bong | 2007/05/30 at 09:23:33

    What about the presumed drop in core speeds? If you put 100 cores in a processor, it is *not* possible to keep the core speed even close to the level of current multicores without a house-size cooling system…

    I think that all applications need to use parallelism in the future for the computing power of a single core in such a processor is not enough. Current programming models, frameworks and languages surely won’t be enough.

    For a reference, read: “The Landscape of Parallel Computing Research: A View from Berkeley” by Asanovic et al. []

    What is your take on this?

  5. Comment by Tom Kirby-Green | 2007/05/30 at 10:23:25

    Well first of all if I look at the threads in a potentially runnable state on my Windows box then it’s fairly clear that an OS scheduler could make useful use of a lot of cores. Which would give me a ‘smoother ride’ as a end user. With regard to what I’d use those cores for today – dealing with I/O latencies and servicing some PLINQ type infrastructure would be my first bet – I most certainly wouldn’t want to be doing 80 * _beginthreadex explicitly 🙂

  6. Comment by Terrence Liao | 2007/05/30 at 14:10:04

    Companies who have some sort of scientific modeling have been doing parallel on parallel system from the beginning. It is a question how fast those desktop applications will be (or will not be) migrated to multi-core arch on PC. The “do-nothing” approach will improve the performance as long as the cpu’s clockspeed does not decrease (hopping with the 4.7GHz POWER6 in the market now, we will see clock speed to increase again.) . But long-term solution might be parallelism. But does it really matter to most users that runtime of a desktop app drop from, say 4 seconds to 1 second in quad-core. There is also question how many developers can think parallel, if not, their parallel port apps might post more runtime problem. (Take me about 2 months to fix a Heisenbug which has been sleep for many years! Picture this sometime in the future, a app on my many-core PC suddenly frozen, and the user support told me, it is O.K. only 1 very 1000 people will have this problem….. What a nightmare!)

  7. Comment by Bart Samwel | 2007/05/30 at 20:50:51

    Or present you with a few Ebay-listings. Wikipedia. Google search results. Related books on Amazon.

    And of course ads. Remember Google’s eavesdropping software? More cores could definitely help provide us with more targeted ads, based on anything we’re doing. I wonder what business models that will lead to. I hope it will help to keep all of the content we’re accessing free of charge. 🙂

  8. Comment by Rintoul | 2007/05/31 at 01:36:28

    Much is already being done in parallel for sure. Superscalar, parallel pipelines, VLIW, this kind of stuff is done at the hardware level already – a lot of parallelism types of things have been implemented at the hardware level that started out as stuff uncovered in compiler research… I say keep as much invisible to the programmer as possible!

  9. Comment by Michael Suess | 2007/05/31 at 09:56:27

    Thanks for all your very insightful comments, keep them coming, I am enjoying them a lot. I don’t have the time to comment on every comment, as I am preparing for a trip to China right now, but insightful as they are, most of them could not benefit much from a comment from me anyways 🙂

  10. Comment by Alexander Petrov | 2007/05/31 at 19:38:35

    This second group views parallelism as a chance to do new things and provide value that was not possible before.

    Google think so too… –
    Google equip his new offline\online application with workerpool – host for JavaScript “processes” with message-oriented collaboration architecture (smth like Erlang style).

  11. Comment by rodrigob | 2007/06/04 at 13:23:30

    What if […]your email application analyzed the text you are typing as you type it, looking for similar mails you have written or received in the background and keeps them at a handy place? An application like that is possible with today’s computing power – yet I have never seen it.

    Dashboard, almost vapourware, but a functional alpha version exist.

  12. Comment by Naveen | 2007/07/26 at 00:00:26

    Regarding your comment about non-intrusively, intelligently presenting information based on current context (ie searching other emails while typing a current one) – Remembrance Agent was an early research prototype.

    It did something similar, although it indexed your entire home directory and used emacs for the interface:

    But it’s definitely a good use of a multicore architecture.

Trackbacks & Pingbacks »»

  1. […] Seuss ponders one of my favorite questions: How much of the software industry will have to deal with the concurrent computing [opportunity]? He hits the vital […]

Comments are closed