The real problem exists in determining the critical section of your source code. Game Developer or Hobiest, you still have to deal with splitting up what parts you want to run where. As of now I don’t know of any smart scheduler that will handle that for you.

Recalling what I read of PS3 developers, they had to either run sections of sound, video, mesh parsers on individual cores or designate critical sections to some sort of scheduler.

Nobody really knows what will work best. It takes development time that nobody seems to have anymore. It’s all about release dates. But that is a story for another time.

I have a dual-core 32-bit intel laptop. My real problem with the industry is that my parents’ single core 32/64-bit amd desktop leaves it behind as one would a bad dream.

What happens is if you want real performance then you need real tools. Logic sims for the processor and assembler commands “reference code, docs, .
You really need to know the architecture you are working on. Registers if you will. Think about the Jaguar, Atari developers. They knew the processors and interleave times. Once you have all that information gathered you can start programming for your then out of date processor which is another problem with the industry.

My first question is who cares about that multicore crisis? Should I care? And if, why?
We are talking about a problem software developers have or maybe will have (oh yes, WE already had the crisis in 1999). All developers? Me? No. Not me. Why?

Let me go into greater detail to explain what I mean. As cpus became faster and faster new CASE tools became available that allowdevelopers using more and more abstract designs for application development. Those tools create code, integrate libraries and put general purpose code into your application. The effect is a shorter time to market paid with more calculation overhead.

An example: Create a hello world application in 1970, 1990 and 2008. They all will say “Hello world” but the first will run 5000 cpu cycles, the second 500,000 and the last more than 10,000,000 cycles. The hello world app of 2008 won’t be slower though. Why? Because the cpu it will run on is more than 10,000 times faster that that of 1970.

Now back to the crisis. My problem is that the application I just created dosn’t meet the performance requirements on the target architecture. A faster cpu isn’t available and it doesn’t scale in a multicore environment. You can call that a multicore crisis if you want. Why not call it an overhead crisis?

There is so called redshifting business, right. Google, Ebay and the military won’t solve their capacity problems by going back to assembler programming. But as smoothspan wrote in his blog in 2007 - they already have a solution! So they aren’t hit by the crisis right now and won’t be in the nearer future.

Maybe the mid-sized software companies are. Those that deliver 95% of the applications in use by 95% of the people. Those that deliver applications that don’t scale well in multicore environments. Those that use the tools that generate code that keeps a cpu busy with overhead most of the time. Runtime analysis of legacy software systems (that, in my opinion, make up more than 50% of all software systems) shows that a cpu spends most of its time in calculating things the programmer has no idea of. Listen to them, they will tell you something like “what the hell is …” or “which crazy bastard created that routine?”.

Back to my optimization problem. I don’t have the knowledge and skills to transform my algorithms in well scaling parallel ones. Maybe I get it running on two cpus, maybe on four. But N is unreachable. But if I had, I wouldn’t do it anyways. This is mainly a question of my application doing the right things rather than doing the (wrong) things right (in this case: faster).

As long as applications are as described there will be no multicore crisis. Today’s business applications don’t perform operations 10,000 times more complex than those of 1970. Because today’s cpus deliver 10,000 times more calculation power there is no need for more cpus. After that it is all about a cost problem. The question is: Is parallel programming cheaper than creating less redundant code?

Maybe it’s true that we most programmers are not going to face the problem right now. And surely is true that up to a limit, having multiple cores can be automatically handled by the OS to do things in background or to be more responsive.

But still, it’s true that what is really matter is not how many threads do you have, but how many are you using at a time. Most of the time you are only running one application that is consuming all of your CPU and it does not matter that you have 10 threads in a background service at all. So that remark was simply stupid.

And the main problem is not that we have to face now with 2-4-8 cores, but that the tendency is to have more cores (and this is a fact) faster than we have adeguate solutions to multithreading (c++ is not ready, java neither)

And right now I’m working in the videogame industry and right now I have to face with 6-8 hardware threads (x360-ps3), and right now most videogame developers are trying really hard to refactor their engines to gain advantage of that, so it’s not something that is going to be ready for some distant future!

The only place where mulithreading is easy as of now is in numerical kernel codes that can be mapped to a stream programming model and that’s why GPUs are so fast, but I won’t call them a general strategy to deal with the problem at all!!!

Why don’t the CPUs with 32+ cores have cache coherency problems? I’m guessing that they must have greatly simplified caching schemes. Still the GPUs feel more like very fancy microcontrollers as opposed to general-purpose microprocessors. The GPUs are still rather specialized after all.

Yes I do believe that multicores have a lot of hype, many people are clamoring to make multithreading as “a mountain out of a molehill”. Sometimes it feels like multithreading is a solution just waiting for a problem to be solved. Sure the wall has been hit because the tricks and advances of the last 30 years (faster single CPU) can no longer occur and the common CPU must now have multiple cores.

That being said, it’s nice to have a machine with more than one processor. However, I/O contention and virtual memory thrashing will always any benefits from having extra processor(s).

On a Windows machine, the possibilities for parallelism are rather limited but I think this makes the parallelism game much simpler to play, so for Windows you really only have these cases where it is proper to use parallelism:
1) You want to run a long operation without holding up the UI thread (this makes sense for both single and SMP systems)
2) You must communicate with another process (again applies to both single and multi)
3) You have some large amount of data processing that could fit into a scalable solution (applies to multiprocessor of course)

Regardless of the reason involved for multithreading/parallelism, I believe the programmer is ultimately responsible for knowing what states must be shared across different threads. Use of general-purpose libraries can solve some but not all of these problems, and of course, these libraries will incur their own limitations as well.

I suppose this blog is mostly about #3, so the options for that are:
a) SIMD
b) OpenMP
c) DIY: create 1 thread / processor that does the heavy computing, these can of course use SIMD, but the coder gets to split up the work/data involved

To further complicate things, the quad Xeon chip, for example, has two separate dual-core dies next to each other in the same CPU package. So do we expect intra-CPU communication to be slower when moving off die? The many types of hardware flavors would wreak havoc for designing general purpose solutions for large data, scalable processing. Can the compiler or run-time actually minimize CPU waits caused by caches misses and/or memory synchronizations (flush caches)? (And to complicate even more, we could consider NUMA systems as well).

David: thanks for posting about the applications, that’s an excellent overview.

Michael: since you’re the academic, what is the true cost of thread synchronization? For example on an 8 CPU system, suppose CPUs 1 and 5 have their caches accessing the same memory. CPU 1 hits a lock and so must synchronize with CPU 5’s cache, i.e. they both must flush their caches? Is this correct? Is there a way to see cache misses in action, there doesn’t seem to be a perfmon counter for this?

Many thanks for the great blog!

-Chris O

http://www.panologic.com/why-pano/features.php

>>Thus I currently do not see a generally applicable, well-scaling model for these applications to benefit from many cores.

I’m certainly with you on this, at least when only single instances of these apps are running. Considering that hardware CPU virtualization is now ubiquitous, and OpenGL/DirectX virtualization is coming online, I wouldn’t be surprised if application servers for the home and office gain traction quickly. Who’d need actual thin clients…just KVM over wireless/ethernet and each new monitor buys you a new “computer”.

Especially on OS like Windows, which has the notion of an ‘UI thread’ which needs to handle all window messages it is really hard for such an application to distribute work among many threads. Sure you can spin of worker threads doing I/O etc. but sooner or later you have to update the UI and have to synchronize with the UI thread again, which can easily become quite tedious. Thus I currently do not see a generally applicable, well-scaling model for these applications to benefit from many cores.