Comments on: Parallel Programming is Entering the Mainstream - or isn’t it? http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/ A Blog on Parallel Programming and Concurrency by Michael Suess Wed, 09 Jul 2008 02:27:04 +0000 http://wordpress.org/?v=2.5 By: David Blake http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-21887 David Blake Tue, 04 Sep 2007 17:42:12 +0000 http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-21887 Consumers and business customers probably don't understand why it is difficult, and they wouldn't understand it if your tried to explain it to them. How could they possibly know how to ask for it then? Customers do want faster software, but that's the limit of their ability to express what they want--faster. We build and sell mainstream software that uses advanced concurrency, but our algorithms to date have dependencies that prevent us from doing parallel algorithms. We do have unreleased code that uses parallel algorithms. We still have managed to design the software from the ground up with the multi-core consumer desktop in mind. The big problem is that today's university curriculum is moving away from teaching about pointers and other "complex" issues in favor of languages with runtimes that do garbage collection and other services. This school of thought creates graduates who don't know how to debug real software, and who believe issues that you face in parallel programming should and will be handled by the runtime of the language flavor that is taught. If these C.S. departments had any vision, they would leave teaching .NET and Java to the Information Systems department in the business schools and keep the hardcore engineering topics in the engineering classes. Instead, we have a dilution of the quality that the engineering degree used to represent. Consumers and business customers probably don’t understand why it is difficult, and they wouldn’t understand it if your tried to explain it to them. How could they possibly know how to ask for it then? Customers do want faster software, but that’s the limit of their ability to express what they want–faster.

We build and sell mainstream software that uses advanced concurrency, but our algorithms to date have dependencies that prevent us from doing parallel algorithms. We do have unreleased code that uses parallel algorithms. We still have managed to design the software from the ground up with the multi-core consumer desktop in mind.

The big problem is that today’s university curriculum is moving away from teaching about pointers and other “complex” issues in favor of languages with runtimes that do garbage collection and other services. This school of thought creates graduates who don’t know how to debug real software, and who believe issues that you face in parallel programming should and will be handled by the runtime of the language flavor that is taught.

If these C.S. departments had any vision, they would leave teaching .NET and Java to the Information Systems department in the business schools and keep the hardcore engineering topics in the engineering classes. Instead, we have a dilution of the quality that the engineering degree used to represent.

]]> By: Vladimir Frolov http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-7024 Vladimir Frolov Fri, 20 Apr 2007 21:00:13 +0000 http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-7024 In my opinion parallel programming should be massively used on desktop to become mainstream practice. On one hand multicore and hyperthread desktop processors force software on desktop to be parallel, on other hand it is expensive now to create parallel software even for server side not to speak of desktops. Parallel programming systems (OpenMP, MPI, POSIX Threads, .NET and Java threads etc) are very complex to use it on desktop for most of applications. Developers have to have a lot of qualification to use such systems. Thus I think parallel programming becomes mainstream only when easy enough parallel programming system will be invented. Moreover, this system has to do the same thing which garbage collection doing for memory management. It has to allow developers to create parallel programs without explicit manipulations with threads and with any kind of synchronization. And I hope such kind of systems will be invented extremely soon. I know at least one attempt to devise such system, it is <a href="www.stressflow.com" rel="nofollow">stressflow</a>. Stressflow is not good enough to use it as is, but it is one tiny step in right direction. In my opinion parallel programming should be massively used on desktop to become mainstream practice. On one hand multicore and hyperthread desktop processors force software on desktop to be parallel, on other hand it is expensive now to create parallel software even for server side not to speak of desktops. Parallel programming systems (OpenMP, MPI, POSIX Threads, .NET and Java threads etc) are very complex to use it on desktop for most of applications. Developers have to have a lot of qualification to use such systems. Thus I think parallel programming becomes mainstream only when easy enough parallel programming system will be invented. Moreover, this system has to do the same thing which garbage collection doing for memory management. It has to allow developers to create parallel programs without explicit manipulations with threads and with any kind of synchronization. And I hope such kind of systems will be invented extremely soon. I know at least one attempt to devise such system, it is stressflow. Stressflow is not good enough to use it as is, but it is one tiny step in right direction.

]]> By: Haibing Shao http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1436 Haibing Shao Mon, 29 Jan 2007 22:18:57 +0000 http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1436 I am in Germany now and also in the Ivory Tower. :-) My field is ground-system modeling software development, dealing with groundwater contamination, geothermal and geochemical processes. For us, parallel computing is badly needed! Well, but we are too scientific... Personal opinion: Most of Current software might not need parallel. This is the list of my frequently used software: *SKYPE --> yes. not fast enough. parallel might help(well, dual-core/ one thread solve the problem) *MSN Messenger --> fast enough, does not help *emule --> The bottle neck is the internet *bittorrent --> same as emule. next bottle neck is the hard disk *media player --> fast enough, does not help *VS .net --> bottle neck is the hard disk *Mozilla Firefox --> fast enough *Office suit --> fast enough *Anti virus --> not fast enough, but dual-core/one thread solves it Ok, so why we need parallel for people with a normal life? Maybe: *games!!! of course visualization and AI will be much faster and better! (very possible) *voice-driven softwares (if vs.net has a function that can recognize my voice and type in " , ; -> . if () else for while true false int double ........ for me, I will be very very very happy:-)Heard that someone is developing that) My point is that: Parallel gives an opportunity to innovative ideas. So far there is not that much software on the market really which take that advantage. But the most prize is always for those astonishing ones. I am in Germany now and also in the Ivory Tower. :-) My field is ground-system modeling software development, dealing with groundwater contamination, geothermal and geochemical processes. For us, parallel computing is badly needed! Well, but we are too scientific…

Personal opinion: Most of Current software might not need parallel.
This is the list of my frequently used software:

*SKYPE –> yes. not fast enough. parallel might help(well, dual-core/ one thread solve the problem)
*MSN Messenger –> fast enough, does not help
*emule –> The bottle neck is the internet
*bittorrent –> same as emule. next bottle neck is the hard disk
*media player –> fast enough, does not help
*VS .net –> bottle neck is the hard disk
*Mozilla Firefox –> fast enough
*Office suit –> fast enough
*Anti virus –> not fast enough, but dual-core/one thread solves it

Ok, so why we need parallel for people with a normal life? Maybe:
*games!!! of course visualization and AI will be much faster and better!
(very possible)
*voice-driven softwares (if vs.net has a function that can recognize my voice and type in ” , ; -> . if () else for while true false int double …….. for me, I will be very very very happy:-)Heard that someone is developing that)

My point is that: Parallel gives an opportunity to innovative ideas. So far there is not that much software on the market really which take that advantage. But the most prize is always for those astonishing ones.

]]> By: Joshua Volz http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1027 Joshua Volz Sun, 21 Jan 2007 19:05:31 +0000 http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1027 I believe there is little or no customer demand for parallelism, particularly for the business software users. Their software is bottlenecked at the server (which is already running multiple processors and is already parallel) and a faster or more parallel desktop isn't going to help them at all. This is true of both LAN desktop and web based applications. How much parallel processing do you need to run IE or Firefox when the limiting performance step is your internet connection, the web server (already parallel) or the database (already parallel)? I believe most business users fall into this category. I do agree that game companies are going to take advantage of parallelism to great effect. From that perspective, a normal end user might be interested in parallelism. There is a lot of performance gain to be had because graphics programming is largely a mathematical exercise. We haven't heard about it in the gaming world (customers demanding parallelism) mostly because the customers would have to know what parallelism is before they could ask for it. Computer users are undereducated on this topic. I don't believe parallelism is stillborn. I am just not convinced that it is going to change the entire nature of programming over the next 5 -10 years. As you mention it is an infrastructure change, requiring retraining programmers that are already in short supply. Just because we can do something (have the hardware), doesn't mean we are going to. I believe there is little or no customer demand for parallelism, particularly for the business software users. Their software is bottlenecked at the server (which is already running multiple processors and is already parallel) and a faster or more parallel desktop isn’t going to help them at all. This is true of both LAN desktop and web based applications. How much parallel processing do you need to run IE or Firefox when the limiting performance step is your internet connection, the web server (already parallel) or the database (already parallel)? I believe most business users fall into this category.

I do agree that game companies are going to take advantage of parallelism to great effect. From that perspective, a normal end user might be interested in parallelism. There is a lot of performance gain to be had because graphics programming is largely a mathematical exercise. We haven’t heard about it in the gaming world (customers demanding parallelism) mostly because the customers would have to know what parallelism is before they could ask for it. Computer users are undereducated on this topic.

I don’t believe parallelism is stillborn. I am just not convinced that it is going to change the entire nature of programming over the next 5 -10 years. As you mention it is an infrastructure change, requiring retraining programmers that are already in short supply. Just because we can do something (have the hardware), doesn’t mean we are going to.

]]> By: Michal Migurski http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1026 Michal Migurski Sun, 21 Jan 2007 19:04:41 +0000 http://www.thinkingparallel.com/2007/01/21/parallel-programming-is-entering-the-mainstream-or-isnt-it/#comment-1026 My own experience has been an increased interest in parallel programming at the multi-CPU rather than multi-thread model. I've started to experiment in parallel, distributed methods after encountering problems that are too big to fit on one computer. I think this is a pattern among friends as well - no one seems to be feeling the horsepower pinch on individual processors, but there is definitely growing interest in big problem sets such as data mining, graphs, and so on. I continue to view threads as needlessly complex, and choose asynchronous, single-thread models such as <a href="http://twistedmatrix.com/" rel="nofollow">Twisted</a> over multi-thread ones. The interaction between machines is fascinating, though, and I find myself dealing with the kinds of problems raised by threads as I coordinate processes over network connections. <a href="http://aws.amazon.com/ec2" rel="nofollow">Amazon's EC2 service</a> has stepped in to help fill this need and support experimentation cheaply. My own experience has been an increased interest in parallel programming at the multi-CPU rather than multi-thread model. I’ve started to experiment in parallel, distributed methods after encountering problems that are too big to fit on one computer. I think this is a pattern among friends as well - no one seems to be feeling the horsepower pinch on individual processors, but there is definitely growing interest in big problem sets such as data mining, graphs, and so on. I continue to view threads as needlessly complex, and choose asynchronous, single-thread models such as Twisted over multi-thread ones. The interaction between machines is fascinating, though, and I find myself dealing with the kinds of problems raised by threads as I coordinate processes over network connections. Amazon’s EC2 service has stepped in to help fill this need and support experimentation cheaply.

]]>