GPU/CPU Supercomputers

[kren.Z]

I need to know the computational differences between a CPU and a GPU supercomputer and the strengths and weaknesses of each. I have been asking around on this subject but have been given a number of mixed responses.

As i've taken it, the CPU supercomputer runs tasks in a serialized, sequential order whereas a GPU supercomputer can run tasks in parallel. Can someone provide a scenario where one is more suited to a given task than the other?

Can these systems run a UNIX variant and be programmed with UNIX flexibility, or am i given an interface to the hardware with an esoteric language like CUDA?

[VaeVictis]

even with any posix compliant OS such as unix/bsd/linux/osx you will need to use cuda or stream to take advantage of the GPU power for computation, consumer grade graphics cards can get performance roughly 10x faster than processors in the same class

any specifics about how it handles things im not sure about as i havent played with many GPU computation systems

i think, not entirely sure, that GPU super computers will still run an x86/arm/64 processor and just use GPU cards like the higher end ones from nvidia specifically for use in super computers (like the tesla i guess) for any heavy computation for what you are using the super computer for

i also believe the price per performance is greater for GPU systems

not the most educated on this as super computers arent that accessible to most people but i have done a little digging in the past

[kren.Z]

In a general case, I think its safe to assume:

for i in range(10):
       print i

will run n times faster on any sc.


However, my judgement becomes clouded when I look at a more specific scenario.
e.g.

I/O Device >> Data >> Analyze Data >> Draw Frame

Steps 3 and 4 both require crunching data and rendering geometry. However, wouldnt the last step be more taxing to the CPU?

Amazon offers a set of services to "rent" time on a similar cluster, but I'd really like cut out the latency for a more real-time system, plus the costs would add up quick.
Development would be much more easier on UNIX with a simple shell script redirecting output using pipes. CUDA bindings are available, just more of a hassle.





unrelated use-case -
http://www.youtube.com/watch?v=RuZQpWo9Qhs   etc...

[reaper]

You're not going to build something faster than Amazon unless you have a lot of money.  They don't have a supercomputer, they have fairly new commodity machines with various types of storage.

[peewee_RotA]

for i in range(10):
       print i

will run n times faster on any sc.

I'm not sure it's that simple. From what I've read, GPUs only show their power when being applied to large grids of data. Something like printing in sequence would just directed to the CPU anyway.

[QwazyWabbit]

The power of super computers or super GPU's is the massive parallelism. Each CPU runs the same code on arrays of similar data, as you might see in the smoke demo or in weather systems where each particle in the system interacts with the system and neighbor particles. The image space becomes the array of data and all the processors run vector code against the same data sets. The time element (video frames) becomes the 4th dimension in the array. The trick of getting performance out of parallel processing is preventing contention between the cores over access to the data store.

GPU processing is an example of Newton's "knowing all the initial conditions of the universe, you can compute the future for all time".

[quadz]

In a general case, I think its safe to assume:

for i in range(10):
       print i

will run n times faster on any sc.

Only if you want that loop executed hundreds of thousands of times in parallel.

Note the current performance leader (assuming the problem you need to solve can be parallelized like linpack ) is a CPU-only beast... but it has 705,024 cores:

http://www.theregister.co.uk/2011/11/02/fujitsu_k_super_10_petaflops/


.......GAH.... There was an article last year about research in parallelizing a particular kind of algorithm... can't find it now.  But the upshot was they were finding the best performance was to give up on the idea of avoiding "wasted" duplicate calculations on various cores... but rather to let various cores compute duplicate data if the result was to minimize the communication bottleneck with adjacent cores.....

Meh... I'll post the link if I can find it...

[kren.Z]

http://www.dwavesys.com/en/dev-tutorials.html

Snippets from the article...

[..] A quantum mechanical bit of information (qubit) can take one of two values, but the interesting thing about qubits is that they can also reside in what is known as a superposition state. You can think of this as being a situation where the qubit has not yet decided which state it wants to be in [..]

[..] Remember that with quantum computing, everything is probabilistic. It is important to start thinking in terms of interpreting statistical results over large numbers of samples rather than trying to interpret data from a single sample when you are energy programming [..]

[..]
Uncertainty is a feature
Another interesting point to note about the quantum computer is that it returns multiple answers. Some of these might be the answer that you are looking for, and some might not.
[..]


I have a lot of questions about this particular topic, but i will keep it brief.

How does a quantum machine, theoretically, solve problems faster?

[QwazyWabbit]

A QC solves only a particular class of problems faster than a classical computer. It does this by being in a multitude of states until the answer is read. Conceptually you can think of it as computing all probable answers at once, yielding the "right" answer when the computation is finished.

It solves storage problems since any number of qubits can hold a superposition of states where a classical computer needs a storage region for each unique state. In other words, a classical computer must examine each possible solution sequentially and deterministically where a QC can search the solution state probabilistically.

The Earth, after all, is a quantum computer designed to answer the Ultimate Question of Life, the Universe and Everything._TM