Popular Science has an entertaining plot of rising computing power which is really good from the 1930s thru 1990 or so, but at that point misses the point entirely. It plots current day desktop computing at 500 billion (5 x 10^11) operations/sec for $2000 (about right for a 3 GHz 8-core superscalar desktop) and then at 8.5 x 10^15 ops/sec the Japanese K Computer for a cool $1.25 bn. What it omits is the ease of networking tens of thousands of desktops together in datacenters to form distributed computing systems such as Amazon's EC2 cloud or Google's Skynet (yes, they don't call it that officially but I think we all know where this is headed).
If I put together 20k desktops I'd get a raw 1 x 10^16 ops/sec for a mere $60mm (adding 50% for networking gear and datacenter-level power, cooling which seems about right). You'll note that this is more computing power than the K Computer for 5% of the cost. That is the real triumph of modern day computing - the software and networking to harness efficiently the computing potential of thousands of commodity hardware PCs. The challenge now is to shape problems in a way that such machines can help us solve them.
[For readers expecting another Greek-related tirade, my apologies - normal service will resume shortly.]