This 3 times faster than the previous known record.
For those who don’t know, Jim Gray et al established a series of tests, including the 1TB sort, in order to give database vendors a playground for honest comparisons. The results are maintained online. Here are the two related papers:
Google managed to sort 1TB in 68secs using their MapReduce infrastructure on 1,000 machines. Then, they attempted to sort 1PB of data on 4,000 machines. It’s interesting how when sorting 1PB of data one hits the hard disk failure rates.
Interesting stuff. I am looking forward to the paper.
In my last post, I wrote that "the typing of code was parallelized and delegated.…
In February, I wrote about the small team I'd stood up instead of hiring humans:…
Assembling a dream team without a single hire I've been making great progress on CVOYA's…
As 2025 is now behind us, I wanted to share a few reflections from my…
Few months ago, we bought a sculpture from a local art fair for our Palm…
There’s a unique energy that comes with starting something new — a blend of excitement,…