My friend, cloud thought leader and all-around nice guy Dave McCrory has, over the past couple of years been espousing his view on Data Gravity and the impact it has upon the way organizations work. It’s well worth having a read of McCrory’s stuff but essentially he believes that data has an inherent gravity and that this impacts upon the way technology needs to be built – there’s no point putting mass processing resource at the other end of the globe of the mass data that needs to be processed – the natural laws of data gravity make this a sub-optimal approach.
Joyent has obviously been listening hard to McCrory’s Data Gravity thoughts since they’ve just announced Manta, a solution that essentially offers compute directly in-place on top of data storage. It’s an intriguing offering – if one takes a step away from the intricacies of data storage and compute, it’s somewhat counter-intuitive that these two functions, essentially two sides of the same coin, are seen as separate and divisible. If the amount of data that exists will continue to increase, and if the processing of that data will continue to accelerate, then surely there is a logical argument that converging the storage and the compute is a logical approach.
Clearly processing stored data in place greatly reduces the time involved in that processing, but it also has a tendency to make further processing more viable. Massive storage at lower-than-ever before pricing has made it viable for higher levels of big data analysis than ever before. In the same way removing the transport cost and delays away from big data analysis will make it economically viable to run analyses that otherwise simply wouldn’t have got over the price/benefit bar.
In terms of pricing – Manta is changed by the second – 0.004 cents per GB of DRAM per second. Joyent suggests that this is a similar price level as simple storage on AWS‘ S3 costs. Joyent is taking advantage of commodity servers chock full of storage – eking out their own efficiencies to reduce transport costs.
Manta is a pretty intriguing idea and one which, frankly, strikes me as entirely logical. The benefit of processing close to data storage, and the ability to derive insights as quickly as possible from the ever increasing amount of data that is extant is a no-brainer. I’d expect other infrastructure vendors to follow Joyent’s lead and begin to offer converged storage/compute offerings to customers.
Ben this is in fact a step to something far far bigger – mobility of (code) execution across services