[Lilug] AWS, et al, for big data

John Cholewa jcholewa at gmail.com
Tue Jun 13 18:43:12 PDT 2017


Blast it, I wish I could get to a meeting, as this would be a great
thing to chat about with a bunch of people face to face, but the best
I can do at the moment is ask on the list....

I'm managing a 200-core research cluster with a quarter petabyte of
storage capacity -- it's about a third full but may be ramping up
pretty quickly over the next couple years.  My scientists have been
wondering about throwing all our data into the cloud in order to avoid
ongoing dangers involved with storing server hardware locally and
ongoing upfront costs.  While I suspect that throwing 85+ TB onto a
VPS with massive guaranteed compute resources would be prohibitively
expensive by possibly an order of magnitude above what they're
expecting, I've been bitten by the recon bug and want to find some
hard numbers.

Are there any good resources online that might help me figure out what
the ballpark costs would be based on relatively arbitrary variable of
storage, bandwidth, cpu, and memory availability?  I plan to contact
AWS and the Google Cloud folk to get some sort of basic quote that I
can pass along, but it'd be nice if there's some guidance I can look
into before actually contacting salespersons (especially since I'm not
the one who'd be making final purchase decisions in this case).


More information about the Lilug mailing list