[Lilug] AWS, et al, for big data

John Cholewa jcholewa at gmail.com
Tue Jun 13 19:43:41 PDT 2017


I haven't seen the calculators yet.  If you have a link to any of
them, it'd probably help speed me up.

And I definitely have already gone through the pain of migrating
dozens of terabytes from server to server.  Even on a LAN it can be an
arduous process!

On Tue, Jun 13, 2017 at 10:21 PM, Joe Goldberg <joehgoldberg at gmail.com> wrote:
> Both Amazon and Google have pricing calculators.  Basically you are paying for compute, storage and network.  If you have a good handle on those 3 metrics, you should be able to get a decent pricing estimate.
>
> I am not too familiar with Google pricing but in AWS you can get nice discounts for 1-3 year commits and even more if you prepay part or all of the fee.
>
> Also don’t discount Microsoft Azure.  They are very feature competitive with AWS and probably more advanced than Google.  They also give extreme discounts to educational and healthcare related institutions.
>
> Now moving that amount of data to the cloud will be a challenge.  I am sure you will want to do a local replication and then ship it to them and then replicate the delta.
>
> Joe
>
>
> On 6/13/17, 9:43 PM, "Lilug on behalf of John Cholewa" <lilug-bounces at lists.lilug.org on behalf of jcholewa at gmail.com> wrote:
>
>     Blast it, I wish I could get to a meeting, as this would be a great
>     thing to chat about with a bunch of people face to face, but the best
>     I can do at the moment is ask on the list....
>
>     I'm managing a 200-core research cluster with a quarter petabyte of
>     storage capacity -- it's about a third full but may be ramping up
>     pretty quickly over the next couple years.  My scientists have been
>     wondering about throwing all our data into the cloud in order to avoid
>     ongoing dangers involved with storing server hardware locally and
>     ongoing upfront costs.  While I suspect that throwing 85+ TB onto a
>     VPS with massive guaranteed compute resources would be prohibitively
>     expensive by possibly an order of magnitude above what they're
>     expecting, I've been bitten by the recon bug and want to find some
>     hard numbers.
>
>     Are there any good resources online that might help me figure out what
>     the ballpark costs would be based on relatively arbitrary variable of
>     storage, bandwidth, cpu, and memory availability?  I plan to contact
>     AWS and the Google Cloud folk to get some sort of basic quote that I
>     can pass along, but it'd be nice if there's some guidance I can look
>     into before actually contacting salespersons (especially since I'm not
>     the one who'd be making final purchase decisions in this case).
>     _______________________________________________
>     Lilug mailing list
>     Lilug at lists.lilug.org
>     http://lists.lilug.org/listinfo.cgi/lilug-lilug.org
>
>
>
> _______________________________________________
> Lilug mailing list
> Lilug at lists.lilug.org
> http://lists.lilug.org/listinfo.cgi/lilug-lilug.org


More information about the Lilug mailing list