[Lilug] AWS, et al, for big data

[Joe Goldberg]

Both Amazon and Google have pricing calculators.  Basically you are paying for compute, storage and network.  If you have a good handle on those 3 metrics, you should be able to get a decent pricing estimate.

I am not too familiar with Google pricing but in AWS you can get nice discounts for 1-3 year commits and even more if you prepay part or all of the fee.

Also don’t discount Microsoft Azure.  They are very feature competitive with AWS and probably more advanced than Google.  They also give extreme discounts to educational and healthcare related institutions.

Now moving that amount of data to the cloud will be a challenge.  I am sure you will want to do a local replication and then ship it to them and then replicate the delta.

Joe


On 6/13/17, 9:43 PM, "Lilug on behalf of John Cholewa" <lilug-bounces at lists.lilug.org on behalf of jcholewa at gmail.com> wrote:

    Blast it, I wish I could get to a meeting, as this would be a great
    thing to chat about with a bunch of people face to face, but the best
    I can do at the moment is ask on the list....
    
    I'm managing a 200-core research cluster with a quarter petabyte of
    storage capacity -- it's about a third full but may be ramping up
    pretty quickly over the next couple years.  My scientists have been
    wondering about throwing all our data into the cloud in order to avoid
    ongoing dangers involved with storing server hardware locally and
    ongoing upfront costs.  While I suspect that throwing 85+ TB onto a
    VPS with massive guaranteed compute resources would be prohibitively
    expensive by possibly an order of magnitude above what they're
    expecting, I've been bitten by the recon bug and want to find some
    hard numbers.
    
    Are there any good resources online that might help me figure out what
    the ballpark costs would be based on relatively arbitrary variable of
    storage, bandwidth, cpu, and memory availability?  I plan to contact
    AWS and the Google Cloud folk to get some sort of basic quote that I
    can pass along, but it'd be nice if there's some guidance I can look
    into before actually contacting salespersons (especially since I'm not
    the one who'd be making final purchase decisions in this case).
    _______________________________________________
    Lilug mailing list
    Lilug at lists.lilug.org
    http://lists.lilug.org/listinfo.cgi/lilug-lilug.org