[Lilug] AWS, et al, for big data

[Joe Goldberg]

Excellent, let me know if you need anything else.

I do cloud migrations etc. as part of my job. (

Joe


On 6/13/17, 10:44 PM, "Lilug on behalf of John Cholewa" <lilug-bounces at lists.lilug.org on behalf of jcholewa at gmail.com> wrote:

    ...scratch that, found the AWS calculator.  Turns out that my google
    searches were just a bit too complex, and I just needed to search for
    "AWS calculator"!
    
    On Tue, Jun 13, 2017 at 10:43 PM, John Cholewa <jcholewa at gmail.com> wrote:
    > I haven't seen the calculators yet.  If you have a link to any of
    > them, it'd probably help speed me up.
    >
    > And I definitely have already gone through the pain of migrating
    > dozens of terabytes from server to server.  Even on a LAN it can be an
    > arduous process!
    >
    > On Tue, Jun 13, 2017 at 10:21 PM, Joe Goldberg <joehgoldberg at gmail.com> wrote:
    >> Both Amazon and Google have pricing calculators.  Basically you are paying for compute, storage and network.  If you have a good handle on those 3 metrics, you should be able to get a decent pricing estimate.
    >>
    >> I am not too familiar with Google pricing but in AWS you can get nice discounts for 1-3 year commits and even more if you prepay part or all of the fee.
    >>
    >> Also don’t discount Microsoft Azure.  They are very feature competitive with AWS and probably more advanced than Google.  They also give extreme discounts to educational and healthcare related institutions.
    >>
    >> Now moving that amount of data to the cloud will be a challenge.  I am sure you will want to do a local replication and then ship it to them and then replicate the delta.
    >>
    >> Joe
    >>
    >>
    >> On 6/13/17, 9:43 PM, "Lilug on behalf of John Cholewa" <lilug-bounces at lists.lilug.org on behalf of jcholewa at gmail.com> wrote:
    >>
    >>     Blast it, I wish I could get to a meeting, as this would be a great
    >>     thing to chat about with a bunch of people face to face, but the best
    >>     I can do at the moment is ask on the list....
    >>
    >>     I'm managing a 200-core research cluster with a quarter petabyte of
    >>     storage capacity -- it's about a third full but may be ramping up
    >>     pretty quickly over the next couple years.  My scientists have been
    >>     wondering about throwing all our data into the cloud in order to avoid
    >>     ongoing dangers involved with storing server hardware locally and
    >>     ongoing upfront costs.  While I suspect that throwing 85+ TB onto a
    >>     VPS with massive guaranteed compute resources would be prohibitively
    >>     expensive by possibly an order of magnitude above what they're
    >>     expecting, I've been bitten by the recon bug and want to find some
    >>     hard numbers.
    >>
    >>     Are there any good resources online that might help me figure out what
    >>     the ballpark costs would be based on relatively arbitrary variable of
    >>     storage, bandwidth, cpu, and memory availability?  I plan to contact
    >>     AWS and the Google Cloud folk to get some sort of basic quote that I
    >>     can pass along, but it'd be nice if there's some guidance I can look
    >>     into before actually contacting salespersons (especially since I'm not
    >>     the one who'd be making final purchase decisions in this case).
    >>     _______________________________________________
    >>     Lilug mailing list
    >>     Lilug at lists.lilug.org
    >>     http://lists.lilug.org/listinfo.cgi/lilug-lilug.org
    >>
    >>
    >>
    >> _______________________________________________
    >> Lilug mailing list
    >> Lilug at lists.lilug.org
    >> http://lists.lilug.org/listinfo.cgi/lilug-lilug.org
    _______________________________________________
    Lilug mailing list
    Lilug at lists.lilug.org
    http://lists.lilug.org/listinfo.cgi/lilug-lilug.org