[Lilug] AWS, et al, for big data

[John Cholewa]

...scratch that, found the AWS calculator.  Turns out that my google
searches were just a bit too complex, and I just needed to search for
"AWS calculator"!

On Tue, Jun 13, 2017 at 10:43 PM, John Cholewa <jcholewa at gmail.com> wrote:
> I haven't seen the calculators yet.  If you have a link to any of
> them, it'd probably help speed me up.
>
> And I definitely have already gone through the pain of migrating
> dozens of terabytes from server to server.  Even on a LAN it can be an
> arduous process!
>
> On Tue, Jun 13, 2017 at 10:21 PM, Joe Goldberg <joehgoldberg at gmail.com> wrote:
>> Both Amazon and Google have pricing calculators.  Basically you are paying for compute, storage and network.  If you have a good handle on those 3 metrics, you should be able to get a decent pricing estimate.
>>
>> I am not too familiar with Google pricing but in AWS you can get nice discounts for 1-3 year commits and even more if you prepay part or all of the fee.
>>
>> Also don’t discount Microsoft Azure.  They are very feature competitive with AWS and probably more advanced than Google.  They also give extreme discounts to educational and healthcare related institutions.
>>
>> Now moving that amount of data to the cloud will be a challenge.  I am sure you will want to do a local replication and then ship it to them and then replicate the delta.
>>
>> Joe
>>
>>
>> On 6/13/17, 9:43 PM, "Lilug on behalf of John Cholewa" <lilug-bounces at lists.lilug.org on behalf of jcholewa at gmail.com> wrote:
>>
>>     Blast it, I wish I could get to a meeting, as this would be a great
>>     thing to chat about with a bunch of people face to face, but the best
>>     I can do at the moment is ask on the list....
>>
>>     I'm managing a 200-core research cluster with a quarter petabyte of
>>     storage capacity -- it's about a third full but may be ramping up
>>     pretty quickly over the next couple years.  My scientists have been
>>     wondering about throwing all our data into the cloud in order to avoid
>>     ongoing dangers involved with storing server hardware locally and
>>     ongoing upfront costs.  While I suspect that throwing 85+ TB onto a
>>     VPS with massive guaranteed compute resources would be prohibitively
>>     expensive by possibly an order of magnitude above what they're
>>     expecting, I've been bitten by the recon bug and want to find some
>>     hard numbers.
>>
>>     Are there any good resources online that might help me figure out what
>>     the ballpark costs would be based on relatively arbitrary variable of
>>     storage, bandwidth, cpu, and memory availability?  I plan to contact
>>     AWS and the Google Cloud folk to get some sort of basic quote that I
>>     can pass along, but it'd be nice if there's some guidance I can look
>>     into before actually contacting salespersons (especially since I'm not
>>     the one who'd be making final purchase decisions in this case).
>>     _______________________________________________
>>     Lilug mailing list
>>     Lilug at lists.lilug.org
>>     http://lists.lilug.org/listinfo.cgi/lilug-lilug.org
>>
>>
>>
>> _______________________________________________
>> Lilug mailing list
>> Lilug at lists.lilug.org
>> http://lists.lilug.org/listinfo.cgi/lilug-lilug.org