[Lilug] query: handling exponentially increasing storage requirements on a rack

[Cholewa, John]

So I ordered a 36-drive, >250 TB nfs storage server for my lab. I would 
have preferred avoiding needing to deal with drives that I'd normally 
consider too new for use in this environment (they're 8 TB each), but 
I've been limited by our heavy requirements.  The vendor is having 
difficulty in getting this large amount of space to behave as a single 
xfs mount point.  I may have to accept hacks along the line of splitting 
up the array to multiple volumes.

Some of the people on this list are at much larger labs, wherein pithily 
small storage amounts like a mere quarter petabyte are easily laughed 
off as chumpspace.  While I can't change what I'm getting here this time 
around, I wouldn't mind hearing what kind of solutions are used to get 
large amounts of storage to look like single mounts for end-user 
researchers and the like.  Do you cluster your storage among multiple 
computers and transparently make them look like a single server?  If so, 
what are some good practice routes towards achieving this?  I know that 
there are alternate means of doing storage over a network, so right now 
I'm definitely reaching the limits of my old-school way of thinking and 
wouldn't mind some pokes in the right direction.  :)


The information contained in this electronic e-mail transmission and any attachments are intended only for the use of the individual or entity to whom or to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this communication is not the intended recipient, or the employee or agent responsible for delivering this communication to the intended recipient, you are hereby notified that any dissemination, distribution, copying or disclosure of this communication and any attachment is strictly prohibited. If you have received this transmission in error, please notify the sender immediately by telephone and electronic mail, and delete the original communication and any attachment from any computer, server or other electronic recording or storage device or medium. Receipt by anyone other than the intended recipient is not a waiver of any attorney-client, physician-patient or other privilege.