The Komprise team is at the SuperComputing Conference in Dallas this week (come and see our team at booth 4413 if you happen to be attending).
I thought I would take a moment to share my thoughts on the challenges I hear from our HPC customers.
Our HPC customers are generating data at a rapid pace. I’d like to share a top medical research center customer based out of New York as an example. This particular research organization has, through its various research projects and studies, accumulated petabytes of data over the years. This data is immensely valuable, even lifesaving, and must be retained and kept accessible—either due to the value of the data for future research or for compliance reasons. The challenge they faced is that while their data footprint was growing by 45% YOY the budget they had to manage that growing footprint was only growing 2.5%.
The pressure that IT is put under to stretch a relatively flat budget is a common theme that I have noticed in my conversations with our users. As a result of the scale of data being produced today in HPC environments, I believe it is no longer feasible for organizations to continue to keep all their data on Tier One NAS solutions and it comes as no surprise to me that Komprise is helping more and more HPC organizations adopt a tiered approach to keep costs under control while still keeping data accessible to their users. In the case of the medical research center, our analysis showed 76% of data had not been accessed or modified in over a year. For them, this meant that they could cut more than 70% of their storage costs by moving this data to IBM tape which the organization already owned but had not fully utilized. Because the data was moved via our Transparent Move Technology TM, it is still accessible to the organization as if it was still on the primary filer from the original namespace with no change to user access. And users can search and access all the metadata locally for data that is now on tape(albeit with the latency upon full retrieval that is inherent to tape).
After talking with our HPC customers over the last months one of the more requested features was support for the Lustre protocol. I am happy to share that we announced our Lustre Beta at SuperComputing this week. If you are interested in signing up for the Lustre Beta, send an email to lustrebeta@komprise.com
