An increasing number of companies are watching their central databases grow through the multi-petabytes towards the exabyte category writes Wikibon Analyst John McArthur in "Hyperscale Storage: Not if, When." The Wikibon alert headlines the latest Peer Incite newsletter triggered by the January 22 Wikibon Peer Incite meeting on hyperscaling with Cleversafe VP Russ Kennedy. Databases at this scale have been the province of large government agencies and dominant cloud companies like Facebook, YouTube, and Shutterfly. However, McArthur predicts, increasing numbers of more traditional companies are going to see their data grow to the multi-petabyte range as video, still images, and audio recording become more common parts of business communications.
Databases of this size make traditional RAID and replication data protection strategies impractical. It is just too expensive to maintain multiple copies of a multi-petabyte database. Traditional customized hardware becomes impractically expensive, and with huge numbers of disk spinning, failures become increasingly common. This reality has driven Facebook and other online companies to develop a new IT infrastructure, including storage through the Open Compute Project, he writes.
As a result, writes Wikibon Analyst Scott Lowe, "CIOS must watch hyperscale trends and jump when ready." "CIOs must maintain and open mind as times change and new opportunities and offerings make their way to market." Lessons from the hyperscale experience can help CIOs improve their services, decrease costs, increase availability and increase capacity and capability as demands grow, even if those demands do not reach the petabyte range. "In 1999 who would have thought that we could run 100 servers worth of workload on just five physical hosts with room to spare?"
Hyperscale databases demand a new approach to data protection, adds Wikibon CTO David Floyer in Software-led Storage: Hyperscale Storage Requirements. "The only technology available to address these issues is erasure coding, the ability to use compute cycles to split up and transform the data into n slices with the ability to restore it with only m slices." These slices can be distributed geographically or locally. Metadata becomes vital in this approach as it is the key to finding the slices and rebuilding the file correctly.
Cleversafe's information dispersal system uses a REST protocol-based approach, explains Wikibon Analyst Gary McFadden. In such systems, object-based storage has several advantages. However, erasure coding and information dispersal comes with a performance tax. It takes time to reassemble an object. Under ideal conditions this means a millisecond-scale response time, rather than the microsecond response required by high-performance, transactional systems. This means that vendors need to determine the vertical markets in which they wish to compete and examine the requirements of specific applications in these markets before deciding on the architecture on which they will build their systems. And users need to be conscious of the trade-offs when matching an application to a technology.
So what does the the hyperscale architecture eliminate? For a start, maintenance contracts and repair expenses and detailed vendor management. But one of the biggest savings, writes Wikibon Analyst John McArthur, is "Eliminating Backup and Replication in Hyper-Scale Storage." This is absolutely necessary. Companies simply cannot afford to maintain multiple copies of multi-petabyte databases.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Wikibon Whiteboards. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For Wikibon Whiteboards
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Wikibon Whiteboards.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Wikibon Whiteboards. If you don’t think you received an email check your
spam folder.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to Wikibon Whiteboards
Please sign in with LinkedIn to continue to Wikibon Whiteboards. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Hyperscale Invades the Enterprise and the Impact on Converged Infrastructure
An increasing number of companies are watching their central databases grow through the multi-petabytes towards the exabyte category writes Wikibon Analyst John McArthur in "Hyperscale Storage: Not if, When." The Wikibon alert headlines the latest Peer Incite newsletter triggered by the January 22 Wikibon Peer Incite meeting on hyperscaling with Cleversafe VP Russ Kennedy. Databases at this scale have been the province of large government agencies and dominant cloud companies like Facebook, YouTube, and Shutterfly. However, McArthur predicts, increasing numbers of more traditional companies are going to see their data grow to the multi-petabyte range as video, still images, and audio recording become more common parts of business communications.
Databases of this size make traditional RAID and replication data protection strategies impractical. It is just too expensive to maintain multiple copies of a multi-petabyte database. Traditional customized hardware becomes impractically expensive, and with huge numbers of disk spinning, failures become increasingly common. This reality has driven Facebook and other online companies to develop a new IT infrastructure, including storage through the Open Compute Project, he writes.
As a result, writes Wikibon Analyst Scott Lowe, "CIOS must watch hyperscale trends and jump when ready." "CIOs must maintain and open mind as times change and new opportunities and offerings make their way to market." Lessons from the hyperscale experience can help CIOs improve their services, decrease costs, increase availability and increase capacity and capability as demands grow, even if those demands do not reach the petabyte range. "In 1999 who would have thought that we could run 100 servers worth of workload on just five physical hosts with room to spare?"
Hyperscale databases demand a new approach to data protection, adds Wikibon CTO David Floyer in Software-led Storage: Hyperscale Storage Requirements. "The only technology available to address these issues is erasure coding, the ability to use compute cycles to split up and transform the data into n slices with the ability to restore it with only m slices." These slices can be distributed geographically or locally. Metadata becomes vital in this approach as it is the key to finding the slices and rebuilding the file correctly.
Cleversafe's information dispersal system uses a REST protocol-based approach, explains Wikibon Analyst Gary McFadden. In such systems, object-based storage has several advantages. However, erasure coding and information dispersal comes with a performance tax. It takes time to reassemble an object. Under ideal conditions this means a millisecond-scale response time, rather than the microsecond response required by high-performance, transactional systems. This means that vendors need to determine the vertical markets in which they wish to compete and examine the requirements of specific applications in these markets before deciding on the architecture on which they will build their systems. And users need to be conscious of the trade-offs when matching an application to a technology.
So what does the the hyperscale architecture eliminate? For a start, maintenance contracts and repair expenses and detailed vendor management. But one of the biggest savings, writes Wikibon Analyst John McArthur, is "Eliminating Backup and Replication in Hyper-Scale Storage." This is absolutely necessary. Companies simply cannot afford to maintain multiple copies of multi-petabyte databases.
