Sam Lightstone, Distinguished Engineer & IBM Master Inventor at IBM, sits down with Jeff Frick at Chief Data Scientist, USA in San Francisco, CA.
Data confluence: handling the scale of distributed computing | #CDSUSA
https://siliconangle.com/2016/11/23/data-confluence-handling-the-scale-of-distributed-computing-cdsusa/
We live inside an explosion of data. More information is being created now than ever before. More devices are networked than ever before. This trend is likely to continue into the future. While this makes data easy to collect for companies, it also presents the challenge of sheer scale. How does a business handle data from millions, possibly billions, of sources?
To gain some insight into the cutting edge of distributed data collection, Jeff Frick (@JeffFrick), co-host of theCUBE*, from the SiliconANGLE Media team, visited the Chief Data Scientist, USA event in San Francisco, CA. There, he met up with Sam Lightstone, distinguished engineer and chief architect for data warehousing at IBM.
The discussion opened with a look at a recently announced concept technology called “Data Confluence.” Lightstone explained that data confluence was a whole new idea they’re incubating at IBM. It came from a realization that vast amounts of data is about to come upon business from distributed sources like cellphones, cars, smartglasses and others.
“It’s really a deluge of data,” Lightstone said.
The idea behind data confluence is to leave the data where it is. Lightstone described it as allowing the data sources to find each other and collaborate on data science problems in a computational mesh.
Using the power of processors at scale
Lightstone mentioned a great advantage of this concept, being able to bring hundreds of thousands, even millions of processors to bear on data where it lives. He called this a very powerful and necessary concept. Such a network must be automatic if it is to scale for hundreds of thousands of devices.
The complexities of such a system are too much for humans to deal with. Lightstone stated his goal was to make this automatic and resilient, adapting to the state of the devices connected to it. He related that with data confluence, they hoped to tap into data science for Internet of Things, enterprise and cloud use cases.
*Disclosure: Corinium Global Intelligence and other companies sponsor some Chief Data Scientist, USA segments on SiliconANGLE Media’s theCUBE. Neither Corinium Global Intelligence nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.
Watch the complete video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of the Chief Data Scientist, USA event.
@IBM #IBM @SiliconANGLE theCUBE @theCUBE #theCUBE #CDUSA
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Chief Data Scientist, USA 2016 | San Francisco. If you don’t think you received an email check your
spam folder.
Sign in to Chief Data Scientist, USA 2016 | San Francisco.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For Chief Data Scientist, USA 2016 | San Francisco
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for Chief Data Scientist, USA 2016 | San Francisco.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
Chief Data Scientist, USA 2016 | San Francisco. If you don’t think you received an email check your
spam folder.
Sign in to Chief Data Scientist, USA 2016 | San Francisco.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to Chief Data Scientist, USA 2016 | San Francisco
Please sign in with LinkedIn to continue to Chief Data Scientist, USA 2016 | San Francisco. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Sam Lightstone, IBM | Chief Data Scientist 2016
Sam Lightstone, Distinguished Engineer & IBM Master Inventor at IBM, sits down with Jeff Frick at Chief Data Scientist, USA in San Francisco, CA.
Data confluence: handling the scale of distributed computing | #CDSUSA
https://siliconangle.com/2016/11/23/data-confluence-handling-the-scale-of-distributed-computing-cdsusa/
We live inside an explosion of data. More information is being created now than ever before. More devices are networked than ever before. This trend is likely to continue into the future. While this makes data easy to collect for companies, it also presents the challenge of sheer scale. How does a business handle data from millions, possibly billions, of sources?
To gain some insight into the cutting edge of distributed data collection, Jeff Frick (@JeffFrick), co-host of theCUBE*, from the SiliconANGLE Media team, visited the Chief Data Scientist, USA event in San Francisco, CA. There, he met up with Sam Lightstone, distinguished engineer and chief architect for data warehousing at IBM.
The discussion opened with a look at a recently announced concept technology called “Data Confluence.” Lightstone explained that data confluence was a whole new idea they’re incubating at IBM. It came from a realization that vast amounts of data is about to come upon business from distributed sources like cellphones, cars, smartglasses and others.
“It’s really a deluge of data,” Lightstone said.
The idea behind data confluence is to leave the data where it is. Lightstone described it as allowing the data sources to find each other and collaborate on data science problems in a computational mesh.
Using the power of processors at scale
Lightstone mentioned a great advantage of this concept, being able to bring hundreds of thousands, even millions of processors to bear on data where it lives. He called this a very powerful and necessary concept. Such a network must be automatic if it is to scale for hundreds of thousands of devices.
The complexities of such a system are too much for humans to deal with. Lightstone stated his goal was to make this automatic and resilient, adapting to the state of the devices connected to it. He related that with data confluence, they hoped to tap into data science for Internet of Things, enterprise and cloud use cases.
*Disclosure: Corinium Global Intelligence and other companies sponsor some Chief Data Scientist, USA segments on SiliconANGLE Media’s theCUBE. Neither Corinium Global Intelligence nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.
Watch the complete video interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of the Chief Data Scientist, USA event.
@IBM #IBM @SiliconANGLE theCUBE @theCUBE #theCUBE #CDUSA
