From Big Data to New Insights
Today, Microsoft is announcing the availability of a new tool called Daytona that will make it easier for researchers to harness the power of “cloud computing” to discover insights in huge quantities of data.
Daytona, which will be freely available to the research community, builds on an existing cloud computing collaboration between the National Science Foundation and Microsoft. In April, NSF announced that it was funding 13 teams to take advantage of Microsoft’s offer to provide free access to its Windows Azure cloud. Among other things, these projects will improve our understanding of large watersheds such as the Savannah River Basin, enable more and better use of renewable energy through improved weather forecasting, predict the interactions between proteins, and make cloud computing more secure, reliable, and accessible over mobile devices.
The new partnership, along with NSF collaborations with other leading IT companies, will help researchers access the computing power and storage capacity they need to tackle the big questions in their field. That’s important because researchers in a growing number of fields are generating extremely large data sets, commonly referred to as “Big Data.” For example, the size of DNA sequencing databases is increasing by a factor of 10 every 18 months! Researchers need better tools to help them store, index, search, visualize, and analyze these data, allowing them to discover new patterns and connections.
Increases in the ability to make predictions and more informed decisions on the basis of Big Data will have implications far beyond science and engineering. They have the potential to improve healthcare, increase economic productivity, personalize education, strengthen our national security, and improve the management of America’s infrastructure, including transportation and the electric grid. Given the scope and magnitude of Big Data’s potential economic, societal and scientific impact, the Office of Science and Technology Policy has asked Federal agencies to form a Senior Steering Group on this subject. Agencies are working to identify the investments that the government should be making in areas such as research, education and workforce development, Big Data-related prizes and challenges, infrastructure, and pilot projects that explore a broad range of potential applications.
Harnessing the power of Big Data will require new partnerships between government, industry, and academia—partnerships like the Microsoft-NSF collaboration. It will also require efforts to shape the national research agenda, such as the Computing Community Consortium’s papers on data analytics. I hope that more companies and research communities will join with the Administration to make the most of this opportunity.
Tom Kalil is Deputy Director for Policy at OSTP