Microsoft, NSF Open Cloud Computing to Scientists

Dan Reed, corporate vice president for technology strategy and policy at Microsoft, refused to put a dollar amount on the project, but said his company was willing “to invest millions of dollars in the service and that it could support thousands of scientific research programs,” according to an article in the New York Times.

Choosing which projects would be granted access to the service would be left up to the National Science Foundation. On its website, the NSF mentions three mechanisms through which researchers can request use of the Microsoft Windows Azure platform. Both new and ongoing projects will be eligible.

The agreement comes at a time when many scientific research projects face the challenges of coping with too much data. According to Dr. Jeannette M. Wing, assistant director of computer and information science and engineering directorate at the NSF, “We are generating streams and rivers of data.” Imagine, if you will, a single project that generates as much as a terabyte of data – 1,000 gigabytes – every minute. Multiply that by many projects. Genetic sequencing systems generate this kind of data, and the more we learn, the more information we collect and need to use.

It's not just the genetics research, either. With the cost of sensors coming down, scientists are using more of them. And naturally, they're generating more data that needs to be analyzed. Ed Lazowska, a University of Washington computer scientist, noted the changing research landscape by pointing out how the staffing needs of projects have changed. Once upon a time, the typical scientific research program on his campus called for one day of work a week from a half-time graduate student. Now, managing the data from that same project requires a full-time employee.

As you might expect, this isn't the first time the NSF has turned to cloud computing to help ease the burden of sifting through data. The organization has offered computing power to scientists through the following initiatives:

• Cluster Exploratory (CluE) let scientists access a set of cloud-based software services supported by Google and IBM, in addition to another cluster supported by HP, Intel and Yahoo housed at the University of Illinois at Urbana-Champaign.
  
• Teragrid, which includes a Condor pool at Purdue University.
  
• Other university-based initiatives at the San Diego Supercomputing Center and Indiana University.

Microsoft's Contribution

In addition to providing the cloud computing research projects identified by NSF with access to Windows Azure for three years, Microsoft will supply a support team to help researchers quickly integrate cloud technology into their research. Of particular interest is that Microsoft's researchers and developers will work with grant recipients to equip them with a set of common tools, applications and data collections that can be shared with the broad academic community.

The ability to share one's research easily can move science forward more quickly. Dave Patterson, professor of computer science at the University of California, Berkeley, noted that collaborating with people, especially across distances, works better if you “keep your data in the cloud rather than in your own ivory tower.” Sharing is easier in the cloud; once it's working there, there's no need to ship files, install them on other systems, and try to get them working. “If I'm getting my work done,” Patterson said, “there's no reason everybody else can't use it as well.”

Microsoft's contribution is different from other NSF cloud computing projects in that, instead of running on the open source Hadroop programming interface, it supports a Windows-based programming interface. Some users may find this easier to deal with. In any case, it lets the NSF add some diversity to their offerings. Microsoft is hoping that their custom applications will simplify access to Azure, and make it easier for scientists to use older software applications, such as Microsoft Excel, for crunching data.

That should be good news for many scientists. According to David Lifka, director of the Cornell University Center for Advanced Computing, “Cloud computing offers a great deal of promise for many types of research applications, but one important factor will be how easy it is to develop, deploy and control cloud applications from the average scientist's workstation without requiring them to have advanced system administration skills.” He notes that Microsoft's Azure platform is a “step in the right direction.”

While the current incarnation of the project is only for three years and only for US scientists, Microsoft hopes to expand the program, eventually making the new service global. Such a plan could accelerate scientific research worldwide. As Paul Watson, NE e-Science Center Director of the University of Newcastle-Upon-Tyne in the UK, explained, “Cloud computing gives scientists the computational resources they need, when they need them. It means that researchers can have a good idea on their way into work in the morning, and when they get there immediately grab the compute power and storage capacity they need to pursue it.”