Boverhof’s App Earns Honorable Mention in Amazon’s Web Services Competition
January 12, 2013
Jon Bashor, Jbashor@lbl.gov, +1 510 486 5849
The Turbine Gateway, an application developed by the Computational Research Division’s Joshua Boverhof for DOE’s Carbon Capture Simulation Initiative, recently earned honorable mention in a competition sponsored by Amazon Web Services (AWS). Amazon officially announced the winners of its EC2 Spotathon on Monday, Dec. 10.
“We received many impressive applications and there was fierce competition at the top; even though we ultimately didn’t select your entry for the Grand Prize or Runner-Up, given the caliber of your Spot application, we will be recognizing the Turbine Science Gateway as one of the honorable mentions on our web page and in an upcoming blog post,” Amazon Web Services Senior Product Manager Stephen A. Elliott wrote to Boverhof. “Congratulations on the honorable mention and, more importantly, for developing such a sophisticated Spot application!”
Boverhof learned of his award at an AWS Reinvent developers conference a few weeks ago and really didn’t expect to win any recognition, but was quite happy with the result. He showed that a single experiment that ran for 11 hours saved more than $300 by bidding on unused compute resources on the spot market versus paying for on-demand instances.
Amazon set up the Spotathon “to publicly recognize some of the most exciting applications of EC2 (Elastic Compute Cloud) Spot Instances, assessing each submission based on the cost savings, performance benefits, and compute scale achieved on Spot, as well as the overall elegance of the solution.” EC2 provides scalable, pay-as-you-go compute capacity in the cloud.
Spot Instances are excess EC2 instances whose price is based on real-time supply and demand. When users request an EC2 Instance via a Spot request, they specify a bid price; as long as the bid exceeds the Spot price, the user is provisioned a Spot Instance. When the Spot price exceeds the bid, the Spot Instance is interrupted.
In its announcement, Amazon stated “The Turbine Science Gateway (TSG), developed by Joshua Boverhof, supports the Department of Energy’s Carbon Capture Simulation Initiative (CCSI) by providing a web application and execution environment for running and managing scientific applications (like AspenTech’s AspenPlus) and storing and archiving results. Utilizing TSG, simulation runs that would take months on a single machine can be done overnight on EC2, running tens of thousands of simulations on hundreds of Spot instances and saving over 70 percent on EC2 compute costs.”
This past June AWS announced the integration of Spot instances with their auto scaling capabilities. Boverhof integrated that into TSG so the application could automatically take advantage of additional resources as they became available. By the time he submitted his entry, Boverhof and the CCSI team had run more than half a million simulations, giving him plenty of data on which to build his case.
When submitting his entry, Boverhof described a recent experiment of a hybrid circulating fluidized bed process model, in which the CCSI project’s Uncertainty Quantification (UQ) team of scientists and engineers at Lawrence Livermore National Lab ran 50,000 simulations utilizing 240 spot instances. The experiment ran overnight, and took a total of 11 hours to run. Boverhof states in his submission, “While the experiment was running with 240 spot instances, we were paying between $7.84 and $12.78/hour for EC2 instances. Running the experiment on-demand would have cost $41.94/hour for EC2 instances. Total saving on just this experiment was over $300.”
The TSG is a web application and execution environment for running and managing scientific applications and storing and archiving results. It’s part of an integration framework, developed to support application scientists in the Carbon Capture Simulation Initiative (CCSI), a partnership among DOE laboratories, industry and academic institutions. The project aims to provide the utilities industry with advanced tools to further leverage computational modeling and simulation tools to accelerate the use of carbon capture technologies from discovery to development, and ultimately widespread deployment to hundreds of power plants.
About Computing Sciences at Berkeley Lab
The Lawrence Berkeley National Laboratory (Berkeley Lab) Computing Sciences organization provides the computing and networking resources and expertise critical to advancing the Department of Energy's research missions: developing new energy sources, improving energy efficiency, developing new materials and increasing our understanding of ourselves, our world and our universe.
ESnet, the Energy Sciences Network, provides the high-bandwidth, reliable connections that link scientists at 40 DOE research sites to each other and to experimental facilities and supercomputing centers around the country. The National Energy Research Scientific Computing Center (NERSC) powers the discoveries of 7,000-plus scientists at national laboratories and universities, including those at Berkeley Lab's Computational Research Division (CRD). CRD conducts research and development in mathematical modeling and simulation, algorithm design, data storage, management and analysis, computer system architecture and high-performance software implementation. NERSC and ESnet are Department of Energy Office of Science User Facilities.
Lawrence Berkeley National Laboratory addresses the world's most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. Founded in 1931, Berkeley Lab's scientific expertise has been recognized with 13 Nobel prizes. The University of California manages Berkeley Lab for the DOE’s Office of Science.
DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.