CRD’s PyGlobus Tools Proving Popular
January 1, 2006
CRD’s Distributed Systems Department (DSD), which has led the development of the de facto standard tools for developing Grid Services, applications and portals using the Python programming language, proved a popular draw at the LBNL booth at the SC05 conference in Seattle.
Python is a high-level interpreted language that supports a rapid application development cycle. Python’s minimal syntax makes it an ideal language for use by non-computer scientists. It also easily supports binding together C/C++ and Fortran codes and exposing them through a thin Python interface. By enabling scientists to focus less on computer science details, our Python Grid tools allow scientists to focus on their science.
Keith Jackson led DSD’s development of the Python Commodity Grid (CoG) Kit, or pyGlobus, to provide access to the original Globus Toolkit developed at Argonne National Lab and USC’s Information Sciences Institute. The Python CoG Kit has been an important part of Grid development for a number of projects, including the DOE- funded Access Grid project and the NSF- funded Laser Interferometer Gravitational Wave Observatory (LIGO).
The Python CoG Kit provides a thin veneer of Python over the underlying Globus Toolkit C code. pyGlobus provides a simple, easy to use, object-oriented interface to Globus while still providing the full power and performance of the underlying C code.
The department also developed a Python implementation, called pyGridWare, of the next generation of Grid standards based on Web services, i.e., the Web Service Resource Framework (WSRF) and Web Service Notification (WS-N) specifications. In addition to building lower-level toolkits, DSD has also developed a Visual Composition Environment, or ViCE (see sidebar) to support the collaborative development and exe- cution of complex scientific workflows.
The pyGridWare Toolkit provides a vital set of tools for current Python Grid projects that are transitioning to the new Web service- based Grid architecture. The most recent Grid standards, WSRF and WS-N, are based on industry-standard Web services. A Web service is simply any service that describes its interface in a standard format based on XML and is accessible via standard Web protocols. The use of standard protocols enables the scientific world to leverage the significant corporate investment in Web service infrastructure, and allows multiple interoperable implementations to be developed. pyGridWare is interoperable with the Java and C implementations from Argonne and is mostly compatible with the .Net implementation from the University of Virginia.
In addition to support for developing WSRF applications from scratch, DSD has developed a tool to automatically wrap an existing command-line application to expose it as a WSRF service. This allows a scientist to take an existing application that is run locally and expose it as a Grid service that is accessible over the network.
By allowing existing applications to be easily converted into Grid services, the goal is to leverage the significant investment DOE has made in high performance codes, such as those developed under the SciDAC Integrated Software Infrastructure Centers, while still exposing these applica- tions as Grid services as part of the emerging national middleware infrastructure.
About Computing Sciences at Berkeley Lab
The Lawrence Berkeley National Laboratory (Berkeley Lab) Computing Sciences organization provides the computing and networking resources and expertise critical to advancing the Department of Energy's research missions: developing new energy sources, improving energy efficiency, developing new materials and increasing our understanding of ourselves, our world and our universe.
ESnet, the Energy Sciences Network, provides the high-bandwidth, reliable connections that link scientists at 40 DOE research sites to each other and to experimental facilities and supercomputing centers around the country. The National Energy Research Scientific Computing Center (NERSC) powers the discoveries of 6,000 scientists at national laboratories and universities, including those at Berkeley Lab's Computational Research Division (CRD). CRD conducts research and development in mathematical modeling and simulation, algorithm design, data storage, management and analysis, computer system architecture and high-performance software implementation. NERSC and ESnet are DOE Office of Science User Facilities.
Lawrence Berkeley National Laboratory addresses the world's most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. Founded in 1931, Berkeley Lab's scientific expertise has been recognized with 13 Nobel prizes. The University of California manages Berkeley Lab for the DOE’s Office of Science.
DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.