Keith Beattie leads SciData’s Scientific Software Engineering Group. He recently discussed with us how research software engineers are stepping in to ensure better code, reproducible results, and lab readiness for the age of AI.
What is a research software engineer?
It’s a good question, and is now a common term for what was previously covered by literally hundreds of different job titles. In short, a research software engineer is anyone who works in research anywhere between being a traditional software developer to being a dedicated researcher working anywhere in the full stack from bare metal to user interfaces. The role combines professional computational and software engineering knowledge with technical know-how to make it all happen. Research software engineers help scientists create high-quality and effective software for their projects. They support scientists allowing them to focus their attention on discovering new ideas, not debugging their software.
How long has the term existed?
It started back in 2012 with an effort in the United Kingdom to better recognize the role and the contributions of those who support research via their computational work and expertise. Having a distinct term like RSE, makes it possible to create a community for those in the role, support their careers and promote their contributions to scientists and funders alike. There are now multiple RSE associations across the world, including here in the US, the US Research Software Engineer Association (US-RSE)
What are some examples of the ways that research software engineers have helped scientists at Berkeley Lab?
Research software engineers bring a computational perspective and software engineering approach to scientific projects, ensuring their solutions utilize best practices and tools that can evolve beyond individual projects. For example: version control, issue tracking, continuous integration testing, release management, are just a few examples of software development practices that might not be on the forefront of a researcher’s mind when starting a new project. We assemble these tools and practices to achieve large scale engineering and data analysis for scientific work.
Because RSEs have interdisciplinary experience, they can communicate knowledgeably across different scientific domains—from physics to biology to astronomy—to develop unique solutions. We use our expertise to then deploy solutions that take advantage of tools and practices already in existence. The longer an RSE is embedded in a project, the more their expertise can deepen and evolve.
I lead the Scientific Software Engineering (SSE) Group, which, along with other groups in the Scientific Data Division, serves as a hub for the broader research community at the lab. We provide state-of-the-art software development and computational solutions supporting all aspects of publicly funded science. Every project begins with a scientific collaboration, whether it spans the Scientific Data Division, Berkeley Lab, or the entire DOE complex.
We’ve helped physicists search for dark matter particles and hydrologists build stronger water filtration systems. For example, on the Materials Project, SSE staff have partnered closely with the project’s infrastructure team to prototype new features and significantly enhance the software architecture and quality of the platform. We are also advancing quantum research through collaborations like the QUANT-NET project, developing frameworks that provide precise control over experimental setups.
While the “Research Software Engineering” role has now gained recognition, similar functions have existed within the lab for some time. Researchers and engineers across all the science divisions at the lab have taken on part or all of the RSE role in order to support their scientific missions. My belief is that increased support for RSEs will have follow on effects enhancing the support they provide to researchers across the lab. Ultimately, it’s exciting that research software engineers are finding each other and building a sense of community.
Speaking of community, can you tell us more about US-RSE and what your role is in this organization?
The US Research Software Engineer Association (US-RSE) is a community-driven organization dedicated to elevating the role of the research software engineer all across the US. We strive to recognize the unique value of this expertise and foster a strong sense of community across the entire research ecosystem, including universities, national laboratories, and industry.
Since our founding, US-RSE has experienced remarkable growth, expanding from under 50 members in 2019 to over 4,000 today. As an all-volunteer organization, we serve as a vital forum, connecting and encouraging RSEs through newsletters and Slack channels to share best practices and drive industry knowledge. Ultimately, we want to empower RSEs by recognizing their unique contributions and helping them envision successful career paths in this field.
Although the majority of members are from academia or industry, national laboratories play an outsized role in working groups and organizing functions. These groups facilitate technical discussions—covering topics like code review, testing research software, and training—while also connecting members for career path discussions and mentorship from seasoned practitioners.
I’m on the steering committee for this organization and am also co-leading our annual conference, which will be held this year from October 19–21, very close to us in San Jose. The conference’s theme is “Advancing Science in the Age of AI.” There will be 72 technical talks on a range of scientific computing topics (and not just about AI), including many from LBNL staff, eight social community building and skill building sessions between workshops,birds of a feather (BoF) activities, and daily plenaries.
For more information about the US-RSE conference, please visit the US-RSE conference website.
Learn more about the Scientific Software Engineering Group’s work at Berkeley Lab on their SciData webpage.
About Computing Sciences at Berkeley Lab
High performance computing plays a critical role in scientific discovery. Researchers increasingly rely on advances in computer science, mathematics, computational science, data science, and large-scale computing and networking to increase our understanding of ourselves, our planet, and our universe. Berkeley Lab's Computing Sciences Area researches, develops, and deploys new foundations, tools, and technologies to meet these needs and to advance research across a broad range of scientific disciplines.
