National Energy Research Scientific Computing Center (NERSC) engineer Rollin Thomas has been selected as a Project Jupyter Distinguished Contributor in appreciation of “substantial contributions to Jupyter itself in both quality and quantity over at least two years.” Announced in Paris at JupyterCon 2023 in May, the recognition is a validation of Thomas’s efforts to create bridges between the high performance computing (HPC) community and Project Jupyter, and his proactive engagement with the Project. He joins eight others honored for their work on the project this year.

Recognized by Nature as one of the “Ten computer codes that transformed science,” Project Jupyter is an open-source computational notebook that combines code, results, graphics, and text in one document, capturing and consolidating the bits and pieces of a researcher’s process and codes. Digital notebooks can replace the laborious work of keeping physical notebooks of computational or data-intensive science workflows.

Thomas came to NERSC in 2015 with a basic familiarity with Jupyter (then recently rebranded from its previous iteration, iPython). However, it was just one of several applications NERSC was deploying experimentally; figuring out how to launch Jupyter on the Cori system was one of Thomas’s first group projects at NERSC, with little inkling of how pivotal it would become. “And then it grew kind of out of control, really,” said Thomas. “It’s gone from this ‘let’s see what it does’-type project in 2016 to one of the primary ways people interact with our systems.”

The numbers agree: when Thomas began tracking the number of unique monthly NERSC users using Jupyter at the end of 2017, there were fewer than 200. By May 2023, approximately 1,300 NERSC users employed Jupyter as part of their workflows each month. To support that growth, NERSC staff recently formed a council to share the work of wrangling and improving the center’s Jupyter deployment.

The number of unique NERSC users employing Jupyter each month has grown from fewer than 200 in 2018 to over 1300 in 2023. (Credit: Rollin Thomas, Berkeley Lab)

“More and more users coming to NERSC expect to be able to use Jupyter to manage their research workflows similar to how they do it on their laptop or smaller clusters,” said Thomas. “Jupyter is already a key component of modern data science infrastructures in so many contexts. Supercomputers help people with complex data sets, so why shouldn’t you be able to use Jupyter to do data science on a supercomputer?”

Faced with the question of what to do about increasingly complex HPC systems that seem less user-friendly to the average user, Thomas and others at NERSC look to Jupyter as one way to bridge the usability gap. “How can we engage with Jupyter to take it a step further and make it a really robust platform for interactive supercomputing?” he said.

A robust community

Building this kind of strong engagement with a tool like Jupyter isn’t something that happens in isolation. To support the expansion of Jupyter for users, NERSC has joined the robust Jupyter open-source community, learning, contributing, and building relationships along the way. More broadly, Thomas noted, the program is supported by a powerful community that’s invested in considering Jupyter’s place in the present and future of scientific computing. “The community that supports it is so excited about what they’re doing with Jupyter,” he said, “and they have this mission to change how people interact with computers, and what the conversation between a human and a computer should be like.”

Thomas himself has bolstered that community by convening an interest group to allow collaboration and consultation between HPC centers that deploy Jupyter beyond NERSC. Staff from facilities supported by the U.S. Department of Energy, the National Science Foundation, and some university computer clusters in the U.S. and beyond trade information and advice for working with Jupyter. He also co-founded the Jupyter Security Subproject and is a member of Jupyter’s Community Building Committee.

Thomas sees his Project Jupyter recognition as confirmation of NERSC’s commitment to providing Jupyter to users and deploying it as a platform that offers them features on the leading edge. Further, it is an affirmation of NERSC’s experimentation-based approach to making supercomputing more productive.

“I don’t feel like this recognition is just for me,” he said. “It’s also about how NERSC does things. I was encouraged to explore what Jupyter could do at NERSC and how it could help our users, and both management and my colleagues have been supportive of those experiments. And I think that our willingness to try new things and innovate and put different pieces together to try and create new capabilities for users makes NERSC a special place to work and for our users to do science.”