Build with our next generation AI systems

Explore models chevron_right

Gemini

Our most intelligent AI models

2.5 Pro
2.5 Flash
2.0 Flash-Lite
Learn more

Gemma

Lightweight, state-of-the-art open models

Gemma 3
Gemma 3n
ShieldGemma 2
Learn more

Generative models

Image, music and video generation models

Imagen
Lyria
Veo

Experiments

AI prototypes and experiments

Project Astra
Project Mariner
Gemini Diffusion

Our latest AI breakthroughs and updates from the lab

Explore research chevron_right

Projects

Explore some of the biggest AI innovations

Learn more

Publications

Read a selection of our recent papers

Learn more

News

Discover the latest updates from our lab

Learn more

Unlocking a new era of discovery with AI

Explore science chevron_right

AI for biology

AlphaFold
AlphaMissense
AlphaProteo

AI for climate and sustainability

WeatherNext

AI for mathematics and computer science

AlphaEvolve
AlphaProof
AlphaGeometry

AI for physics and chemistry

GNoME
Fusion
AlphaQubit

AI transparency

SynthID

Our mission is to build AI responsibly to benefit humanity

About Google DeepMind chevron_right

News

Discover our latest AI breakthroughs, projects, and updates

Learn more

Careers

We’re looking for people who want to make a real, positive impact on the world

Learn more

Milestones

For over 20 years, Google has worked to make AI helpful for everyone

Learn more

Education

We work to make AI more accessible to the next generation

Learn more

Responsibility

Ensuring AI safety through proactive security, even against evolving threats

Learn more

The Podcast

Uncover the extraordinary ways AI is transforming our world

Learn more
Models
Research
Science
About

Build with our next generation AI systems

Explore models chevron_right

Gemini

Our most intelligent AI models

2.5 Pro
2.5 Flash
2.0 Flash-Lite
Learn more

Gemma

Lightweight, state-of-the-art open models

Gemma 3
Gemma 3n
ShieldGemma 2
Learn more

Generative models

Image, music and video generation models

Imagen
Lyria
Veo

Experiments

AI prototypes and experiments

Project Astra
Project Mariner
Gemini Diffusion

Our latest AI breakthroughs and updates from the lab

Explore research chevron_right

Projects

Explore some of the biggest AI innovations

Learn more

Publications

Read a selection of our recent papers

Learn more

News

Discover the latest updates from our lab

Learn more

Unlocking a new era of discovery with AI

Explore science chevron_right

AI for biology

AlphaFold
AlphaMissense
AlphaProteo

AI for climate and sustainability

WeatherNext

AI for mathematics and computer science

AlphaEvolve
AlphaProof
AlphaGeometry

AI for physics and chemistry

GNoME
Fusion
AlphaQubit

AI transparency

SynthID

Our mission is to build AI responsibly to benefit humanity

About Google DeepMind chevron_right

News

Discover our latest AI breakthroughs, projects, and updates

Learn more

Careers

We’re looking for people who want to make a real, positive impact on the world

Learn more

Milestones

For over 20 years, Google has worked to make AI helpful for everyone

Learn more

Education

We work to make AI more accessible to the next generation

Learn more

Responsibility

Ensuring AI safety through proactive security, even against evolving threats

Learn more

The Podcast

Uncover the extraordinary ways AI is transforming our world

Learn more
Models

Build with our next generation AI systems

Explore models chevron_right

Gemini

Our most intelligent AI models

2.5 Pro
2.5 Flash
2.0 Flash-Lite
Learn more

Gemma

Lightweight, state-of-the-art open models

Gemma 3
Gemma 3n
ShieldGemma 2
Learn more

Generative models

Image, music and video generation models

Imagen
Lyria
Veo

Experiments

AI prototypes and experiments

Project Astra
Project Mariner
Gemini Diffusion
Research

Our latest AI breakthroughs and updates from the lab

Explore research chevron_right

Projects

Explore some of the biggest AI innovations

Learn more

Publications

Read a selection of our recent papers

Learn more

News

Discover the latest updates from our lab

Learn more
Science

Unlocking a new era of discovery with AI

Explore science chevron_right

AI for biology

AlphaFold
AlphaMissense
AlphaProteo

AI for climate and sustainability

WeatherNext

AI for mathematics and computer science

AlphaEvolve
AlphaProof
AlphaGeometry

AI for physics and chemistry

GNoME
Fusion
AlphaQubit

AI transparency

SynthID
About

Our mission is to build AI responsibly to benefit humanity

About Google DeepMind chevron_right

News

Discover our latest AI breakthroughs, projects, and updates

Learn more

Careers

We’re looking for people who want to make a real, positive impact on the world

Learn more

Milestones

For over 20 years, Google has worked to make AI helpful for everyone

Learn more

Education

We work to make AI more accessible to the next generation

Learn more

Responsibility

Ensuring AI safety through proactive security, even against evolving threats

Learn more

The Podcast

Uncover the extraordinary ways AI is transforming our world

Learn more
Try Google AI Studio Try Gemini
Google DeepMind
Google AI Learn about all of our AI Google DeepMind Explore the frontier of AI Google Labs Try our AI experiments Google Research Explore our research
Gemini app Chat with Gemini Google AI Studio Build with our next-gen AI models
Models Research Science About
Try Google AI Studio Try Gemini

Science

Accelerating fusion science through learned plasma control

Published
16 February 2022
Authors

Pulsar Team, Swiss Plasma Center* (* External authors )

Share

  • ×

Note:This blog was first published on 16 Feb 2022. Following the release of TORAX plasma simulator code in May 2024, we’ve made minor updates to the text to reflect this.

Successfully controlling the nuclear fusion plasma in a tokamak with deep reinforcement learning

To solve the global energy crisis, researchers have long sought a source of clean, limitless energy. Nuclear fusion, the reaction that powers the stars of the universe, is one contender. By smashing and fusing hydrogen, a common element of seawater, the powerful process releases huge amounts of energy. Here on earth, one way scientists have recreated these extreme conditions is by using a tokamak, a doughnut-shaped vacuum surrounded by magnetic coils, that is used to contain a plasma of hydrogen that is hotter than the core of the Sun. However, the plasmas in these machines are inherently unstable, making sustaining the process required for nuclear fusion a complex challenge. For example, a control system needs to coordinate the tokamak's many magnetic coils and adjust the voltage on them thousands of times per second to ensure the plasma never touches the walls of the vessel, which would result in heat loss and possibly damage. To help solve this problem and as part of DeepMind’s mission to advance science, we collaborated with the Swiss Plasma Center at EPFL to develop the first deep reinforcement learning (RL) system to autonomously discover how to control these coils and successfully contain the plasma in a tokamak, opening new avenues to advance nuclear fusion research.

In a paper published today in Nature, we describe how we can successfully control nuclear fusion plasma by building and running controllers on the Variable Configuration Tokamak (TCV) in Lausanne, Switzerland. Using a learning architecture that combines deep RL and a simulated environment, we produced controllers that can both keep the plasma steady and be used to accurately sculpt it into different shapes. This “plasma sculpting” shows the RL system has successfully controlled the superheated matter and - importantly - allows scientists to investigate how the plasma reacts under different conditions, improving our understanding of fusion reactors.

“

In the last two years DeepMind has demonstrated AI’s potential to accelerate scientific progress and unlock entirely new avenues of research across biology, chemistry, mathematics and now physics.

Demis Hassabis, Co-founder and CEO, DeepMind

This work is another powerful example of how machine learning and expert communities can come together to tackle grand challenges and accelerate scientific discovery. Our team is hard at work applying this approach to fields as diverse as quantum chemistry, pure mathematics, material design, weather forecasting, and more, to solve fundamental problems and ensure AI benefits humanity.

Photos of the Variable Configuration Tokamak (TCV) at EPFL seen from outside (left, credit: SPC/EPFL) and inside (right, credit: Alain Herzog / EPFL) and a 3D model of TCV with vessel and control coils (centre, credit: DeepMind and SPC/EPFL)

Learning when data is hard to acquire

Research into nuclear fusion is currently limited by researchers’ ability to run experiments. While there are dozens of active tokamaks around the world, they’re expensive machines and in high demand. For example, TCV can only sustain the plasma in a single experiment for up to three seconds, after which it needs 15 minutes to cool down and reset before the next attempt. Not only that, multiple research groups often share use of the tokamak, further limiting the time available for experiments.

Given the current obstacles to access a tokamak, researchers have turned to simulators to help advance research. For example, our partners at EPFL have built a powerful set of simulation tools that model the dynamics of tokamaks. We were able to use these to allow our RL system to learn to control TCV in simulation and then validate our results on the real TCV, showing we could successfully sculpt the plasma into the desired shapes. Whilst this is a cheaper and more convenient way to train our controllers; we still had to overcome many barriers. For example, plasma simulators are slow and require many hours of computer time to simulate one second of real time. In addition, the condition of TCV can change from day to day, requiring us to develop algorithmic improvements, both physical and simulated, and to adapt to the realities of the hardware.

Success by prioritising simplicity and flexibility

Existing plasma-control systems are complex, requiring separate controllers for each of TCV’s 19 magnetic coils. Each controller uses algorithms to estimate the properties of the plasma in real time and adjust the voltage of the magnets accordingly. In contrast, our architecture uses a single neural network to control all of the coils at once, automatically learning which voltages are the best to achieve a plasma configuration directly from sensors.

As a demonstration, we first showed that we could manipulate many aspects of the plasma with a single controller.

The controller trained with deep reinforcement learning steers the plasma through multiple phases of an experiment. On the left, there is an inside view in the tokamak during the experiment. On the right, you can see the reconstructed plasma shape and the target points we wanted to hit. (credit: DeepMind & SPC/EPFL)

In the video above, we see the plasma at the top of TCV at the instant our system takes control. Our controller first shapes the plasma according to the requested shape, then shifts the plasma downward and detaches it from the walls, suspending it in the middle of the vessel on two legs. The plasma is held stationary, as would be needed to measure plasma properties. Then, finally the plasma is steered back to the top of the vessel and safely destroyed.

We then created a range of plasma shapes being studied by plasma physicists for their usefulness in generating energy. For example, we made a “snowflake” shape with many “legs” that could help reduce the cost of cooling by spreading the exhaust energy to different contact points on the vessel walls. We also demonstrated a shape close to the proposal for ITER, the next-generation tokamak under construction, as EPFL was conducting experiments to predict the behaviour of plasmas in ITER. We even did something that had never been done in TCV before by stabilising a “droplet” where there are two plasmas inside the vessel simultaneously. Our single system was able to find controllers for all of these different conditions. We simply changed the goal we requested, and our algorithm autonomously found an appropriate controller.

We successfully produced a range of shapes whose properties are under study by plasma physicists. (credit: DeepMind & SPC/EPFL)

The future of fusion and beyond

Similar to progress we’ve seen when applying AI to other scientific domains, our successful demonstration of tokamak control shows the power of AI to accelerate and assist fusion science, and we expect increasing sophistication in the use of AI going forward. This capability of autonomously creating controllers could be used to design new kinds of tokamaks while simultaneously designing their controllers. Our work also points to a bright future for reinforcement learning in the control of complex machines. It’s especially exciting to consider fields where AI could augment human expertise, serving as a tool to discover new and creative approaches for hard real-world problems. We predict reinforcement learning will be a transformative technology for industrial and scientific control applications in the years to come, with applications ranging from energy efficiency to personalised medicine.

In May 2024, we released TORAX, a new open source plasma simulator. TORAX models the “core” (interior) of the plasma, and predicts changes in temperature, density, and electric current. This expands our ability to train advanced tokamak AI controllers. TORAX is written in JAX, a Python framework originally developed to train AI, and offers exciting capabilities for scientific computing through fast and scalable computation, increased prediction accuracy, and sensitivity analysis. By making TORAX available to the fusion community with access to these new capabilities we hope it enables development of new workflows for general purpose tokamak design and optimization.

Notes

Read the paper: Magnetic control of tokamak plasmas through deep reinforcement learning

Read the paper: TORAX: A Fast and Differentiable Tokamak Transport Simulator in JAX

Follow us
footer__x
footer__instagram
footer__youtube
footer__linkedin
footer__github
Build AI responsibly to benefit humanity
Models
Build with our next generation AI systems
Gemini
Gemma
Veo
Imagen
Lyria
Science
Unlocking a new era of discovery with AI
AlphaFold
SynthID
WeatherNext
Learn more
About News Careers Research Responsibility & Safety
Sign up for updates on our latest innovations

I accept Google's Terms and Conditions and acknowledge that my information will be used in accordance with Google's Privacy Policy.

Please enter a valid email (e.g., "[email protected]")
About Google
Google products
Privacy
Terms