How AI is revolutionizing image analysis

Several pictures of brains scanned organs with colored parts.

What can you see on the screen there? Is that part of a lung carcinoma? Or healthy lung tissue? Several million times a minute, the algorithm that medical computer scientist Fabian Isensee trained in Klaus Maier-Hein's team at the German Cancer Research Center (DKFZ) asks this question using a new method: What can be seen here?

For each pixel of an image from a patient's body, for example an MRI or CT image, the program searches for and finds an answer. "In radiation therapy, doctors usually laboriously mark the cancerous tissue and surrounding structures themselves, which often takes several hours," says Fabian Isensee. "We try to relieve them of this work to some extent." To do this, they feed their program called nnU-Net with images that have already been marked. It learns from this using artificial intelligence – and applies what it has learned to new images.

The program nnU-Net

The name nnU-Net is a phonetic abbreviation for "no new net". With this, the team around Klaus Maier-Hein and Fabian Isensee wants to set an example: Their program does not require a new network or architecture; instead, it is based on a further developed standard method.

Maier-Hein and his team are not alone in this approach. The use of artificial intelligence in medical imaging is a rapidly growing field today. In many large cancer treatment centers, radiation therapists are already using algorithms to mark tumor tissue. The field is considered one of the major future topics in medicine.

Versatile benefits without time-consuming programming work

However, the new method nnU-Net developed by the experts at DKFZ has shaken up the research field in the past year. What is special about the program is that it is not optimized for a specific segmentation task, but can be used in a variety of ways – without a lot of programming work. And that is a small revolution in the analysis of images using AI.

The first time it became clear that the Heidelberg team had initiated a paradigm shift was when they took part in a competition for AI-supported image analysis, the so-called Medical Segmentation Decathlon. "You can think of a single image type analysis task as being like a race track," explains Klaus Maier-Hein. "Typically, you have a base car – the program, how the algorithm is trained – and then with various add-ons and tuning, you specifically improve the car so that it's better in certain corners and on certain parts of the track, so it's better overall." In the decathlon, however, not just one data set was put up for analysis, but ten different ones: from the lungs, from the brain, from the abdominal cavity and from other body regions. Customized attachments were no longer an option, and so most teams simply participated with a rigid base algorithm that, on average, reasonably fit the given data.

A self-tuning program

Isensee and his colleagues, however, chose a different path. "We optimized the fine-tuning of the base car by first systematizing it and then automating it." In other words, the medical computer scientists' basic car looks at the race track – and tunes itself based on it. "This means we're not just faster on one track, but everywhere," says Maier-Hein.

The new automatic tuning was convincing: Isensee and his team won first place by far at the Medical Segmentation Decathlon. The fact that they had indeed created a new basic method became apparent in the months that followed: "Since then, most teams have relied on nnU-Net for segmentation tasks and competitions to this day," says Isensee.

The free open-source program can also be used by laypersons

Another advantage is that even medical informatics laypersons can use the tool in an uncomplicated manner and require comparatively little technical support. For broad application in everyday clinical practice, this should be a huge advantage in the future. The team around Klaus Maier-Hein makes nnU-Net available for download free of charge as an open source tool. This means that the program is ready for immediate use.

Possible applications in nuclear physics, space and deep-sea research

By the way, the program can be applied not only in medicine. Basically any kind of image analysis to a good quality is possible. All that is needed is a sample data set that has already been added – so that the program can learn from it. If this is the case, nnU-Net could, for example, help marine researchers in the future to analyze seafloor photos from the deep sea; in nuclear physics, images could be broken down using an electron microscope. And in space research, for example, images from the surface of the moon could be examined. In this way, the program could develop into a kind of "AI all-rounder" in research.

This text is a shortened versioned of an article originally published on the website of the Helmholtz Association in March 2021. For the full version, please visit www.helmholtz.de > How AI is revolutionizing image analysis.

“The next big step towards becoming an AI nation”

German Federal Research Minister Anja Karliczek recently signed an agreement to implement institutional funding of up to 100 million euros per year for the German Competence Centers for AI.