NebulaStream Goes Open Source: High-Performance Data Management for the Internet of Things (Now Publicly Available)

NebulaStream, an extensible, high-performance streaming engine for multi-modal edge applications is now open-source. Developed for the Internet of Things by BIFOLD, DIMA, and DFKI researchers, this novel stream processing engine eases the analysis of sensor data in real-time. The source code is now freely available under an Apache 2.0 license.

Jul 4, 2025, 9:00:00 AM
Jean-Paul Olivier , Berlin Institute for the Foundations of Learning and Data – BIFOLD

Logo: Berlin Institute for the Foundations of Learning and Data – BIFOLD

The NebulaStream architecture for real-time stream processing at the edge. The open-source system facilitates the efficient analysis of distributed sensor data across diverse hardware and use cases like smart medicine and predictive maintenance.

Berlin, July 4, 2025 – The Berlin Institute for the Foundations of Learning and Data (BIFOLD) announces the open-source release of NebulaStream, a next-generation stream processing engine built for the unique challenges faced in IoT environments. As of today, the software is freely available under an Apache 2.0 license. Furthermore, the NEEDMI Project, which leverages NebulaStream to implement a smart co-pilot for an intensive care unit (ICU), recently won the SIGMOD 2025 Best Demo Honorable Mention award. The demo showcases NebulaStream's ability to integrate and process multi-modal, multi-frequency data streams, to improve the health assessment of patients in a smart ICU. Built for the edge NebulaStream was designed to process data directly where it is generated – at the sensor, edge device, or gateway. The system supports diverse hardware architectures (ARM, x86) and compiles user queries into hardware-tailored code. Adaptive resource management ensures robust performance, even under changing conditions or without Internet connectivity. Target applications include predictive maintenance, environmental monitoring, smart manufacturing, or smart medicine applications. Flexible, extensible, ready for production NebulaStream comes with out-of-the-box support for common stream tasks, such as data alignment and inference. Developers can easily add custom operators, formats, and optimizations, making the system highly adaptable to specialized requirements – without having to deal with low-level internals. Demos NebulaStream powers innovative applications across diverse domains. The IoTropolis project demo simulates a smart city to demonstrate how NebulaStream addresses data management challenges in IoT, eHealth, and Smart Grid scenarios [Video: https://www.youtube.com/watch?v=Yt00270NkP0&t=2s]. In the healthcare sector, a NEEDMI project demo was developed to showcase how NebulaStream can be employed to support medical staff in intensive care units by enabling real-time data processing at a patient’s bedside [Project: https://nebula.stream/needmi/index.html], [Video: https://www.youtube.com/watch?v=7tH6zkCdMG8&t=9s]. From research to real-world impact NebulaStream was developed as part of a collaborative research initiative led by BIFOLD, the DIMA Group at TU Berlin, and the IAM Group at DFKI. First presented in a scientific preprint in 2020, the system has now been released as open-source in the SIGMOD 2025 conference. “We want to thank all contributors that made this possible, with their huge efforts over the last years. Let us celebrate this together,” says Steffen Zeuch, Project Lead and System Architect of NebulaStream. Get started now: The source code, documentation, and contribution guidelines are available at: https://nebula.stream Paper overview: https://dl.acm.org/doi/10.1145/3722212.3725118

Contact for scientific information:

Volker Markl and Steffen Zeuch BIFOLD / TU Berlin / DIMA Group Einsteinufer 17, 10587 Berlin, Germany Tel: +49 30 314 23555 Mail: nebulastream@dima.tu-berlin.de

Original Publication:

NebulaStream: An Extensible, High-Performance Streaming Engine for Multi-Modal Edge Applications. Adrian Michalke, Aljoscha Lepping, Volker Markl, Ricardo Martinez, Nils Schubert, Lukas Schwerdtfeger, Taha Tekdogan, Steffen Zeuch, Ariane Ziehn, Christoph Falkensteiner, Kyle Krüger, Alexander Meyer, Tobias Röschl, Svea Wilkending. https://dl.acm.org/doi/10.1145/3722212.3725118

Source:

https://idw-online.de/de/news854916