Overview
The Cumulocity Team are excited to introduce you to Cumulocity’s brand new Data Preparation application as a public preview – a powerful addition to our IoT platform that’s going to make device integration and data transformation simpler, more flexible, and more scalable.
In this article, we will give an overview of what it is about and what you can expect.
If you are interested in more information, check out the latest video:
What is it about?
What is “Data Preparation” and why is it necessary?
A challenge today for many companies is managing the vast amounts of raw data generated by connected devices. Incoming data is often spread across multiple product generations, versions, or device types, causing discrepancies and inconsistencies in the data. This greatly complicates fleet-wide analytics and decision-making, leading to delayed insights, operational inefficiencies, and potential quality issues.
As organizations increasingly explore AIoT and AI-driven operational use cases, the importance of consistent and trusted data continues to grow. Analytics applications, automation workflows, and AI-powered solutions all rely on accurate context to interpret device information correctly. Raw device data often arrives in different formats, uses different naming conventions, and follows different structures depending on the device or vendor. Without a common structure, both people and systems can struggle to interpret information consistently, making it harder to scale analytics, automation, and AI initiatives across an entire fleet.
Data Preparation, Cumulocity’s solution to transform, harmonize, and standardize incoming data, provides a simple answer to this challenge. It offers a robust, resilient, and performant approach for converting complex device payloads into a unified format within Cumulocity. Rather than requiring teams to build and maintain custom transformation logic for every device integration, Data Preparation helps create consistent, reusable data structures that can be consumed across applications, dashboards, analytics workflows, and future AI-driven solutions.
By standardizing data at the point of ingestion, organizations can establish a trusted operational context across their connected assets. This helps ensure that downstream systems, whether they are dashboards, reporting tools, automation workflows, or AI-powered applications, are working from the same consistent and reliable information foundation.
Data Preparation has been designed specifically to work with the new Cumulocity MQTT Service to enable quick, easy, and seamless integration of device data. In addition, the Data Preparation application makes use of the latest AI innovations, allowing you to easily create transformation rules by describing them in natural language.
Welcome to the Public Preview of the new Data Preparation application, featuring:
Creation and execution of rules
A highly performant data engine for processing and transforming incoming data at scale.
Support for a wide variety of incoming payloads
Your devices can speak their native language, sending any arbitrary payloads onto the Cumulocity MQTT Service. Data Preparation is capable of processing and applying transformation logic to those payloads. The UI, however, is focused on JSON payloads.
Live Capture from MQTT Service
You can quickly and easily select which payloads you want to process by viewing the incoming message stream from the Cumulocity MQTT Service. This makes it simple to connect a device, set up a rule in Data Preparation, and onboard your data to Cumulocity.
AI-enabled code generation
Using the AI Agent Manager, you can connect your AI to Data Preparation, enabling you to quickly and easily create rules in natural language. All you need to do is say “convert this to a Cumulocity measurement” and the rest is done for you.
This significantly reduces the effort required to onboard new device types and accelerate data standardization. Instead of spending time writing and maintaining custom transformation logic, teams can focus on defining the outcomes they want to achieve while Data Preparation assists with generating the required rules. By helping organizations establish a consistent data foundation more quickly, Data Preparation supports analytics, automation, and emerging AI-driven initiatives across the enterprise.
The Data Preparation application does not replace the existing open-source Dynamic Mapper. If you have data transformation needs today, the Dynamic Mapper remains available for production use. As Data Preparation continues to evolve, we are keen to reach feature parity with the existing Dynamic Mapper and are leveraging the learnings from that project to guide our development roadmap.
Try it out!
If you are interested in trying Data Preparation, follow the information below.
The Data Preparation application is available on all Cumulocity SaaS instances. To activate it, go to the Administration application and select your user profile in the top right corner. Then, find the section called Preview features and select ‘Manage preview features.’ From the available options, enable the Data Preparation feature.
To activate the AI functionality in Data Preparation, please ensure that you have the AI Agent Manager set up and configured on your tenant. Instructions for doing this can be found here.
Note: Please be aware that Data Preparation is made available as a Public Preview and may be altered prior to GA, which may introduce breaking changes.