With the upcoming Cyber Resilience Act (CRA), continuous software and vulnerability monitoring isn’t just a “nice-to-have” security feature anymore—it’s quickly becoming a regulatory requirement. For IoT product manufacturers and fleet operators, this means knowing exactly what software, firmware libraries, and OS versions are running across your entire fleet at any given moment, and actively matching them against known vulnerabilities.
To address this challenge, I’ve kicked off an open-source project that extends Cumulocity Device Management with a smart security plugin. It scans the software installed on device fleets, checks for vulnerabilities, and uses an AI Agent to interpret the findings into actionable remediation advice.
However, as exciting as AI agents are, they introduce a massive architectural challenge unique to IoT: Scale vs. Token Costs.
In this post, I want to talk about how we can build AI agents for IoT without breaking the bank, sharing a highly cost-effective design pattern you can deploy in your own tenant.
The IoT-AI Paradox: The Million-Device Token Trap
Imagine you manage an enterprise IoT fleet of 1 million devices. You want to use an AI agent to analyze the software inventory of each device, look up CVEs, and write a human-readable recommendation for your operations team.
If you take a naive, brute-force approach—running one agent analysis per device—the economics quickly fall apart:
-
Fleet Size: 1,000,000 devices.
-
Average Tokens per Analysis: 1,500 tokens (vulnerability context + recommendation).
-
Cost: At a typical rate of 15$ per million tokens, a single device check costs roughly $0.0225.
Do the math for the whole fleet:
Naive Fleet Cost = 1,000,000 devices * $0.0225$ = $22,500 per scan
Running this daily or even weekly is a financial impossibility. To make AI viable in the real world of IoT, we must reduce and normalize our data before it ever reaches an AI context.
The Solution: “Pool Testing” Your Fleet Data
To solve this, we can borrow a highly successful approach from healthcare: pool testing (famously used during the COVID-19 pandemic).
Instead of processing 10,000 individual tests, labs would mix samples from 50 people together. If the pool came back negative, all 50 people were cleared in a single test. Only if a pool came back positive would they run individual tests. This resulted in massive cost and throughput optimization.
We can apply the exact same logic to IoT software inventories by focusing on the software packages rather than individual devices:
-
Aggregate & Normalize: Query the software fragments of all 1,000,000 devices from the Cumulocity Inventory.
-
Generate a Global Fleet SBOM: Extract and deduplicate every single software package and version running across your entire fleet to create one master list. Even if you have 1,000,000 devices, they likely run the same set of operating systems, libraries, and agent versions.
-
Scan the Pool: Run the NVD lookup and AI agent analysis only on the deduplicated, unique software components in this global list.
-
Fan-Out Results: Map the discovered vulnerabilities and AI recommendations back to every individual device running that affected software.
Even for a massive fleet, your total variety of software packages might only be a few hundred unique components and versions. If your global fleet SBOM boils down to just $150 unique software/version combinations:
Pooled Fleet Cost = 150 unique software packages * $0.0225 = $3.38 per scan
By pooling and deduplicating at the software package level, we achieve a 99.98% cost reduction, turning a financial roadblock into a highly viable, daily security feature!
System Architecture & The Cumulocity AI Agent Manager
Rather than calling an LLM directly, this project is built to integrate with Cumulocity IoT’s ecosystem. The backend microservice registers and manages the AI Agent directly through the Cumulocity AI Agent Manager.
This is the recommended approach for enterprise applications: it ensures secure, governed access to AI models, centralizes prompt configuration, and utilizes Cumulocity’s built-in security and execution safeguards.
Here is how the architecture fits together:
+---------------------------------------------------------------------------------------+
| CUMULOCITY AIoT PLATFORM |
| |
| +-----------------------------+ +-------------------------------------+ |
| | Devicemanagement UI | | Inventory API | |
| | (Custom Vulnerability Tab) | | (Fetches Installed Software) | |
| +--------------+--------------+ +------------------+------------------+ |
| | | |
| | (Displays results) | (Raw inventories) |
| v v |
| +--------------+-----------------------------------------------+------------------+ |
| | PYTHON MICROSERVICE BACKEND | |
| | | |
| | 1. Normalizer & Aggregator (Generates Unique Fleet-Wide Global SBOM) | |
| | 2. CVE Lookup Manager (Calls NVD API per Software & Caches Results) | |
| +--------------------------------------+------------------------------------------+ |
| | |
| | (Invokes Registered Security Agent) |
| v |
| +--------------------------------------+------------------------------------------+ |
| | CUMULOCITY AI AGENT MANAGER | |
| | - Centralized Agent Governance & Secure LLM Connection | |
| +--------------------------------------+------------------------------------------+ |
+-----------------------------------------|---------------------------------------------+
|
v
+-----------+-----------+
| Enterprise LLM |
| (Gemini / Azure / etc)|
+-----------------------+
The Workflow:
-
The Aggregator pools all device software inventories from across the fleet into a single, deduplicated Global Fleet SBOM.
-
The CVE Lookup Manager queries the National Vulnerability Database (NVD) for each unique software component/version in that global list.
-
The microservice formats the discovered vulnerabilities for each affected component into a structured prompt and sends it to the Cumulocity AI Agent Manager.
-
The AI Agent interprets the technical CVEs for that specific software package and generates human-readable remediation advice (e.g., “This version of Mosquitto is vulnerable to CVE-XXXX. Update to version 2.0.16 to patch”).
-
The microservice maps these package-level recommendations back to the corresponding devices in the Cumulocity Inventory that run them.
Front-End Integration: Device Management Plugin
To make this actionable for operations teams, the results are displayed directly in Cumulocity Device Management:
- Vulnerability Dashboard: Highlights the fleet-wide security posture.
Aligning with the Cyber Resilience Act (CRA)
Our proposed model isn’t just about keeping your cloud bills low—it directly solves real-world compliance. Under the CRA, manufacturers and operators must:
- Maintain an active SBOM for all devices actively deployed in the market.
- Actively monitor and report vulnerabilities on those devices.
- Provide clear risk mitigations and remediation paths.
By combining Cumulocity’s device inventory with automated, pooled AI scanning, you get continuous security audits and instant compliance documentation at a fraction of the cost.
The Evolution: From Connectivity to the Semantic AIoT
The real shift we are seeing today isn’t just about moving from “read-only” to “bi-directional” communication—IIoT has been capable of that for years. The true breakthrough is the creation of a semantic layer on top of raw IoT data.
In “old school” IIoT, data is often siloed or context-free; an LLM wouldn’t know what to do with a raw software fragment without a human expert explaining it. Modern AIoT platforms, powered by tools like the Cumulocity AI Agent Manager, provide the structured context—the meaning—that allows AI agents to act as autonomous collaborators.
By transforming raw inventory fragments into a normalized, global SBOM, we are building the semantic bridge that allows AI to reason across a million devices as easily as it does a single document. This ability to build intelligent, context-aware applications directly on your IoT data is what defines the next generation of industry leaders.
Give it a Spin!
The entire setup is available as an open-source template for you to try out in your own Cumulocity tenant. It includes:
- The Python Backend Microservice to handle the NVD integration, pooling logic, and AI Agent Manager registration.
- The Frontend Web SDK Plugin to add the custom security tabs to your Device Management application.
Quick Start:
- Deploy the Microservice: Upload the Python microservice to your Cumulocity tenant.
- Configure the AI Agent: Define your model credentials and prompts in the Cumulocity AI Agent Manager.
- Install the UI Plugin: Build and upload the frontend package using
c8y-cli. - Scan Your Fleet: Head to your Device Fleet view, trigger a scan, and watch your security posture update in real-time—all while keeping token usage to an absolute minimum.
Join the Conversation & Build With Us!
Architecture is always an evolving craft. While the “Pool Testing” pattern solves the scale problem, I’m curious to hear how you handle edge cases—like custom proprietary binaries or regional CVE databases.
Do you have ideas on how to further optimize the prompt engineering or integrate different vulnerability sources? I’d love to hear your thoughts!
-
Contribute: Check out the GitHub Repository to submit PRs or open issues.
-
Discuss: Drop a comment below to share how you’re tackling CRA compliance.
-
Brainstorm: Have a unique IoT use case that needs a custom AI agent? Let’s chat!
Disclaimer: This is an open-source community project and is not officially supported by Cumulocity. Always test AI-suggested software updates in a staging environment before updating production fleets.