Hi everyone,
I’m currently trying to use the Dynamic Mapper to periodically poll data from an external REST API and map it as e.g. measurements into Cumulocity.
trigger (e.g. 5min) – > REST GET (Extr. API) – > Mapping result – > Create MEA Cumulocity
For Outbound, there is the Webhook Connector, which allows sending data to an external REST API.
For Inbound however, I don’t see any connector that can actively poll an external REST API.
I’m aware that it is possible to use a custom Microservice that handles the polling and then either writes the data directly into Cumulocity or forwards it to the Dynamic Mapper.
A native REST polling feature directly in the Dynamic Mapper would be much more elegant and helpful, as it would allow leveraging the mapping functionalities of Dynamic Mapper, without having to develop and maintain an additional Microservice. Especially for scenarios where external systems only offer a REST API and don’t support push data, this would be a great added value.
Is there a way to actively poll an external REST API with the Dynamic Mapper and map the data towards Cumulocity that I’m not aware of?
Is a REST Polling Connector (Inbound) planned or on the roadmap?
If anyone has experience with this topic or can provide information on how this can be solved without significant effort (i.e. without a custom Microservice). I’d really appreciate it.
Kind regards
currently this is not supported in Dynamic Mapper, but this looks like a natural features we can add easily.
Before we look into this in more deail some qualifying questions beforehand:
How often do you need new data? (e.g. every 30 seconds, every 5 minutes, hourly)
Which endpoints do you need, and do they live behind one host with one set of
credentials? (e.g. everything under https://api.example.com/v1 with a single API key) — or are some on different hosts, tenants, or auth altogether?
How does the API authenticate a request? (e.g. a static API key/Bearer token, Basic auth with username/password) — or something more involved like OAuth2 with token refresh, mTLS, or a self-signed/internal CA?
What does a response look like, and does the API support fetching only what’s new? (e.g. a JSON array with a ?since=<timestamp> filter, a Link: rel="next" header, or a nextPage token in the body) — or does every call just return the same full dataset?
How should duplicate records be handled if the same one arrives twice? (e.g. downstream dedup on an ID, upsert semantics) — or does every record need to arrive exactly once?
thanks for the detailed questions , here are our answers:
· Polling frequency: We would need new data every 60 seconds. It would be nice if the interval could be configurable.
· Endpoints / hosts: For our current setup, a single endpoint with one set of credentials would be sufficient.
· Authentication: Authentication would be done via a static API key or Basic Auth (username/password).
· Response format / incremental fetching: The API call will return the same full dataset each time, with the most recent value at the top. There is no support for incremental fetching (I’m not fully aware of all available options at this point, so that might be worth exploring ).
· Duplicate handling: If the same measurement value appears in the response after the configured interval (i.e. the value hasn’t changed), it should still be transmitted — no deduplication needed. Every response is treated as valid and forwarded as-is.
Hi Christof,
thank you very much for the quick implementation! I’ve tested the function and it works as expected. I’m receiving the data and I’m able to map it to the required information.
The only issue I encountered is adding additional headers. When I click the plus icon, nothing happens. However, once the connector has been created and I go back to make changes, adding headers works again.
Thanks again for the fast support and implementation!
