Cumulocity IoT Microservices - Monitoring through OTLP integration

[Robert_Pretz]

Overview - current situation

Cumulocity supports the deployment of custom logic through Kubernetes-based microservices. During development, developers can use a wide range of tools to analyze an application’s correctness, performance, and stability. However, once the application is packaged as an image and deployed as a Kubernetes microservice on the Cumulocity platform, access to its runtime environment becomes significantly restricted.

Monitoring microservice behaviour in production is essential to ensure stability, reliability, and performance. This article demonstrates how to greatly enhance observability in Cumulocity version 2025 and later - using zero-code instrumentation and minimal configuration - through the OpenTelemetry standard (OpenTelemetry).

Monitoring of microservices on the Cumulocity platform is already possible to some extent, however with additional effort and some limitations:

• All logging data is sent to the standard output and persisted temporarily by the infrastructure. You can access logging data via the Rest interface but limited to the last 35 MB. Logs are lost in case of a restart.

• Although log aggregation is possible already (Log Aggregation for Cumulocity IoT Microservices - Knowledge Base), you have to program your microservices against a specific technology like Grafana.

• Server runtime metrics like memory and CPU consumption are not logged (unless written to the log by the application).

With recent versions - refer to the “Prerequisites” section for detailed version information - Cumulocity now supports the OpenTelemetry standard for collecting and exporting monitoring data to monitoring systems of various vendors. An out of the box configuration of microservices has been introduced for generating and exporting metrics, logs, and traces with the OpenTelemetry Java agent (Java Agent | OpenTelemetry ) attached to the application.

A wide range of new monitoring options is available, activated by just setting a few configuration parameters:

• JVM metrics - CPU, memory, garbage collection, etc. - are collected by the Java agent out of the box and available in the monitoring system with zero code changes.

• Logging data is continuously exported by the Java agent and available in the monitoring system without any limitation of log file sizes and related coding.

• With spans and traces, the processing chain of requests to the application is made transparent and allows for analysis of runtime behaviour.

• It is just a matter of configuration of a microservice to turn on/off collection of each of these so-called signals (logs, metrics, traces).

• The export destination of data - e. g. Grafana - can be reconfigured at any time.

This article focuses on developing microservices with the Cumulocity Microservice SDK for Java which provides a zero-code OTLP instrumentation with a minimum of configuration steps (refer to Microservice SDK for Java - Cumulocity documentation ) .

Prerequisites

To directly implement the steps in this guide, you need:

• Access to a tenant in an environment running Cumulocity Core version 2025.331.0 or higher
• An account on a monitoring system like Grafana Cloud
• Basic knowledge in microservice development (using Microservice SDK for Java)
• Cumulocity Microservice Java SDK version 2025.75.0 or higher

Creating the microservice image

We use the Cumulocity Microservice SDK for Java to create the image and the deployable
zip file (Microservice SDK for Java - Cumulocity documentation ). To basically provision OTLP instrumentation of the microservice, the “opentelemetry-javaagent.jar” file must be included in the microservice image at build time. This is done by setting the parameter otelJavaAgentInclude to true in the microservice-package-maven-plugin section of the Maven pom.xml file of the custom microservice project:

  <plugin>
      <groupId>com.nsn.cumulocity.clients-java</groupId>
      <artifactId>microservice-package-maven-plugin</artifactId>
      <version>${c8y.version}</version>
      <executions>
          <execution>
              <id>package</id>
              <phase>package</phase>
              <goals>
                  <goal>package</goal>
              </goals>
              <configuration>
                  ...
                  <otelJavaAgentInclude>true</otelJavaAgentInclude>
                  ...
              </configuration>
          </execution>
      </executions>
  </plugin>

Configuring the microservice

Activating OTLP instrumentation at runtime

You can activate and configure OTLP through tenant options after uploading the microservice. The tenant options API is documented in Cumulocity - OpenAPI

Let’s assume we have a Cumulocity microservice application named “echo”, and the microservice is to be monitored. To turn on instrumentation, the OTLP parameter otel.javaagent.enabled must be set to true with a json object like this:

{
"category": "echo",
"key": "otel.javaagent.enabled",
"value": "true"
}

As an example, the cURL command line will look like this:

curl --location --request POST "https://<TENANT_DOMAIN>/tenant/options" \
--header "Authorization: Basic <AUTHORIZATION>" \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--data-raw '{"category": "echo", "key": "otel.javaagent.enabled", "value": "true"}'

You need to unsubscribe and re-subscribe the microservice for configuration changes to take effect.
To verify that the Opentelemetry Java agent is attached to the microservice application, check the first lines of the microservice log file after startup for a string like otel_agent_attach = -javaagent:/otel/opentelemetry-javaagent.jar.

OTLP configuration parameters

To export OTLP signals (metrics, logs, traces) to e. g. Grafana, the minimum set of configuration parameters is

• otel.exporter.otlp.endpoint
• otel.exporter.otlp.protocol
• otel.exporter.otlp.headers

The corresponding JSON objects sent to the tenant options endpoint are:

{
"category": "echo",
"key": "otel.exporter.otlp.endpoint",
"value": "https://otlp-gateway-...-prod.grafana.net/otlp"
}

{
"category": "echo",
"key": "otel.exporter.otlp.protocol",
"value": "http/protobuf"
}

{
"category": "echo",
"key": "credentials.otel.exporter.otlp.headers",
"value": "Authorization=Basic MTAxNTI...WZRPT0="
}

Don’t forget to unsubscribe and re-subscribe after parameter changes.

The prefix credentials in credentials.otel.exporter.otlp.headers requires some explanation. Sensitive information like access tokens, passwords, etc. should not be stored as plain text but in an encrypted form. Cumulocity supports encrypted storage of tenant options with the credentials prefix added to the original name. Retrieving such parameters shows "<<Encrypted>>" as value:

curl --location --request GET "https://<TENANT_DOMAIN>/tenant/options/echo/credentials.otel.exporter.otlp.headers"

Response:

{
  "self": "https://<TENANT_DOMAIN>/tenant/options/echo/credentials.otel.exporter.otlp.headers",
  "category": "echo",
  "value": "<<Encrypted>>",
  "key": "credentials.otel.exporter.otlp.headers"
}

The OTLP parameters otel.service.name and otel.resource.attributes are set automatically. They are used to identify the data source, in our case the originating microservice in Grafana. The parameter values are set as:

otel.service.name = <microservice-pod-name>

otel.resource.attributes = 
  service.name=<application-name>,
  service.instance.id=<microservice-pod-name>,
  service.namespace=<tenant-name>,
  service.version=<microservice-version"

The actual values of otel.service.name and otel.resource.attributes can be examined by retrieving the corresponding environment variables (OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES) of the microservice at runtime (refer to the section “Verifying configuration parameters” at the end of this document.

More OTLP configuration parameters are described in Configure the SDK | OpenTelemetry .

Microservice metrics

Gaining insight into the Java application runtime characteristics was only possible with support from system operators so far. Now the OTLP Java agent offers metrics data of various JVM parameters out of the box, like

• jvm_memory_used_bytes
• jvm_memory_committed_bytes
• jvm_cpu_time_seconds_total
• jvm_gc_duration_seconds_count
• jvm_thread_count
• jvm_class_loaded_total
• etc.

No additional configuration is required to get information about the JVM status in the connected monitoring system. In the monitoring tool, metrics data of your microservice can be selected with e. g. the service_name attribute:

Typically, metrics of more than 20 JVM parameters will be provided:

Metrics export configuration

The export destination for metrics and the other OTLP signals is the configured monitoring system by default. To modify the export destination of metrics data, a tenant option can be set like this:

{
"category": "echo",
"key": "otel.metrics.exporter",
"value": "otlp | console | none"
}

Setting otel.metrics.exporter to console can be temporarily useful for verification purposes in a development phase. Metrics data will then be sent to the console output in the microservice log file. With otel.metrics.exporter set to none, no data is exported at all.

By default, metrics data is exported once per minute. To export data e. g. every 10 seconds, the interval can be modified with a tenant option like this:

{
"category": "echo",
"key": "otel.metric.export.interval",
"value": "10000"
}

A list of the tenant options we applied so far can be retrieved from the Cumulocity platform with this command:

curl --location --request GET "https://<TENANT_DOMAIN>/tenant/options/echo"

Response:

{
  "otel.javaagent.enabled": "true",
  "otel.exporter.otlp.endpoint": "https://otlp-gateway.grafana.net/otlp",
  "otel.exporter.otlp.protocol": "http/protobuf",
  "credentials.otel.exporter.otlp.headers": "<<Encrypted>>",
  "otel.metrics.exporter": "otlp",
  "otel.metric.export.interval": "10000"
}

Microservice logging

Logging information from the microservice is automatically exported to the configured monitoring system. There is no need any more to retrieve such information from the microservice log files via REST requests. Logging data is sent to both, its log file and to the OTLP endpoint in parallel.

In the monitoring tool, logging data of your microservice can be selected with e. g. the service_name attribute:

By default, logging data is exported to the OTLP endpoint. Like for metrics data, the export destination can be configured with a tenant option:

{
"category": "echo",
"key": "otel.logs.exporter",
"value": "otlp | console | none"
}

Choosing none disables export of logging data. console sends data to the log file.

Microservice tracing

OTLP traces and spans are a powerful tool to analyze an application’s runtime behaviour (refer to Traces | OpenTelemetry ). Spans represent a method call in the application providing detailed information about the properties and the environment of this call. Traces are concatenations of spans which are created during processing of e. g. a HTTP or a database request. Such information allows to identify e. g. the number of method executions over time or to identify bottle necks in terms of processing time in a chain of method executions for certain requests.

In the monitoring system, traces and spans for your microservices can be selected by e. g. the service.name or by the resource.service.name attribute:

A trace with 3 spans representing a GET request to an “echo-agent” microservice:

Detailed trace with spans of a call of the “/owners” endpoint representing a chain of JDBC requests:

Such analytic views can help to analyze application timing and bottlenecks.

Like for logging data, the export destination for traces can be configured with a tenant option:

{
"category": "echo",
"key": "otel.traces.exporter",
"value": "otlp | console | none"
}

Choosing none disables export of tracing data. console sends data to the log file.

Manual instrumentation extension

On top of the auto-instrumentation by the Java agent, you can add custom manual instrumentation using the OpenTelemetry API to create application specific metrics and spans. The “GlobalOpenTelemetry” object is already set by the agent, and its “get()” method returns the “OpenTelemetry” instance (refer to Extending instrumentations with the API - Java ).

When using the Cumulocity Microservice SDK for Java, a simplified example could be set up like shown below (derived from opentelemetry-java-examples )

Required library

As a prerequisite, the OpenTelemetry API library must be included in the Maven pom.xml file:

<dependencies>
  <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-api</artifactId>
    <version>1.55.0</version>
  </dependency>
</dependencies>

Java classes

In the Spring context provided by the Microservice SDK, create an application class and a controller class with the code snippets below.

import com.cumulocity.microservice.autoconfigure.MicroserviceApplication;
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.metrics.LongCounter;
import org.springframework.boot.SpringApplication;
import org.springframework.context.annotation.Bean;

@MicroserviceApplication
public class App {
    public static void main(String[] args) {
        SpringApplication.run(App.class, args);
    }

    @Bean
    public OpenTelemetry openTelemetry() {
        return GlobalOpenTelemetry.get();
    }
}

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.metrics.LongCounter;
import io.opentelemetry.api.metrics.Meter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class CounterExample {

    private final LongCounter counter;

    @Autowired
    CounterExample(OpenTelemetry openTelemetry) {
        Meter meter = openTelemetry.getMeter(CounterExample.class.getName());
        counter = meter.counterBuilder("AccessCount").build();
    }

    @GetMapping("/ping")
    public String ping() throws InterruptedException {
        counter.add(1);
        return "OTLP counter incremented";
    }
}

Watch the custom counter

Each time when the “/ping” endpoint is called, the counter “AccessCount” gets incremented. The counter value is exported together with the JVM metrics generated by the Java agent to the monitoring system:

Troubleshooting

Basic verification

To attach the Java agent to the microservice application, the “opentelemetry-javaagent.jar” file must be contained in the image, and the OTLP parameter otel.javaagent.enabled must be set to true. To verify that both conditions are met, you could check the first lines of the microservice log file after startup for a string like otel_agent_attach = -javaagent:/otel/opentelemetry-javaagent.jar.

Verifying configuration parameters

The OTLP configuration parameters set in the tenant options are forwarded to environment variables of the microservice at runtime. To convert a tenant option to an environment variable, these steps are applied as described in Environment variables and system properties :

• Convert the name to uppercase
• Replace all “.” and “-” characters with “_”

For example, the otel.exporter.otlp.endpoint tenant option is equivalent to the OTEL_EXPORTER_OTLP_ENDPOINT environment variable.

To verify the effective OTLP parameter values in the microservice, the application code could contain a few diagnostic code lines printing the content of the corresponding environment variables to the console, e. g.:

System.out.println(System.getenv("OTEL_EXPORTER_OTLP_ENDPOINT"));

Conclusion

Monitoring the runtime behaviour of a microservice-based application is critical to ensuring reliability, stability, and efficiency. The introduction of a zero-code instrumentation option with the OTLP Java Agent addresses many previous limitations, such as the absence of continuous logging and the reliance on system operators for CPU and memory consumption data.

With OTLP traces and spans, data flows between application components become fully transparent, enabling detailed performance analysis. This enhanced tooling provides valuable insights for resource optimization, bottleneck detection, and bug resolution - all without writing any additional code. Configuration alone is sufficient.

For setup details, see the Microservice SDK for Java - Cumulocity documentation.

If you are not using the Microservice SDK for Java but any other programming language like Python, Go or .NET, there are multiple OTLP SDKs and instrumentation tools available:

SDKs:

Language APIs & SDKs

Zero-Code Instrumentation:

Zero-code Instrumentation

Summary

With 2025 release versions, Cumulocity provides significantly enhanced monitoring support for microservice applications with zero-code OpenTelemetry Java agent instrumentation. Logging data, JVM metrics, and Traces across processing sequences in the application is collected and exported to any monitoring system supporting the OTLP standard.

This feature is activated by just setting configuration parameters for the microservice to be monitored.

References

Microservice SDK for Java - Cumulocity documentation

Log Aggregation for Cumulocity IoT Microservices

Overview In my last article about Microservice Monitoring I explained 2 ways how you can achieve a proper monitoring for your microservices hosted in Cumulocity. I explicitly excluded logging and log aggregation from that topic because it is another article worthy - which follows now What is Microservice Logging & Log Aggregation? In general Logging for Microservices means that relevant information is printed out to the console and/or to some log files. Relevant information is …

Microservice Monitoring in Cumulocity IoT

Overview Cumulocity IoT can be extended with Microservices which are deployed to tenants. Large IoT solutions often consist of multiple microservices serving different purposes and might be critical for the overall success of that solution. In that case, you might not want to rely on “luck” or “prayers” but you want to make sure that your microservices are operating normal and if not instantly get informed about it. In short: You want to monitor your microservices. In this article, I will exp…