Cloud HTTP Proxy 2.13.1: Sporadic Timeouts on Remote Access Despite Locally Reachable Target

Hi everyone,

we are using the Cumulocity Cloud HTTP Proxy to remotely access a RTU’s web interface via Remote Access. While the target device is fully reachable locally, the proxy connection intermittently freezes and times out.
Remote Access initially works partially, but then with no changes to the software stack, gets stuck on pending and eventually times out.
· Initially, mostly CGI calls were affected (/webserver/cgi-bin/spre.cgi?cgi_menu).
· Now, even static files like /webserver/webserver.nocache.js hang.
The behavior occurs both when targeting the RTU directly (192.168.10.3:443) and when going through the local adapter.
A local curl call on the Gateway via the HTTPS adapter downloads the same JavaScript file completely in approximately 3 seconds:
HTTP status: 200 Size: 8,215 bytes
Local unauthenticated CGI calls return the expected 401 response. The target device is therefore fully reachable locally.

During a logged Cumulocity attempt, only / and /spin.gif were forwarded to the local adapter and answered with 200.
Cloud → Gateway Fully forwarded to the adapter
Gateway → Cloud Successfully written to the WebSocket

Observations:

  1. The bootstrap script request pending in the browser (webserver.nocache.js) was not visible in the adapter’s HTTP log the request appears to have never reached the Gateway .
  2. The adapter connection then remained idle.
  3. After approximately 30 seconds, the remote side closed the WebSocket with code 1000 (normal close).

Is this related to the bug fixed in v2.13.1 / PR #282 (“keep connection servers open while in use”)?
We have updated to the newest “Version 2.13.1” but have still the same problem.

Has someone seen a similar problem?

Kind regards,

Valerian

@v.reschke What are you using on the device side, are you using thin-edge.io? If so, there were a few improvements that will be included in the upcoming thin-edge.io release 2.1.0.

Currently the proxying of http/https traffic via the Cumulocity Remote Access PASSTHROUGH isn’t the quickest, as every HTTP/S request (e.g. all the assets being downloaded in the webpage), which generally results in some slowness…though we’ve been looking at using some multiplexing technics using yamux to improve this, however it requires changes on both ends (server and client).

If you can’t wait, then using a SSH PASSTHROUGH connection will be your best bet and using port forwarding (if you have SSH available that is).

Hello @Reuben_Miller2 ,
thank you for the detailed explanation regarding the HTTP proxying limitations.
To clarify our setup we are not using thin-edge.io at this point. We are using an own agent with self-written proxy. All requests from the tunnel are logged. For this incident, some requests are simply missing. However, the issue we’re experiencing is not about slow responses, it’s about missing requests. When we look at our logs, some HTTP requests from the cloud proxy simply never arrive.
The behavior is also inconsistent: Sometimes the initial page loads fine, sometimes it gets stuck on pending, all on the same installation without any changes. This allows us to rule out a systematic failure in our application.

That said, we will set up a variant using thin-edge.io to compare the behavior and narrow down the root cause.

Kind regards,
Valerian