Skip to main content

Video

Browsers play WebRTC or HLS. The Suite never connects a browser to an RTSP source.

The pipeline​

  1. A camera publishes RTSP to the relay on the aircraft.
  2. The relay re-publishes into MediaMTX on the aircraft, keeping the camera's codec and limiting bandwidth with the Jetson's own encoder.
  3. The Suite resolves a deterministic path for the vehicle and camera slot.
  4. The browser receives an authenticated Suite proxy URL and performs the WHEP handshake through it. HLS segments use the equivalent authenticated proxy.
  5. MediaMTX fans the stream out to every viewer.

The vehicle has three fixed camera identities: camera-1 for MT11 zoom, camera-2 for MT11 thermal, and camera-3 for R1M FPV. Suite stream slot N addresses camera-N; its display label does not remap that identity. Integration requires a runtime and gateway client supporting all three cameras; the Suite uses @sab-group-srl/onboard-gateway 0.3.0.

The default MediaMTX path template is {droneId}-camera-{index}, with the gateway id rather than the Suite database id. A valid path returned by the vehicle relay takes precedence over the template; an explicit saved published-path override can also provide the path. Stream definitions are padded to three slots and capped at three, even if environment configuration declares fewer or more kinds. Ordinary camera-source forms accept RTSP/RTSPS addresses, not arbitrary published-path overrides.

The authenticated metadata endpoint is GET /api/drones/streams?droneId=<id>. Playback stays on the Suite origin at /api/drones/:droneId/playback/:path/whep and /api/drones/:droneId/hls/:cameraId/:path/*; the proxy adds the short-lived gateway credential upstream.

WHEP signalling passes through the Suite; WebRTC media follows the selected ICE route between the browser and the vehicle, directly or through a relay. An HTTPS response from the proxy does not prove that the media path works. HLS manifests and segments do pass through the authenticated Suite proxy.

Playback and telemetry​

The playback service authorizes Clients for their visible vehicles as well as staff. The current Drone Control toolbar nevertheless requires isDroneConnected before it enables the camera launcher. Since Client accounts have no live telemetry links in that page, their launcher remains disabled. This is a current UI limitation, not a restriction in the media-proxy authorization. See the user-facing limitation.

Closing a published player, including a detached window, ends its local playback without asking the vehicle to stop the fixed path. Other viewers may still need that camera. No viewers means no video traffic on that path over the mobile link. The explicit stream-stop API remains a separate operation and must not be treated as the browser player's close action.

The shared @sab-group-srl/onboard-gateway client owns WHEP signalling; the Suite player owns its peer connection, playback state and reconnection. HLS playback prefers hls.js with low-latency mode when supported and uses native browser HLS where hls.js is unavailable. Startup, fallback and network recovery measurements remain tracked in issue #53.

Camera controls​

Gimbal and zoom use the selected stream's saved configuration. The gimbal option is per camera slot; the presence of a zoom control does not establish that the physical camera supports optical zoom.

The SIYI optical-zoom flow derives the control host from that stream's RTSP source, reads the current zoom with 0x18, then sends a heartbeat followed by the 0x0F absolute optical-zoom command. Reading the camera's physical state keeps controls synchronized with other stations. An optical-zoom camera such as the UniPod MT11 supports this operation; a fixed-lens R1M does not.

The browser obtains short-lived gateway credentials from the Suite and sends gimbal and zoom commands directly to the vehicle, with authenticated Suite routes as a fallback. See The remote link for the request path and Remote access switch for the effect of disabling access while a token is still valid.

Manual RTSP preview​

A separate, explicit test mode. POST and DELETE /api/drone-control/rtsp-stream create and remove a temporary MediaMTX path that proxies a URL you supply. It needs a MediaMTX control API reachable from the server runtime and started with api: yes.

Codecs​

An H.265 source stays H.265 through the relay. Where the browser cannot play it, the conversion belongs on the aircraft, which has a hardware encoder.

The Suite once carried a local ffmpeg transcoder for this. Encoding remains on the aircraft after the Oracle migration: MEDIA_TRANSCODER_MODE=disabled is the only value that makes sense in production.

When video does not appear​

  • Check that the configured MediaMTX control and playback origins are reachable from the Suite Node service. The browser reaches only the Suite's authenticated proxy paths.
  • Check the source codec, and whether the browser can play it.
  • Check the camera slot and path naming against the template.
  • Localhost defaults are development-only. In production an unset MediaMTX URL stays unset rather than falling back to 127.0.0.1.