Consume a Live RTSP Stream

Field	Value
Difficulty	Intermediate
Estimated Read Time	5-10 minutes
Labels	rtsp, streaming, input-group, live-input

This is the first chapter where input originates outside the program. Earlier chapters manufactured test images or read files from disk; here, frames arrive continuously from a network stream and you consume them as fast as you pull. The mechanism is a reusable Graph fragment, RtspDecodedInput, that bundles the whole RTSP-to-raw-frames front end behind a single node.

The chapter deliberately stops at "pull the decoded frames." Feeding them into a Model is covered elsewhere (001 for a single model run, 007 for plugging a model into a pipeline, 015 for embedding a model inside a graph). By the end you will have connected to an RTSP URL and printed the tensor shape of each decoded frame — proof the stream is flowing.

This is a consumer only. To publish a stream, run a separate RTSP server (e.g. mediamtx) and point --url at it.

Walkthrough

Configure the RTSP client

RtspDecodedInputOptions is the configuration for the input fragment. The two fields that matter here are url (the rtsp://... source, taken from --url) and tcp, which selects the RTSP transport. Setting tcp = true requests RTSP-over-TCP — more robust across NAT and firewalls than the UDP default, at the cost of some latency. These options fully describe where frames come from and how they are carried.

tutorials/018_consume_rtsp_stream/consume_rtsp_stream.cpp

// Configure RtspDecodedInputOptions: the URL and the RTSP transport.
simaai::neat::nodes::groups::RtspDecodedInputOptions rtsp_opt;
rtsp_opt.url = url;
rtsp_opt.tcp = true;

Compose the graph

Build a Graph with just two stages: the RtspDecodedInput fragment (the source) and a bare Output node (the pull endpoint). Adding the fragment is a single add(...) — it expands internally into the connect/depacketize/decode elements, so your composition stays at the level of intent. Because the input originates inside the pipeline, we call the build(RunOptions{}) overload that takes no priming sample: there is no frame to hand build() up front, since the stream produces them.

tutorials/018_consume_rtsp_stream/consume_rtsp_stream.cpp

// Build a Graph whose only stages are the RTSP group and an Output node.
simaai::neat::Graph s;
s.add(simaai::neat::nodes::groups::RtspDecodedInput(rtsp_opt));
s.add(simaai::neat::nodes::Output());
auto run = s.build(simaai::neat::RunOptions{});

Pull decoded frames

With the run live, loop and pull(...) with a timeout. Each successful pull yields a Sample whose tensor is one decoded frame; we print frame=N shape=[...], where the shape is the frame in the decoder's native layout (typically [H, W] or [H, W, C] depending on the output format). A pull that returns nothing (or an empty tensor) prints frame=N rtsp_timeout and breaks the loop — that usually means the URL is wrong or the stream is not delivering. The timeout is what keeps a dead stream from hanging the program.

A frame is extracted with tensors_from_sample(*sample, true); the loop checks for an empty list before reading shape.

tutorials/018_consume_rtsp_stream/consume_rtsp_stream.cpp

for (int i = 0; i < frames; ++i) {
  auto sample = run.pull(/*timeout_ms=*/5000);
  if (!sample.has_value() || simaai::neat::tensors_from_sample(*sample, true).empty()) {
    std::cout << "frame=" << i << " rtsp_timeout\n";
    break;
  }
  const auto tensors = simaai::neat::tensors_from_sample(*sample, true);
  const auto& shape = tensors.front().shape;
  std::cout << "frame=" << i << " shape=[";
  for (std::size_t d = 0; d < shape.size(); ++d) {
    std::cout << shape[d] << (d + 1 < shape.size() ? ", " : "");
  }
  std::cout << "]\n";
}

Run

This chapter consumes a live RTSP stream, so you must supply a reachable --url; if you do not have a camera, publish an MP4 through mediamtx + ffmpeg and point --url at it. Run the Python and C++ (prebuilt) commands from the Neat install root (the directory that contains share/ and lib/); run the build from source commands from the repo root.

CTest reads SIMANEAT_APPS_TEST_RTSP_URL for this chapter's RTSP source. If you manage several sources, set SIMANEAT_APPS_TEST_RTSP_URLS and the first URL is used.

C++ (prebuilt):

./lib/sima-neat/tutorials/tutorial_018_consume_rtsp_stream \
  --url rtsp://host:port/stream --frames 5

C++ (build from source):

./build.sh --target tutorial_018_consume_rtsp_stream
./build/tutorials-standalone/tutorial_018_consume_rtsp_stream \
  --url rtsp://host:port/stream --frames 5

Expected output (shape depends on the stream's resolution and decoder format):

frame=0 shape=[720, 1280, 3]
frame=1 shape=[720, 1280, 3]
frame=2 shape=[720, 1280, 3]
frame=3 shape=[720, 1280, 3]
frame=4 shape=[720, 1280, 3]

If the stream is unreachable you will instead see frame=0 rtsp_timeout. To integrate this chapter's C++ source into your own project with a custom CMakeLists.txt (no extras folder required), see How to Run Tutorials on the landing page.

Full source

Show the complete C++ and Python programs

tutorials/018_consume_rtsp_stream/consume_rtsp_stream.cpp

// Consume a live H.264 RTSP stream via the RtspDecodedInput Graph fragment.
//
// The fragment handles RTSP connect, depacketize, and H.264 decode — you hand it
// a URL and pull decoded frames. This chapter is about the input fragment only.
//
// Usage:
//   tutorial_018_consume_rtsp_stream --url rtsp://host/path [--frames 5]

#include "neat.h"
#include "nodes/groups/RtspDecodedInput.h"

#include <exception>
#include <iostream>
#include <stdexcept>
#include <string>

namespace {

bool get_arg(int argc, char** argv, const std::string& key, std::string& out) {
  for (int i = 1; i + 1 < argc; ++i) {
    if (key == argv[i]) {
      out = argv[i + 1];
      return true;
    }
  }
  return false;
}

int parse_int_arg(int argc, char** argv, const std::string& key, int def) {
  std::string v;
  if (!get_arg(argc, argv, key, v))
    return def;
  return std::stoi(v);
}

} // namespace

int main(int argc, char** argv) {
  try {
    std::string url;
    if (!get_arg(argc, argv, "--url", url)) {
      std::cerr << "Usage: tutorial_018_consume_rtsp_stream --url <rtsp://...> [--frames <n>]\n";
      return 1;
    }
    const int frames = parse_int_arg(argc, argv, "--frames", 5);

    // CORE LOGIC
    // Configure RtspDecodedInputOptions: the URL and the RTSP transport.
    simaai::neat::nodes::groups::RtspDecodedInputOptions rtsp_opt;
    rtsp_opt.url = url;
    rtsp_opt.tcp = true;

    // Build a Graph whose only stages are the RTSP group and an Output node.
    simaai::neat::Graph s;
    s.add(simaai::neat::nodes::groups::RtspDecodedInput(rtsp_opt));
    s.add(simaai::neat::nodes::Output());
    auto run = s.build(simaai::neat::RunOptions{});

    for (int i = 0; i < frames; ++i) {
      auto sample = run.pull(/*timeout_ms=*/5000);
      if (!sample.has_value() || simaai::neat::tensors_from_sample(*sample, true).empty()) {
        std::cout << "frame=" << i << " rtsp_timeout\n";
        break;
      }
      const auto tensors = simaai::neat::tensors_from_sample(*sample, true);
      const auto& shape = tensors.front().shape;
      std::cout << "frame=" << i << " shape=[";
      for (std::size_t d = 0; d < shape.size(); ++d) {
        std::cout << shape[d] << (d + 1 < shape.size() ? ", " : "");
      }
      std::cout << "]\n";
    }

    return 0;
  } catch (const std::exception& e) {
    std::cerr << "[FAIL] " << e.what() << "\n";
    return 1;
  }
}

Source

• C++
• Python
• README