Sharing a project I help maintain — Flowcat, an Apache-2.0 native-Rust runtime for real-time voice AI agents (phone + WebRTC). It runs the whole voice pipeline (STT → LLM → TTS, or a single speech-to-speech model) as one self-contained binary you host yourself — in your own VPC or fully air-gapped. No hosted control plane, bring your own keys, and a call’s audio never leaves your infra.

It’s a clean-room counterpart to pipecat’s architecture, but native Rust:

  • each pipeline stage is its own tokio task behind a bounded mpsc channel (natural backpressure)
  • the hot audio frame is an Arc<AudioFrame>, so each hop moves a pointer, not a PCM copy
  • no GIL, so one process uses all cores and holds a flat tail under load (measured: flat ~0.6ms p99 framework overhead from 10 to 2,000 concurrent calls on one box)
  • native SIP/RTP, so a single binary can terminate phone calls with no FreeSWITCH

The linked writeup has the full Rust design + a reproducible benchmark vs pipecat (with the honest caveat that it’s framework overhead, not end-to-end voice latency — that’s provider-bound).

Repo: https://github.com/AreevAI/flowcat

Apache-2.0, pre-1.0, built in the open. Disclosure: I help maintain it — happy to talk architecture or the SIP stack.

  • TehPers@beehaw.org
    link
    fedilink
    English
    arrow-up
    10
    ·
    21 days ago

    runtime for real-time voice AI agents (phone + WebRTC)

    Oh boy, my favorite…

    It’s a clean-room counterpart to pipecat’s architecture, but native Rust

    If by “clean-room” you mean “I had an AI extract the API and another reimplement it” solely with the intent of changing the license (which was already BSD-2 anyway), then you give credibility to the “they rewrote in Rust to change the license” crowd, which might actually bother me more than anything else. Plus, there’s no guarantee that the LLM didn’t know anything of the original implementation (which is a public GitHub repo… no chance in hell the LLM wasn’t trained on it), and I find it unlikely that you didn’t reference Pipecat if you did this by hand (something tells me that’s not the case).

    not end-to-end voice latency — that’s provider-bound

    Genuine question, though somewhat off-topic: I have a method of doing this on my phone keyboard and, via a certain key combination, my desktop, but I’m curious about your method. What method do you use to type an emdash?