When 2 Seconds Matter: Building Real-Time AI Captions for Live Events

Live events need captions and translations that feel instantaneous, but building real-time AI streaming is very different from processing uploaded audio.

In this talk, I’ll share what I learned building VoxBento, an open source real-time multilingual streaming infrastructure using FastAPI, WebRTC, MediaMTX, and pluggable transcription providers.

I’ll walk through the architecture from live audio ingestion to streaming transcription and caption delivery, and the engineering challenges we encountered along the way. In particular, I’ll cover why we moved from HLS to WHEP to reduce playback handoff latency, how we integrated both cloud and local transcription services, and why naïve chunk-based Whisper inference can cause context and caption consistency problems.

I’ll also discuss practical approaches to handling overloaded transcription pipelines, including bounded queues, dropping stale audio, voice activity detection, and caption aggregation.

The goal is to give attendees a practical look at the tradeoffs involved in building low-latency, extensible AI infrastructure in open source, along with lessons that can be applied to other real-time AI applications.

Most first-time contributors never make it past setup. In this workshop, we set up together and then spend the session on real code in Eventyay, FOSSASIA's open source event platform. Conferences use it for ticketing, calls for speakers, schedules and online events.

You'll clone the repository and bring up the full stack with Docker Compose. This runs Django, PostgreSQL, Redis and Celery on your laptop, with sample data and live reload, so your changes show up instantly.

Next, you'll learn to find your way around a real Django + Vue codebase. We'll look at how the containers fit together, where the code behind each page lives, and how to read the logs. Then you'll pick a small issue prepared for this workshop, such as a template, text or UI fix. You'll create a branch, test your change and are ready for your first pull requests.

You'll leave with a working development setup, and a workflow you can reuse. Ready for more?
Bring: A laptop with Docker and Git installed, plus a GitHub account.

Most first-time contributors never make it past setup. In this workshop, we set up together and then spend the session on real code in Eventyay, FOSSASIA's open source event platform. Conferences use it for ticketing, calls for speakers, schedules and online events.

You'll clone the repository and bring up the full stack with Docker Compose. This runs Django, PostgreSQL, Redis and Celery on your laptop, with sample data and live reload, so your changes show up instantly.

Next, you'll learn to find your way around a real Django + Vue codebase. We'll look at how the containers fit together, where the code behind each page lives, and how to read the logs. Then you'll pick a small issue prepared for this workshop, such as a template, text or UI fix. You'll create a branch, test your change and are ready for your first pull requests.

You'll leave with a working development setup, and a workflow you can reuse. Ready for more?
Bring: A laptop with Docker and Git installed, plus a GitHub account.

In this hands-on workshop, explore how to build powerful developer workflows using AGENTS.md, reusable skills, MCP servers, memory, and multi-agent systems. Through live demos, learn how to leverage these capabilities for open-source contributions and Web3 development, building workflows that are reusable, reliable, and safe.

Bring your questions to the people who build and maintain FOSSASIA's open source projects. In this open session, FOSSASIA maintainers and GSoC mentors answer anything from the audience: how to make your first contribution, what maintainers look for in a pull request, how projects like Eventyay, Voxbento, Visdom, PSLab, Badge Magic are run day to day, and how you can get involved in FOSSASIA CodeHeat, Google Summer of Code and other programs.