Transcription System

The transcription system converts live meeting audio into text in real time, stores structured transcription records, and triggers AI-generated meeting insights. The pipeline spans three components: the custom Jigasi fork, the background-tasks service, and the PostgreSQL database.

Pipeline Overview

Jigasi (Audio Capture)

Jigasi is a custom fork of the Jitsi Jigasi project, modified to stream audio over WebSocket instead of using Google Cloud Speech.

Behavior

• Jigasi joins each conference as a hidden participant — it does not appear in the participant list and does not send any media back to the conference.
• It establishes a Colibri WebSocket connection with JVB so JVB can forward audio via EndpointMessageTransport.
• Audio is encoded as 16-bit signed PCM at 16 kHz mono and streamed as binary WebSocket frames to the background-tasks endpoint.
• JSON control messages are interleaved with audio frames for session management (start, stop, participant mapping).

Colibri WebSocket (JVB Audio Forwarding)

Before Jigasi can receive audio, it must establish a Colibri WebSocket connection with JVB. The URL is extracted from Jingle session-initiate/transport-info IQs via regex. The extraction handles two cases:

1. xmlns directly on the <web-socket> element (standard)
2. xmlns inherited from parent <transport> element (Smack re-serialization)

If the URL is not available when the call starts, a retry mechanism (5 attempts, 2s apart) schedules reconnection attempts. JVB's first-transfer-timeout is extended to 120 seconds (from 15s default) as a safety net.

Connection

Jigasi connects to:

wss://ai.aiqlick.com/transcription/ws

The WebSocket connection is established when the first moderator joins a conference and remains open for the duration of the meeting.

Configuration

In jitsi-deploy/.env:

ENABLE_TRANSCRIPTIONS=1
JIGASI_MODE=transcriber

These variables configure Prosody to invite Jigasi into conferences and set Jigasi to operate in transcription mode rather than SIP gateway mode.

Background-Tasks (Processing)

The background-tasks service receives audio streams and processes them into text using AWS Transcribe (and optionally Whisper for specific language support).

WebSocket Endpoint

/transcription/ws

The endpoint accepts binary PCM frames and JSON control messages on the same WebSocket connection. Processing steps:

1. Receive audio -- buffered PCM frames arrive continuously during the meeting.
2. Stream to Transcribe -- audio is forwarded to AWS Transcribe's StartStreamTranscription API in the eu-west-1 region for real-time speech-to-text.
3. Store segments -- each transcribed segment is written to the Transcription table with the speaker identifier, timestamp, text content, and confidence score.
4. Return captions -- transcribed text is sent back through the WebSocket for real-time caption display in the meeting UI.

Performance

• Audio format: 16 kHz, 16-bit signed, mono PCM.
• Caption latency: approximately 300 milliseconds from speech to displayed caption.
• The system processes audio in near real time, with Transcribe returning partial results that are refined as more context becomes available.

Auto-Start

Transcription starts automatically without any user action. The flow:

1. A moderator joins a conference room.
2. Prosody detects the moderator presence and invites Jigasi to the room (configured via ENABLE_TRANSCRIPTIONS=1).
3. Jigasi joins as a hidden participant and opens the WebSocket connection.
4. Audio capture and transcription begin immediately.

No manual "start recording" button is required. Transcription runs for the full duration of the meeting.

Transcription Records

Each transcription segment is stored in the transcriptions table. This table is the one exception to the workspace's Prisma convention — it is lowercase / snake_case rather than PascalCase / camelCase, and it carries no user_id column (only speaker_id, a Jitsi display identifier, not a User.id FK):

Column	Type	Description
id	uuid	Primary key
meeting_id	varchar(255)	Room name, correlates with Meeting.roomName
speaker_id	varchar(255)	Speaker identifier from the XMPP session (Jitsi display / JID)
speaker_name	varchar(255)	Display name, nullable
text	text	Transcribed text content
language	varchar(10)	Detected or configured language code (default en)
confidence	double	Transcription confidence score from the speech engine
timestamp	timestamp	When the speech occurred in the meeting timeline
created_at	timestamp	When the row was inserted

warning

asyncpg queries targeting this table must use the lowercase / snake_case names — SELECT text FROM transcriptions WHERE meeting_id = $1, not FROM "Transcription" WHERE "meetingId" = $1. Treating it like a normal Prisma table raises UndefinedTableError / UndefinedColumnError which, if swallowed by a broad except, silently degrades to a "no rows" result. This was the secondary root cause behind SUP-00058.

note

Because there is no user_id column, you cannot derive a valid Meeting.organizerId from transcription rows alone. Any recovery path that needs an organizer must resolve one from InterviewBookingAttendee (role = 'INTERVIEWER' + userId IS NOT NULL) or from a pre-existing Meeting.organizerId — never from transcriptions.

Meeting Insights

After transcription data accumulates (or when a meeting ends), the system generates AI-powered meeting insights using Claude Haiku 4.5.

Trigger

Prosody sends webhooks to the background-tasks endpoint at:

/api/events/room/created
/api/events/room/destroyed

Events such as conference creation, participant join/leave, and conference destruction are forwarded. The conference-destroyed event triggers automatic meeting insight generation.

Insight Content

Each MeetingInsight record contains structured analysis:

Section	Description
Summary	Concise overview of the meeting discussion
Action Items	Extracted tasks with assignees where identifiable
Key Decisions	Important decisions made during the meeting
Sentiment Analysis	Overall tone and participant engagement assessment

Versioning

Meeting insights support versioning for live updates. During an ongoing meeting, insights can be regenerated as new transcription data arrives, creating a new version each time. This allows participants to view progressively refined insights without losing previous versions.

Generation Flow

1. The conference-destroyed Prosody webhook arrives at /api/events/room/destroyed.
2. Background-tasks retrieves all Transcription records for the meeting's roomName.
3. The transcription text is sent to Claude Haiku 4.5 with a structured prompt requesting summary, action items, decisions, and sentiment.
4. The response is parsed and stored as a MeetingInsight record linked to the meeting.
5. If the meeting is still active (live insight generation), a new version is created alongside existing versions.

AWS Services

The transcription pipeline uses AWS services in the eu-west-1 (Ireland) region:

Service	Purpose
AWS Transcribe	Real-time speech-to-text via StartStreamTranscription
AWS Bedrock	Claude Haiku 4.5 for meeting insight generation (eu-north-1)

The aiqlick-ecs-bg-task-role-{dev,prod} ECS task role (inline BgTaskPolicy) grants the background-tasks tasks the Transcribe + Bedrock permissions.