Bedrock & AI Services
AIQLick uses four AWS AI services for its machine learning workloads: Amazon Bedrock for LLM inference and embeddings, Amazon Rekognition for face detection, Amazon Transcribe for speech-to-text (meetings and voice chat), and Amazon Polly for text-to-speech (voice chat). All AI service calls originate from the background-tasks service (FastAPI) running on the bg-tasks-{dev,prod} ECS Fargate services in eu-north-1. The runtime IAM permissions are carried by the aiqlick-ecs-bg-task-role-{dev,prod} task roles — see ECS Fargate — IAM.
Amazon Bedrock
Bedrock is the primary AI service, deployed in eu-north-1 (Stockholm). It provides access to foundation models via the InvokeModel and InvokeModelWithResponseStream APIs without managing any model infrastructure.
Models in Use
| Model | Model ID | Purpose | Dimensions |
|---|---|---|---|
| Claude (Anthropic) | Various anthropic.claude-* | LLM inference (text generation) | -- |
| Titan Embed v2 | amazon.titan-embed-text-v2:0 | Document embeddings for RAG | 1024 |
Claude Models (Anthropic)
Claude models power the core AI features across the platform:
| Feature | Description | API |
|---|---|---|
| CV Extraction | Parse uploaded CVs into structured data (11 structured CV tables) | InvokeModelWithResponseStream |
| Job Parsing | Extract structured job requirements from free-text job descriptions | InvokeModelWithResponseStream |
| Matching Engine | Score compatibility between candidates and jobs with explanations | InvokeModelWithResponseStream |
| Meeting Insights | Generate AI summaries and analysis from meeting transcriptions | InvokeModelWithResponseStream |
| Text Rewrite | Rewrite and improve text (bios, descriptions, summaries) | InvokeModelWithResponseStream |
| Agent Chat | Multi-turn conversational AI assistant with tool calling and RAG | InvokeModelWithResponseStream |
All features use streaming inference (InvokeModelWithResponseStream) to deliver progressive results through GraphQL subscriptions. The frontend receives incremental updates as the model generates output.
Prompt Caching
The high-volume Bedrock Converse calls insert a cachePoint marker between the static prefix (system framing + JSON schema + tool definitions) and the variable suffix (user document text, conversation history). On warm reads inside the 5-minute TTL, Bedrock charges the cached prefix at 0.1× the input rate instead of fully re-prefilling it.
What's cached
| Call site | File | Cached prefix |
|---|---|---|
parse_with_schema | app/services/pipeline/llm_services/bedrock_services.py:378 | Framing + JSON schema (CV extraction, job parsing, skill extraction) |
parse_with_reasoning | bedrock_services.py:483 | Reasoning preamble + framing + schema |
| Agent chat (system prompt) | app/services/agent/conversation_service.py:1883 | Agent persona + RAG context wrapper |
| Agent chat (tool definitions) | conversation_service.py:1900 | Full 12-tool definition block |
The schema-based extractors share a (static_prefix, variable_suffix) split exposed by prompt_builder.py — every prompt builder returns a tuple so the cache point lands deterministically between framing and user content.
What's NOT cached (and why)
Bedrock silently ignores cachePoint markers when the static prefix is below the per-model minimum. Adding a marker on a sub-threshold prompt is a no-op at best and a maintenance liability at worst, so we leave them off where the math doesn't work:
| Call site | Static prefix | Reason |
|---|---|---|
Meeting insight (BedrockInsightService.generate_insight) | ~300 tokens | Below Haiku 4.5's ~4 k token cacheable-prefix minimum — Bedrock would ignore the marker. See the inline comment at bedrock_services.py:1252. Re-evaluate if prompts grow or insight moves to Sonnet. |
OCR (BedrockOcrService) | ~50–80 tokens (single + batch) | Same threshold problem; OCR prompts are intentionally minimal. |
Text rewrite (TextRewriteService) | ~50 tokens | System prompt is one sentence; both system and user content vary per request. |
LLM scorer (llm_scorer.py) | 0 — fully dynamic | The whole prompt is generated per call; there's no static prefix to cache. |
| Titan Embed v2 (embeddings) | n/a | InvokeModel API doesn't support prompt caching — caching is Converse-only. |
| Nova Sonic (voice) | n/a | Bidirectional streaming API — caching not applicable. |
Verifying caching is engaging
A cold/warm test inside the deployed container is the canonical check:
# Inside background-tasks container, with PYTHONPATH=/app
from app.services.pipeline.llm_services.bedrock_services import BedrockParserService
svc = BedrockParserService()
await svc.parse_with_schema(schema, sample_cv_1)
print(svc.last_usage) # cold: cache_write > 0, cache_read = 0
await svc.parse_with_schema(schema, sample_cv_2)
print(svc.last_usage) # warm: cache_write = 0, cache_read > 0
Same schema → identical static prefix → second call hits the cache. Last verified on prod 2026-04-27: cache_write=1133 cold, cache_read=1133 warm.
A repeatable benchmark lives at tests/benchmark/prompt_caching_benchmark.py in the background-tasks repo; it covers the production CV-extraction, job-parsing, and agent-chat flows with real schemas.
Titan Embed v2
Used for the RAG (Retrieval-Augmented Generation) pipeline in the AI Agent system:
- Model ID:
amazon.titan-embed-text-v2:0 - Output dimensions: 1024
- Storage: pgvector extension in PostgreSQL (same RDS instance)
- Search: Hybrid search combining 0.7x vector similarity + 0.3x full-text score
The document worker in background-tasks polls for new documents every 30 seconds, chunks them, generates embeddings via Titan, and indexes them in pgvector for retrieval during agent conversations.
Bedrock Invocation Flow
Amazon Rekognition
Rekognition is used for face detection during the CV processing pipeline.
| Property | Value |
|---|---|
| Region | eu-west-1 (Ireland) |
| API | DetectFaces |
| Purpose | Extract face photos from uploaded CV documents |
| Called by | background-tasks (CV extraction pipeline) |
Amazon Rekognition is not available in eu-north-1 (Stockholm). The closest European region with Rekognition support is eu-west-1 (Ireland). The cross-region call adds minimal latency since face detection is a one-time operation during CV processing.
Face Detection Pipeline
- User uploads a CV document (PDF or image) to S3.
- Background-tasks extracts images from the document.
- Each image is sent to Rekognition
DetectFacesin eu-west-1. - Detected face bounding boxes are used to crop and store profile photos.
- The best face image is linked to the candidate's profile.
Amazon Transcribe
Transcribe provides real-time speech-to-text for two features: meeting transcription (via Jigasi) and voice chat STT (via background-tasks voice service).
| Property | Value |
|---|---|
| Region | eu-west-1 (Ireland) |
| API | StartStreamTranscription |
| Purpose | Meeting transcription + voice chat STT |
| Called by | Jigasi (meetings), background-tasks (voice chat) |
Meeting Transcription
- Jigasi joins the Jitsi meeting room as a hidden participant and receives the audio stream.
- Audio is streamed to Amazon Transcribe via
StartStreamTranscriptionfor real-time processing. - Transcription segments are sent to background-tasks over WebSocket (
wss://ai.aiqlick.com/transcription/ws). - Background-tasks stores each segment as a
Transcriptionrecord in PostgreSQL. - When the meeting ends, a Prosody webhook triggers meeting insight generation using the full transcript.
Voice Chat STT
The voice service in background-tasks also uses Transcribe for speech-to-text during voice conversations with AI agents. Audio is streamed from the browser over WebSocket, buffered, and sent to Transcribe for transcription. See Voice Service for the full protocol.
Like Rekognition, Amazon Transcribe Streaming is not available in eu-north-1. The service runs in eu-west-1 (Ireland) with the Transcribe StartStreamTranscription permission carried by the BgTaskPolicy inline policy on aiqlick-ecs-bg-task-role-{dev,prod}.
Amazon Polly
Polly provides neural text-to-speech for voice chat responses from AI agents.
| Property | Value |
|---|---|
| Region | eu-west-1 (Ireland) |
| API | SynthesizeSpeech |
| Engine | neural |
| Purpose | Convert AI agent text responses to spoken audio |
| Called by | background-tasks (voice service) |
| Output format | PCM (16kHz, 16-bit, mono) |
Voice Personas
Two voice personas are mapped to language-specific Polly neural voices:
| Voice ID | Gender | Default |
|---|---|---|
tiffany | Female | Yes |
matthew | Male | No |
Each (voice_id, language) pair maps to a specific Polly VoiceId. For example, tiffany + en-US = Joanna, tiffany + de-DE = Vicki. See Voice Service for the complete mapping table.
Supported Languages
10 languages supported: en-US, en-GB, en-IN, en-AU, es-ES, de-DE, fr-FR, it-IT, pt-BR, hi-IN.
Polly neural voices are not available in eu-north-1. The service runs in eu-west-1 (Ireland) alongside Transcribe and Rekognition.
IAM Permissions
All AI service access is granted through the BgTaskPolicy inline policy attached to the aiqlick-ecs-bg-task-role-{dev,prod} ECS task roles. No static API keys are used.
| Service | Actions | Resource |
|---|---|---|
| Bedrock | bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream | * (all models in eu-north-1) |
| Rekognition | rekognition:DetectFaces | * (eu-west-1) |
| Transcribe | transcribe:StartStreamTranscription + full access via AmazonTranscribeFullAccess | * (eu-west-1) |
| Polly | polly:SynthesizeSpeech, polly:DescribeVoices | * (eu-west-1) |
See IAM Roles & Policies for the complete policy details.
Credit System Integration
Every AI operation is metered through the BillingGate in background-tasks. Credits are consumed per operation based on token usage and model costs.
Billing Flow
Operation Types and Tracking
Each AI call is recorded in the AIOperationLog table with:
| Field | Description |
|---|---|
operationType | One of: CV_EXTRACTION, JOB_PARSING, MEETING_INSIGHT, MATCHING, TEXT_REWRITE, AGENT_CHAT |
modelId | The Bedrock model ID used |
inputTokens | Number of input tokens sent to the model |
outputTokens | Number of output tokens generated |
creditCost | Credits consumed (based on CreditCostConfig) |
userId / companyId | Who was billed |
Credit Routing
- Employer mode (user has
selectedCompanyId): Credits deducted from the company'sCreditBalance. - Job seeker mode (no company context): Credits deducted from the user's personal
CreditBalance.
Pricing per model is configured by admins in the CreditCostConfig table. The BILLING_ENABLED environment variable controls gradual rollout of credit enforcement.
Useful CLI Commands
List Available Bedrock Models
aws bedrock list-foundation-models \
--profile Administrator-842697652860 \
--region eu-north-1 \
--query 'modelSummaries[].{Id:modelId,Name:modelName,Provider:providerName}' \
--output table
Check Bedrock Model Availability
aws bedrock get-foundation-model \
--profile Administrator-842697652860 \
--region eu-north-1 \
--model-identifier anthropic.claude-sonnet-4-5-20250929-v1:0 \
--output json
List Bedrock Model Access
aws bedrock list-foundation-model-agreement-offers \
--profile Administrator-842697652860 \
--region eu-north-1 \
--output table
Check Rekognition (eu-west-1)
# Verify Rekognition is accessible
aws rekognition describe-collection \
--profile Administrator-842697652860 \
--region eu-west-1 \
--collection-id test 2>&1 | head -5
Verify Transcribe Access
aws transcribe list-transcription-jobs \
--profile Administrator-842697652860 \
--region eu-west-1 \
--max-results 5 \
--output table
Region Summary
| Service | Region | Reason |
|---|---|---|
| Bedrock (Claude, Titan) | eu-north-1 | Primary region, available in Stockholm |
| Rekognition | eu-west-1 | Not available in eu-north-1 |
| Transcribe | eu-west-1 | Streaming not available in eu-north-1 |
| Polly (Neural) | eu-west-1 | Neural voices not available in eu-north-1 |