Bulk CV Import

The Bulk CV Import feature allows companies to import multiple CVs at once from a ZIP file. The system extracts each CV using the existing CV extraction pipeline and saves each one as a draft — a parsed CV (CvProfile + structured tables + skills) with no User or Talent yet — that lands in the Draft CV Inbox for a recruiter to review and promote. All processing happens in the background with real-time progress streaming via GraphQL subscriptions.

Draft-first intake

Bulk import used to create a User + Talent + CV for every CV and add the talent straight to the pool. It now goes through the shared draft-first path (save_intake_draft() in app/services/bulk_cv/talent_creator.py), the same model used by inbound email and the AI agent. Two guarantees follow:

Imported CVs land as DRAFT, not ONBOARDED — a recruiter reviews and promotes them from the Draft CV Inbox.
A bulk import never overwrites an active user's CV — if a CV's email matches an already-active platform user, the import does not attach to them or replace their CV; the recruiter is routed to send a connection request instead.

Architecture

Per-CV Processing Flow

Supported File Types

Extension	Format	Notes
`.pdf`	PDF documents	Primary format, processed via OCR
`.doc`	Legacy Word	Processed via python-docx
`.docx`	Modern Word	Processed via python-docx
`.txt`	Plain text	Direct text extraction

Skipped Files

The system automatically skips:

__MACOSX/ folders and ._ prefixed files (macOS ZIP artifacts)
Hidden files (starting with .) in any directory level
Files with unsupported extensions (images, spreadsheets, etc.)
Directory entries

GraphQL Mutation

The mutation returns BulkCvImportProcessing directly (not a union type). Validation errors are returned as standard GraphQL errors.

mutation CreateBulkCvImport($input: BulkCvImportInput!) {
  createBulkCvImport(input: $input) {
    id
    status
    message
    progress
    totalFiles
    processedFiles
    successCount
    failedCount
    skippedCount
  }
}

Input

{
  "input": {
    "fileUrl": "bulk-cv-test/my-upload.zip",
    "companyId": "company-uuid",
    "userId": "user-uuid"
  }
}

Field	Type	Description
`fileUrl`	String	S3 key or S3 URL of the uploaded ZIP file
`companyId`	String	Company UUID — talents are added to this company's pool
`userId`	String	User UUID — the user who initiated the import

Validation

The mutation performs synchronous validation before returning:

URL validation — fileUrl must be an S3 path (not an arbitrary HTTP URL)
ZIP validation — Downloads the file and checks for valid ZIP magic bytes (PK\x03\x04)
Size validation — Compressed size must be < 500 MB (configurable)
ZIP bomb detection — Total uncompressed size must be < 2 GB
File count — Counts supported CV files; rejects if 0 or > 500
Creates DB record — Inserts BulkCvImport row with status=PENDING

Error Responses

Condition	Error Message
Non-S3 URL	`file_url must be an S3 path or storage URL`
Not a ZIP file	`File is not a valid ZIP archive`
ZIP too large	`ZIP file too large (X bytes, max Y bytes)`
ZIP bomb detected	`ZIP uncompressed size too large`
No CV files	`No supported CV files found in ZIP (supported: .pdf, .doc, .docx, .txt)`
Too many files	`Too many CV files (X, max 500)`

GraphQL Subscription

subscription BulkCvImport($importId: String!) {
  bulkCvImport(importId: $importId) {
    ... on BulkCvImportProcessing {
      id
      status
      message
      progress
      totalFiles
      processedFiles
      successCount
      failedCount
      skippedCount
    }
    ... on BulkCvImportSuccess {
      id
      status
      totalFiles
      successCount
      failedCount
      skippedCount
      processingTimeMs
      results {
        filename
        status
        userId
        talentId
        cvId
        email
        errorMessage
      }
    }
    ... on BulkCvImportFailure {
      id
      status
      error {
        code
        message
        recoveryHint
      }
    }
  }
}

Update Types

Type	When	Contains
`BulkCvImportProcessing`	Every 2s while processing	Progress percentage, file counts, success/fail/skip counters
`BulkCvImportSuccess`	Terminal: all or some succeeded	Per-file results with userId, talentId, cvId, email
`BulkCvImportFailure`	Terminal: all failed or not found	Error code, message, recovery hint

Progress Streaming

The subscription polls the database every 2 seconds. Updates are only emitted when processedFiles changes or a terminal status is reached.

Background Worker

File: app/services/bulk_cv/worker.py

The BulkCvImportWorker follows the same pattern as other background workers (Document Worker, Inbound Email Worker):

Setting	Default	Env Variable	Description
Poll interval	30s	`BULK_CV_POLL_INTERVAL`	How often to check for pending jobs
Max concurrent CVs	3	`BULK_CV_MAX_CONCURRENT`	Semaphore limit for parallel CV processing
Max retries	3	`BULK_CV_MAX_RETRIES`	Max attempts before giving up on a job
Max files per ZIP	500	`BULK_CV_MAX_FILES`	Maximum CV files allowed in a single ZIP
Max ZIP size	500 MB	`BULK_CV_MAX_ZIP_SIZE_BYTES`	Maximum compressed ZIP file size
Enabled	true	`BULK_CV_IMPORT_ENABLED`	Feature toggle

Job Claiming

Uses SELECT FOR UPDATE SKIP LOCKED to safely claim jobs in multi-worker scenarios:

Only claims jobs with status IN ('PENDING', 'EXTRACTING') and attemptCount < maxRetries
Stale locks (older than 10 minutes) are automatically reclaimed
Each claim increments attemptCount

Status Lifecycle

Circuit Breaker

The worker uses a circuit breaker to prevent cascading failures:

Failure threshold: 5 consecutive failures
Reset timeout: 300 seconds (5 minutes)
Call timeout: 600 seconds (10 minutes)
When OPEN, the worker backs off for 60 seconds before retrying

What each CV produces

Each successfully processed CV is saved as a draft via save_intake_draft() — a parsed CV plus a container row, with no identity attached until a recruiter promotes it:

There is no dedup-and-attach step against existing users any more. Because a draft carries no User/Talent, the import cannot overwrite anyone's CV. The email-based duplicate and active-user checks now run at promote time, in the Draft CV Inbox preflight — the same checks the "Add Talent" flow uses.

Created Records

For each successfully processed CV:

Record	Details
CV (draft)	A `CV` row with `userId`/`talentId` null, storing the S3 file URL and extracted plaintext
Structured CV tables	The 11 structured tables (`CvProfile`, `CvWorkExperience`, …) written and linked to the CV — best-effort enrichment
Skills	Extracted and normalized skills saved to CVSkill table — proficiency synonyms (`BEGINNER`, `MASTER`, …) are mapped to the DB enum before insert; per-row failures are isolated. See Proficiency normalization on the CV-extraction page.
InboundEmail (`CV_CREATED`)	The synthetic container row (`imapFolder="INTAKE"`, `imapUid` NULL) that surfaces the draft in the Draft CV Inbox and makes it promotable

No User or Talent is created at import time. Both are created only when a recruiter promotes the draft. If a promoted CV's email matches an active platform user, the promote flow sends that user a connection request rather than overwriting their account.

note

If a bulk import lands with cv.text populated but its CVSkill / structured tables empty, that's the exact pattern the CV skills backfill script remediates. The script re-runs the LLM extraction on the cached text without re-fetching from S3.

Database Schema

-- Enum
CREATE TYPE "BulkCvImportStatus" AS ENUM (
  'PENDING', 'EXTRACTING', 'PROCESSING',
  'COMPLETED', 'PARTIALLY_COMPLETED', 'FAILED'
);

-- Table
CREATE TABLE "BulkCvImport" (
  "id"              UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  "companyId"       UUID NOT NULL REFERENCES "Company"("id"),
  "userId"          UUID NOT NULL REFERENCES "User"("id"),
  "status"          "BulkCvImportStatus" DEFAULT 'PENDING',
  "zipFileUrl"      TEXT NOT NULL,
  "totalFiles"      INT DEFAULT 0,
  "processedFiles"  INT DEFAULT 0,
  "successCount"    INT DEFAULT 0,
  "failedCount"     INT DEFAULT 0,
  "skippedCount"    INT DEFAULT 0,
  "results"         JSONB,
  "errorMessage"    TEXT,
  "processingTimeMs" INT,
  "attemptCount"    INT DEFAULT 0,
  "lockedAt"        TIMESTAMP,
  "lockedBy"        TEXT,
  "createdAt"       TIMESTAMP DEFAULT NOW(),
  "updatedAt"       TIMESTAMP DEFAULT NOW()
);

The results JSONB column stores per-file results:

[
  {
    "filename": "resume_john.pdf",
    "status": "SUCCESS",
    "user_id": null,
    "talent_id": null,
    "cv_id": "uuid",
    "email": "john@example.com",
    "error_message": null
  },
  {
    "filename": "broken_cv.pdf",
    "status": "FAILED",
    "user_id": null,
    "talent_id": null,
    "cv_id": null,
    "email": null,
    "error_message": "No email found in CV"
  }
]

File Structure

File	Purpose
`app/graphql/types/bulk_cv_import.py`	GraphQL types (Input, Processing, Success, Failure, FileResult)
`app/graphql/mutations/bulk_cv_import.py`	Mutation: validates ZIP, creates DB record
`app/graphql/subscriptions/bulk_cv_import.py`	Subscription: polls DB, streams progress
`app/services/bulk_cv/__init__.py`	Module init, exports worker singleton
`app/services/bulk_cv/worker.py`	Background worker (polling, processing loop)
`app/services/bulk_cv/repository.py`	AsyncPG database queries
`app/services/bulk_cv/zip_utils.py`	ZIP validation and extraction utilities
`app/services/bulk_cv/talent_creator.py`	Shared draft-intake logic (`save_intake_draft`, `save_draft_results`) used by bulk import, inbound email, and the AI agent
`tests/unit/services/test_bulk_cv_worker.py`	145 unit tests
`tests/e2e/test_bulk_cv_import_e2e.py`	11 E2E tests against dev environment

Billing Integration

The worker calls BillingGate.pre_check(company_id, "CV_EXTRACTION", user_id=user_id) before each CV extraction. If billing blocks the operation (insufficient credits, plan limit exceeded), the CV is marked as SKIPPED (not FAILED) and processing continues with the remaining files.

Logging

Every step of the import pipeline is logged with structured extra fields for monitoring:

Event Type	Description
`bulk_cv_record_created`	DB record created by mutation
`bulk_cv_job_claimed`	Worker claimed a pending job
`bulk_cv_downloading_zip`	Starting ZIP download
`bulk_cv_zip_downloaded`	ZIP downloaded, extracting files
`bulk_cv_processing_start`	File extraction complete, starting CV processing
`bulk_cv_file_start`	Starting individual CV processing
`bulk_cv_file_done`	Individual CV processing complete
`bulk_cv_file_success`	CV processed successfully (includes user/talent/cv IDs)
`bulk_cv_billing_blocked`	Billing pre-check blocked CV extraction
`bulk_cv_extraction_failed`	CV extraction pipeline failed
`bulk_cv_no_email`	No email found in parsed CV data
`bulk_cv_job_completed`	Job finished with final status and counts
`bulk_cv_job_failed`	Job failed with error

Testing

Unit Tests (145 tests)

pytest tests/unit/services/test_bulk_cv_worker.py -v -m unit

Covers: ZIP validation, file filtering (extensions, __MACOSX, hidden files, nested folders, unicode filenames, special characters), skills normalization, qualifications builder, dedup logic, progress tracking, error handling (S3 errors, billing blocks, extraction failures, missing emails), repository operations, subscription behavior, result dict key compatibility.

E2E Tests (11 tests)

pytest tests/e2e/test_bulk_cv_import_e2e.py -v -m e2e

Requires AWS SSO login and runs against ai-dev.aiqlick.com:

Test	Scenario
`test_full_import_flow_with_pdfs`	Full pipeline: upload 2 PDFs → mutation → subscription → COMPLETED
`test_dedup_reuses_existing_users`	Re-import same ZIP, verify user IDs match
`test_non_zip_file_rejected`	PDF file (not ZIP) rejected with error
`test_empty_zip_rejected`	ZIP with no CV files rejected
`test_subscription_not_found`	Non-existent import ID returns FAILED
`test_single_pdf_import`	ZIP with single PDF processes correctly
`test_zip_with_nested_folders`	CVs in deeply nested folders are found
`test_zip_with_macosx_artifacts`	macOS ZIP metadata is filtered out
`test_zip_with_only_unsupported_files`	ZIP with only images/spreadsheets rejected
`test_subscription_streams_incremental_progress`	Progress updates are monotonically increasing
`test_mutation_returns_correct_total_files`	File count excludes non-CV files

Limitations

Text files (.txt) may fail CV extraction if the LLM cannot parse structured data from plain text
No password-protected ZIPs — encrypted ZIP files will fail extraction
Email required — CVs without an email address in the parsed data are marked as FAILED
Drafts, not talents — a successful import creates a draft CV, not a pool talent; the talent is created only when a recruiter promotes the draft from the Draft CV Inbox
No invitation emails — imported drafts carry no user account, so no emails are sent at import time

Architecture​

Per-CV Processing Flow​

Supported File Types​

Skipped Files​

GraphQL Mutation​

Input​

Validation​

Error Responses​

GraphQL Subscription​

Update Types​

Progress Streaming​

Background Worker​

Job Claiming​

Status Lifecycle​

Circuit Breaker​

What each CV produces​

Created Records​

Database Schema​

File Structure​

Billing Integration​

Logging​

Testing​

Unit Tests (145 tests)​

E2E Tests (11 tests)​

Limitations​