Batch-transcribe thousands of files

You have a backlog — thousands of recordings sitting in a bucket — and you want them all transcribed. The naive approach, calling transcribe() in a loop, blocks on each file and holds a live connection open for the entire job. That doesn't scale. This tutorial shows the pattern that does: submit every file (with bounded concurrency), persist the returned job ids, then collect results separately by polling or via webhooks.

Why submit + collect beats N live connections

transcribe() is a convenience: it uploads, waits, and returns the transcript in one call — perfect for one file, wasteful for thousands. Each in-flight call keeps a long-lived connection open for the full duration of the transcription. Run a thousand of those at once and you're holding a thousand sockets, any one of which can drop and lose the result you were waiting on.

submit() uploads the audio, enqueues the job, and returns a job_id immediately — no connection held while Speech Revolutions works. Once you have the ids, the work is durable: you can collect the results minutes or hours later, survive a restart, and retry a single file without redoing the batch. The upload is the only part that needs concurrency control; the transcription itself runs server-side.

The two phases

Submit — upload + enqueue every file, bounded by a semaphore so you don't open thousands of uploads at once. Save the returned job_ids somewhere durable. Collect — fetch each transcript once its job completes, either by polling get_job_status / get_transcript or by having Speech Revolutions POST a callback_url when each job finishes.

Step 1 — submit everything with bounded concurrency

Use the async client and gather all submissions behind a semaphore. The semaphore caps how many uploads run at once (tune it to your bandwidth); every task returns a (path, job_id) pair you persist before moving on. If a submission fails, record the error instead of the id so you can retry just that file.

import asyncio
import json

from speechrevolutions import AsyncSpeechRevolutions

CONCURRENCY = 16  # max simultaneous uploads — tune to your bandwidth


async def submit_all(paths: list[str]) -> dict[str, str]:
    """Submit every file; return {path: job_id}. Failures are recorded, not raised."""
    sem = asyncio.Semaphore(CONCURRENCY)
    job_ids: dict[str, str] = {}

    async with AsyncSpeechRevolutions() as client:
        async def submit_one(path: str) -> None:
            async with sem:  # only CONCURRENCY uploads in flight at once
                try:
                    job_id = await client.submit(path, speaker_labels=True)
                    job_ids[path] = job_id
                    print(f"submitted {path} -> {job_id}")
                except Exception as e:  # keep going; retry this file later
                    print(f"FAILED to submit {path}: {e}")

        await asyncio.gather(*(submit_one(p) for p in paths))

    return job_ids


if __name__ == "__main__":
    files = [line.strip() for line in open("files.txt") if line.strip()]
    ids = asyncio.run(submit_all(files))
    # Persist the ids BEFORE collecting — this is your durable checkpoint.
    with open("jobs.json", "w") as f:
        json.dump(ids, f, indent=2)
    print(f"submitted {len(ids)}/{len(files)} files; saved jobs.json")

Persist ids before collecting

Write the job ids to durable storage (a file, a table, a queue) as soon as you have them, and only then start collecting. The ids are your recovery point: if collection crashes, you re-read them and pick up where you left off — you never re-upload. Speech Revolutions regenerates a job's download URL on demand, so results stay fetchable by id long after upload.

Step 2 (option A) — collect by polling

Read back the ids and poll each job until it completes, then fetch the transcript. The async client with the same semaphore keeps the polling pressure bounded. get_job_status returns a status you check for completed / failed; get_transcript downloads and parses the result.

import asyncio
import json

from speechrevolutions import AsyncSpeechRevolutions
from speechrevolutions.exceptions import JobFailedError

CONCURRENCY = 16
POLL_INTERVAL = 5.0  # seconds between status checks


async def collect_one(client: AsyncSpeechRevolutions, path: str, job_id: str, sem):
    async with sem:
        while True:
            status = await client.get_job_status(job_id)
            if status.is_completed:
                result = await client.get_transcript(job_id)  # downloads + parses
                result.save(f"transcripts/{path}")            # -> transcripts/<name>.json
                print(f"done {path}")
                return
            if status.is_failed:
                raise JobFailedError(f"{path} failed", step=status.failed_stage,
                                     reason=status.reason)
            await asyncio.sleep(POLL_INTERVAL)


async def collect_all(job_ids: dict[str, str]) -> None:
    sem = asyncio.Semaphore(CONCURRENCY)
    async with AsyncSpeechRevolutions() as client:
        await asyncio.gather(
            *(collect_one(client, path, jid, sem) for path, jid in job_ids.items()),
            return_exceptions=True,  # one failure doesn't sink the batch
        )


if __name__ == "__main__":
    job_ids = json.load(open("jobs.json"))
    asyncio.run(collect_all(job_ids))

Step 2 (option B) — collect by webhook

At large scale, polling thousands of jobs is a lot of wasted requests. Pass a callback_url when you submit, and Speech Revolutions POSTs a signed notification the moment each job finishes — you fetch the transcript in the handler and never poll at all. Change one line in Step 1:

# In submit_one(), add a callback_url so Speech Revolutions notifies you on completion:
job_id = await client.submit(
    path,
    speaker_labels=True,
    callback_url="https://your-app.example.com/webhooks/speechrevolutions",
)

The webhook body is { job_id, status, download_url?, step?, reason? }, signed with HMAC-SHA256 in the X-SR-Signature: sha256=<hmac> header — always verify it against the raw request bytes before trusting the payload. In the handler, look up which file the job_id belongs to, then call get_transcript(job_id) (or download download_url directly) and save it. See the Python SDK and Jobs API pages for the full signature-verification receiver.

Polling or webhooks?

Polling is simplest and needs no public endpoint — great for a one-off backfill run from a script. Webhooks scale better and cost fewer requests once you have a server that can receive them — the right default for an ongoing pipeline. Both collect from the same durable job ids, so you can start with polling and switch later without changing Step 1.

Track and resume with the jobs list

You don't have to rely solely on your own jobs.json. list_jobs(limit=, before=) returns your recent jobs, newest first, cursor-paginated — handy for a dashboard or for rebuilding state after losing your local record. Page through with the returned next_before cursor.

from speechrevolutions import SpeechRevolutions

client = SpeechRevolutions()
before = None
while True:
    page = client.list_jobs(limit=100, before=before)
    for job in page["jobs"]:
        print(job["job_id"], job["created_at"])
    before = page["next_before"]
    if not before:
        break

That's the scalable shape: submit with bounded concurrency, persist the ids, collect out of band. For the live single-file experience instead, see the meeting-transcription tutorial and the live progress guide; for more patterns, the cookbook.