Production Python · Async & Concurrency

Concurrent Python that survives real traffic

Diagnostics-first references on event loops, worker pools, network I/O, throttling, cancellation and resilience — written for engineers operating systems under real load, not tutorials.

Start with the fundamentals Diagnose a slow loop

This knowledge base focuses on the decisions and failure modes that surface after systems reach real traffic: event loop saturation, queue backpressure, upstream rate limits, cancellation bugs, shutdown races, and cross-runtime coordination between async tasks, threads and processes. Over six hundred pages across four sections, each one built from a specific production symptom rather than an API tour, and each number in them measured on real hardware rather than quoted.

Explore the library¶

⚙️

Asyncio Fundamentals & Event Loop Architecture

Event loop internals, coroutines, tasks, futures, scheduling, request context with contextvars, AnyIO and trio interop, what changed from Python 3.10 to 3.14, async scripts and CLIs, type-checking async code — and the profiling that makes a stall explainable.

🧵

Start here for a specific symptom¶

Latency spikes with low CPU — a blocking call is freezing the loop: finding blocking calls with asyncio debug mode, then measuring event loop lag in production so it never surprises you again.
Memory grows until the container dies — an unbounded queue or send buffer: bounded asyncio.Queue with backpressure and drain and write backpressure in asyncio streams.
The upstream keeps returning 429 — your limiter is above their real quota: token bucket rate limiter and handling 429 and Retry-After.
Every deploy drops requests — the shutdown sequence is out of order: draining in-flight requests before shutdown and handling SIGTERM in asyncio services.
One slow dependency takes everything down — nothing is contained: bulkhead isolation with per-dependency semaphores and implementing an async circuit breaker.
Background work silently disappears — the loop only holds a weak reference: preventing task garbage collection with strong references.
Log lines carry the wrong request ID — request data is stored per thread, not per task: propagating request IDs with contextvars and carrying contextvars across threads and executors.
Orphaned child processes pile up — a timeout stopped the wait, not the process: running subprocesses with asyncio.create_subprocess_exec and streaming subprocess output without deadlocks.
Retries charged the customer twice — the timeout hid a completed side effect: idempotency keys for safe async retries.
The service dies every few hours with no error — something is leaking: tracking task growth in long-running services and detecting leaked sockets and file descriptors.
Everything is slow but the dependencies are fine — you are saturated, not slow: measuring queue wait and service time separately and load shedding when the event loop is overloaded.
A popular cache key expires and the database falls over — 50 concurrent misses became 50 origin calls: preventing cache stampedes in asyncio.
The event was never published, or was published twice — a dual write with no atomicity: the transactional outbox pattern in asyncio.
The deploy discarded work that was still running — nobody owned the task: running background tasks in FastAPI safely and building a durable job queue on Postgres.
The async test suite is slow and flaky — it waits on the wall clock and on luck: controlling time in asyncio tests and testing asyncio code with pytest-asyncio.
The load test said p99 was fine, production disagrees — the generator stopped sending while the service stalled: avoiding coordinated omission in latency benchmarks and building an open-loop load generator in asyncio.
A job that must run once ran on two replicas — a lease expired under a paused holder: using Postgres advisory locks from asyncio and the fencing tokens in Distributed Locks & Coordination.
A deploy or a cancelled request takes seconds longer than its deadline — something swallows or stretches cancellation: making retry loops cancellation-safe and bounding cleanup time during cancellation.
Memory stays at its peak long after the traffic is gone — freed memory the allocator kept: returning memory to the OS after load spikes and measuring memory per connection.
Messages vanished after a broker restart — publishes returned before anything was stored: publishing with confirms in aio-pika and stopping Kafka consumers gracefully.
Upgrading Python broke the event loop setup — get_event_loop() raises on 3.14: replacing get_event_loop deprecation warnings.

What you will get¶

Practical patterns for timeout, retry, cancellation, throttling and graceful shutdown behaviour.
Trade-off guidance for selecting asyncio, threads, processes, or hybrid models, with the arithmetic behind each choice.
Diagnostics-first examples for tracing starvation, deadlocks, contention, pool exhaustion and leaked resources.
Production-oriented references for I/O scaling, protocol design, connection reuse and throughput tuning.

Audience¶

Python engineers running web services, gateways, streaming systems and data pipelines.
Teams modernizing legacy concurrency stacks with minimal operational risk.
Developers who want architecture-level context and implementation-level examples in one place.

How to navigate the content¶

Use the four overview pages for mental models, boundaries, and system trade-offs.
Use the section overviews beneath them for the patterns and failure modes of one specific area.
Use the deep-dive articles for step-by-step implementation, verification steps and diagnostic hooks.
Follow the inline links across topics to connect a design choice to its operational behaviour.

New in this edition¶

Async Scripts & CLIs — click and Typer commands that silently exited 0 without running, stdin read at 28 million lines per second instead of 38,000, and cron jobs that never overlap.
Typing Async Code — what mypy, pyright and ruff each catch in async code, and a ParamSpec decorator that kept two errors an Any version hid.
Actors & Supervision — mailbox actors that applied all 10,000 concurrent increments where a lock-free read-modify-write kept one.
Concurrent LLM API Calls — token budgets, retries on 429 and 529, and cancellation that stops generation instead of leaving 2,000 tokens streaming.
Django Async Views & ORM — 1,546 requests per second on one uvicorn worker against 39 on four sync workers, and where Django's thread adapters sit.
Async GraphQL — Strawberry resolvers, DataLoader batching and query limits, with a blocking resolver measured at 19 requests per second.
Browser Automation with Playwright — browser and context lifecycles measured in milliseconds and MiB, and waiting without sleeps.
TLS & DNS — lookups that took 951 ms behind a busy thread pool, Happy Eyeballs connecting in 0.25 s, and handshake costs.
Containers & Serverless — PID 1 signals, CPU quotas that pushed loop lag p99 to 59.6 ms, memory limits Python cannot see, and event loops on Lambda.
Securing Async Services — bcrypt that stalled the loop for 5.4 s, slow-client exhaustion, webhook signatures and bounded fan-out.
Three to five new guides in each of the 41 existing topics — from asyncio's 256 KiB-per-connection TLS buffer to a retry loop that swallowed its own cancellation.

Earlier additions¶

Load Testing & Benchmarking — an open-loop generator that reported a 1 s stall as p99 1,472 ms where ten closed-loop workers reported 13 ms, percentiles that merge correctly, and microbenchmarks that time the code rather than the loop.
Async Data Pipelines — bounded stages that capped a queue at 50 items without costing a millisecond of throughput, plus chunking, checkpoints and ordering.
Distributed Locks & Coordination — Redis leases with fencing tokens, Postgres advisory locks across asyncpg and psycopg pools, and leader election measured on failover.
Async Web Crawlers — per-host queues, robots.txt, URL deduplication and resumable state, tested on a four-host site with one slow host.
Cloud SDKs & Object Storage — shared aioboto3 clients at 2.1 ms per operation instead of 25.9 ms, and streaming downloads that peaked at 72 MiB instead of 577 MiB.
Asyncio Across Python Versions — one probe run on 3.10 through 3.14, and a safe order to upgrade in.
Message Brokers & Event Streams — Kafka, RabbitMQ and Redis Streams consumers verified against live brokers, plus the transactional outbox that makes publishing atomic with the database write.
ASGI Servers & Frameworks — lifespan-owned resources, streaming that cut peak memory from 14.6 MiB to 0.9 MiB, and a uvicorn worker count derived from measurement.
gRPC & RPC — grpc.aio services, streaming calls, interceptors, and a deadline that propagated across a two-hop chain and cancelled the leaf.
Background Jobs & Task Queues — Celery, arq and taskiq compared on the same workload, and a Postgres job queue measured at 2,001 jobs per second.
Async Caching & Deduplication — stampede protection that took 50 origin calls to one, and a two-tier cache that served 22,000 reads with 50.
Observability & Tracing — loop lag, queue wait against service time, OpenTelemetry through contextvars, and logging that never blocks the loop.
Memory & Resource Leaks — four counters, four tools, and a watchdog that names the leaking line by itself.
AnyIO & Trio Interop — one pipeline running unchanged on asyncio and trio, with cancel scopes and memory object streams.

Concurrent Python that survives real traffic

Explore the library¶

Asyncio Fundamentals & Event Loop Architecture

Concurrent Execution & Worker Patterns

Network I/O & Protocol Handling

Resilience, Cancellation & Error Handling