Building a Video Processing Microservice with GPU Acceleration
A FastAPI Microservice : Video Processing with GPU Acceleration
The demand for intelligent video processing is exploding across industries—from security systems that detect threats in real-time to creative tools that apply artistic styles to footage. However, processing video with deep learning models is computationally intensive, requiring specialized infrastructure to deliver results at acceptable speeds. This article explores how to build a production-ready video processing microservice using FastAPI, PyTorch with CUDA acceleration, and a modern asynchronous architecture.
The Challenge of Video Processing at Scale
Video processing with deep learning models presents unique challenges. A single video contains thousands of frames, each requiring forward passes through neural networks. Object detection, style transfer, or resolution upscaling models like VideoMAE can have over 86 million parameters, making real-time inference computationally demanding.
Traditional synchronous approaches block the API server while processing, exhausting worker threads and causing timeouts under load. For a production service, this is unacceptable. The solution lies in decoupling request acceptance from processing through asynchronous job queues.
System Architecture Overview
Our microservice architecture combines several modern technologies to create a scalable, GPU-accelerated pipeline:
text:
┌─────────────────────────────────────────────────────────────┐
│ Client Application │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Video │ │ WebSocket │ │ Progress │ │
│ │ Upload │ │ Connection │ │ Dashboard │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└────────────────────────┬────────────────────────────────────┘
│ HTTP/WebSocket
┌────────────────────────▼────────────────────────────────────┐
│ FastAPI Gateway │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Upload │ │ Job │ │ Status │ │
│ │ Endpoint │ │ Management │ │ WebSocket │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└────────────────────────┬────────────────────────────────────┘
│
┌────────────────────────▼────────────────────────────────────┐
│ Message Broker (Redis) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Job Queue │ │ Status │ │ Result │ │
│ │ (Celery) │ │ Storage │ │ Cache │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└────────────────────────┬────────────────────────────────────┘
│
┌────────────────────────▼────────────────────────────────────┐
│ GPU Worker Pool (Celery Workers) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ PyTorch │ │ CUDA │ │ Object Storage │ │
│ │ Model │ │ Acceleration │ │ (MinIO/S3) │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
This architecture decouples the API layer from GPU processing, ensuring the service remains responsive even under heavy load. The FastAPI gateway accepts uploads, enqueues jobs, and returns job IDs immediately, while Celery workers handle the heavy lifting on GPU nodes.
Setting Up the FastAPI Backend
Core Dependencies
The foundation begins with essential dependencies for both the API gateway and workers:
bash:
pip install fastapi uvicorn celery redis torch torchvision python-multipart
pip install torch --index-url https://download.pytorch.org/whl/cu118 # CUDA 11.8
The PyTorch version must match your CUDA installation. For newer GPUs like RTX 5090 with Blackwell architecture (sm_120), you may need CUDA 12.x wheels (cu128) .
FastAPI Application Setup
Here's a complete FastAPI application with job management endpoints:
python:
from fastapi import FastAPI, File, UploadFile, WebSocket
from fastapi.responses import JSONResponse
from celery import Celery
import uuid
import redis
import os
from datetime import datetime
app = FastAPI(title="GPU Video Processing Microservice")
# Redis for caching and status storage
redis_client = redis.Redis(host="localhost", port=6379, decode_responses=True)
# Celery configuration
celery_app = Celery(
"video_worker",
broker="redis://localhost:6379/0",
backend="redis://localhost:6379/0"
)
# Endpoint to upload video for processing
@app.post("/api/process")
async def process_video(file: UploadFile = File(...)):
job_id = str(uuid.uuid4())
# Save uploaded video to temporary storage
video_path = f"/tmp/{job_id}_{file.filename}"
with open(video_path, "wb") as f:
f.write(await file.read())
# Enqueue processing task
task = process_video_task.delay(job_id, video_path, file.filename)
# Store job metadata
redis_client.hset(
f"job:{job_id}",
mapping={
"status": "queued",
"filename": file.filename,
"task_id": task.id,
"created_at": datetime.utcnow().isoformat(),
"progress": "0"
}
)
redis_client.expire(f"job:{job_id}", 3600) # Auto-expire after 1 hour
return {"job_id": job_id, "status": "queued"}
# Status endpoint for polling
@app.get("/api/status/{job_id}")
async def get_status(job_id: str):
job_data = redis_client.hgetall(f"job:{job_id}")
if not job_data:
return {"error": "Job not found"}
return job_data
# WebSocket for real-time progress updates
@app.websocket("/ws/{job_id}")
async def websocket_endpoint(websocket: WebSocket, job_id: str):
await websocket.accept()
pubsub = redis_client.pubsub()
pubsub.subscribe(f"job:{job_id}:progress")
try:
for message in pubsub.listen():
if message["type"] == "message":
await websocket.send_text(message["data"])
except Exception as e:
pass
finally:
pubsub.unsubscribe()
The GPU Worker Implementation
Loading Models with CUDA
The worker loads models onto the GPU and performs inference on video frames. For memory-constrained environments, enabling memory-efficient attention is critical :
python:
import torch
import torchvision
from celery import Celery
from celery import states
import cv2
import numpy as np
import os
import json
# This code runs on GPU worker nodes with CUDA support
def load_model():
# Load detection model (e.g., Faster R-CNN for object detection [citation:1])
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
model.eval()
model = model.cuda() # Move to GPU
# Enable memory-efficient attention if available
try:
from torch.nn.attention import enable_memory_efficient
enable_memory_efficient()
except ImportError:
pass
return model
def process_video_with_gpu(video_path, model):
cap = cv2.VideoCapture(video_path)
fps = int(cap.get(cv2.CAP_PROP_FPS))
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
processed_frames = []
frame_count = 0
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
# Preprocess frame for PyTorch
tensor_frame = torch.from_numpy(frame).permute(2, 0, 1).float() / 255.0
tensor_frame = tensor_frame.unsqueeze(0).cuda() # Add batch dimension, move to GPU
# Run inference with CUDA acceleration
with torch.no_grad():
predictions = model(tensor_frame)
# Post-process results (draw bounding boxes, etc.)
processed_frame = draw_predictions(frame, predictions)
processed_frames.append(processed_frame)
frame_count += 1
# Report progress via Redis Pub/Sub
progress = int((frame_count / total_frames) * 100)
redis_client.publish(f"job:{job_id}:progress", json.dumps({
"progress": progress,
"frame": frame_count,
"total": total_frames
}))
cap.release()
# Explicit GPU memory cleanup to prevent OOM accumulation [citation:12]
torch.cuda.empty_cache()
return processed_frames
The Celery Task Definition
The Celery task encapsulates the complete processing workflow, including progress reporting and result storage :
python:
celery_app = Celery("video_worker", broker="redis://localhost:6379/0")
@celery_app.task(bind=True)
def process_video_task(self, job_id, video_path, filename):
try:
self.update_state(
state="PROCESSING",
meta={"status": "Loading model...", "progress": 0}
)
# Load model (lazy loading to avoid memory waste)
model = load_model()
self.update_state(
state="PROCESSING",
meta={"status": "Processing video with GPU...", "progress": 10}
)
# Process video on GPU
processed_frames = process_video_with_gpu(video_path, model)
# Save processed video to object storage
output_path = f"/tmp/processed_{filename}"
save_video(processed_frames, output_path, fps)
# Upload to MinIO/S3 (decouples storage from database [citation:4])
object_key = upload_to_storage(output_path, job_id)
# Clean up temporary files
os.remove(video_path)
os.remove(output_path)
# Clear GPU memory
del model
torch.cuda.empty_cache()
# Store final result location
redis_client.hset(
f"job:{job_id}",
mapping={
"status": "completed",
"result_url": object_key,
"progress": "100"
}
)
redis_client.publish(f"job:{job_id}:progress", json.dumps({
"progress": 100,
"status": "completed",
"result_url": object_key
}))
return {"job_id": job_id, "status": "completed"}
except Exception as e:
redis_client.hset(
f"job:{job_id}",
mapping={"status": "failed", "error": str(e)}
)
raise
Real-Time Communication with WebRTC
For applications requiring live video streaming alongside processing, integrating WebRTC enables real-time bidirectional communication. Projects like Hoovik demonstrate how WebRTC can be combined with FastAPI and PyTorch for multimodal emotion AI, streaming video from mobile browsers to GPU backends for low-latency processing.
The WebRTC pipeline follows a similar pattern: the browser streams video via WebRTC/WHIP ingestion to a media server, which forwards frames to GPU processing, and the annotated overlay streams back to the client . FastAPI serves as the signaling server, managing WebRTC session negotiation and participant identities.
Docker Containerization
Containerization ensures consistent deployment across environments. For GPU acceleration, Docker requires the NVIDIA Container Toolkit.
Dockerfile for GPU Worker
dockerfile:
FROM nvidia/cuda:11.8.0-runtime-ubuntu22.04
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
ffmpeg \
libsm6 \
libxext6 \
python3.10 \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
# Install Python dependencies
COPY requirements.txt .
RUN pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118
RUN pip3 install -r requirements.txt
# Copy application code
COPY . .
# Expose port for API
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Docker Compose for Multi-Service Deployment
yaml:
version: '3.8'
services:
redis:
image: redis:alpine
ports:
- "6379:6379"
minio:
image: minio/minio
ports:
- "9000:9000"
- "9001:9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
command: server /data --console-address ":9001"
api:
build: .
ports:
- "8000:8000"
depends_on:
- redis
- minio
environment:
REDIS_URL: redis://redis:6379
MINIO_ENDPOINT: minio:9000
worker:
build:
context: .
dockerfile: Dockerfile.worker
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
depends_on:
- redis
environment:
REDIS_URL: redis://redis:6379
command: celery -A worker worker --loglevel=info --concurrency=1
The --gpus all flag enables GPU access in the container . For cloud deployments, platforms like RunPod.io simplify GPU container deployment.
Production Considerations
Memory Management
GPU memory is a precious resource. For video processing, reduce resolution if experiencing out-of-memory (OOM) errors . Explicit GPU memory cleanup with torch.cuda.empty_cache() after each job prevents OOM accumulation across processing tasks.
Scalability
The architecture supports independent scaling of components :
- Scale API gateways horizontally for increased request throughput
- Scale GPU workers independently to match processing demand
- Isolate GPU compute boundaries makes it easy to scale workers independently of the API layer
Observability
Production systems require comprehensive observability: dashboards for queue depth, GPU utilization, latency percentiles (p95/p99), and logs . Redis provides the metrics backbone for job tracking and system monitoring.
Security
For production deployment, implement JWT authentication, role-based access control (RBAC), rate limiting, and dead-letter queues (DLQ) for failed jobs.
Real-World Implementation Examples
Several production systems demonstrate this architecture:
DanceBits API uses FastAPI and PyTorch for multimodal dance move segmentation, with MediaPipe for pose estimation and audio feature extraction. It runs on CUDA-compatible GPUs for faster inference and supports Docker containerization.
Video Action Recognition Pipeline decouples upload from inference via Redis + Celery, uses MinIO for object storage, and isolates GPU compute boundaries for independent scaling.
Roop-API provides face swapping with job-based asynchronous processing, CUDA support, and Docker deployment with GPU support—supporting both image-to-image and image-to-video processing.
Hoovik combines WebRTC meeting capabilities with multimodal emotion AI using PyTorch, XGBoost, MediaPipe, Redis, and FastAPI, demonstrating real-time video processing with WebRTC streaming.
Conclusion
Building a GPU-accelerated video processing microservice with FastAPI and PyTorch requires careful architectural decisions to balance performance, scalability, and reliability. The combination of FastAPI for the API gateway, Celery and Redis for asynchronous job processing, and Docker for containerization creates a production-ready pipeline suitable for media companies, security systems, or creative tools.
The key to success lies in decoupling request acceptance from processing—ensuring the API remains responsive even under heavy load—and properly managing GPU resources to maximize throughput. With the right architecture, you can build a video processing service capable of handling anything from object detection to style transfer at scale, deployed on cloud or edge infrastructure.