What the system does
The product begins with an athlete recording an athletic movement on their phone. From that single clip, the system identifies joints across the video, extracts movement-specific metrics, returns an annotated video, and turns the results into guidance a person can act on.
It had to feel like a product capability—not a long-running request. Video inference, biomechanics analysis, and rendering are all expensive and variable workloads, so keeping them inside a request-response path would tie up the API and make the experience brittle. I designed a staged pipeline where storage artifacts and events carry work between independently recoverable services.
How the pipeline works
After an athlete uploads their video, the API creates an assessment, saves the source video and timestamp manifest, and publishes an inference job to Pub/Sub. A dedicated GPU worker picks up that job, retrieves the source artifacts from cloud storage, and processes the video frame by frame. It uses a top-down pose-estimation flow: a detector finds the athlete’s bounding box, then an RTMPose model estimates joints within that box.
The worker normalizes the pose data into a stable joint layout, validates that each result lines up with the original timestamps, saves the keypoints as JSON in cloud storage, and publishes a completion event. The API then sends the keypoints and assessment schema to the analysis server, where motion data is filtered and exercise-specific biomechanics metrics are calculated. A rendering stage draws the joints and angles over each video frame, assembles the output video, and returns it to the athlete.
Inside the inference worker
The worker uses a bounded producer-consumer pipeline: one part decodes frames while another feeds the GPU. This keeps the GPU busy without allowing a long or high-frame-rate video to consume memory without limit. Results are written in a portable product-owned format, so model-specific runtime details do not leak into downstream systems.
Separating GPU inference, biomechanics analysis, video rendering, and Curv AI guidance lets each part evolve and scale independently. Durable artifacts, explicit assessment states, transaction locks, and guarded state transitions also make retries safe: a failed stage can be recovered without repeating valid work or overwriting a completed result.
The result
Reworking the inference path reduced per-assessment inference cost by 93% while supporting throughput beyond 10,000 videos per month. The savings came from deliberate GPU batching, controlled concurrency, durable artifacts, and the ability to scale expensive stages independently.
The output is more than a score. Athletes can review a skeletal-overlay video alongside exercise-specific metrics, while Curv AI can consider both the current assessment and past results to provide coaching guidance in context. The architecture turns a raw video upload into a repeatable feedback loop for movement improvement.