Capture Studio

Capture Studio is a suite of tools that I am developing for an upcoming animation project. Capture Studio is a streamlined system that makes performance capture easier. The goal is to make motion capture easier for non-technical users, and to avoid depending on game engines, marker suits, green screens, or web service subscriptions.

Features

• Face / Pose animation processing from ArKit / Google Mediapipe
• Debugging and Video Processing in native app
• Animation importing tools (currently only in Houdini)
• Automatically exports extracted audio, runs Whisper Text-to-Speech and generated Rhubarb animation data

Desktop App

The desktop app is written in Python. It uses PySide for the GUI and PyInstaller to create the application package.

Videos recorded with the iOS app can be processed to produce the output for Houdini. Additionally any video can be processed using MediaPipe to produce pose and blendshape data.

Capture Studio iOS

Capture studio for iOS leverages the front facing depth camera and Apple’s blendshape detection model to estimate the blendshape poses.

Rather than storing raw blendshape value data we encode the blendshape values into the recorded pixel grid so we can edit, retime, and concatenate videos together before exporting the animation data.