Recent Projects

4D Synthetic Gaussian Splatting Tool

I created this pipeline to solve two separate problems:

Some 4DGS captures pose specific issues during the sync and segmentation process. In the example video above, the golf club was moving far too fast for accurate sync between clips, and was far too thin to be consistently masked during segmentation, so the final 4DGS did not feature the golf club. This tool allows the team to re-introduce props that pose risk to the current camera rig and pipeline.

There are some props that are a logistically impossible to capture on the day. For example – my team has never been able to capture a drummer actually drumming, because it is impossible to set up and break down a drum set in the limited time we have with the artist(s). With this tool, we can capture the artist air-drumming, then light, animate, and train the drum set into a gaussian splat sequence.

Step-by-step 4D Synthetic Splatting process:

Capture: We use an 80-camera rig to capture a performance. I also capture an HDRI of the space to ensure accurate lighting later in the pipeline.

Train 4DGS sequence/extract EasyMocap data: These steps are run concurrently. As we run the videos through our internal training software, we also run the videos through EasyMocap using a bespoke script.

Aligning the motion capture data with the 4DGS sequence: Using MLSLabsGaussianSplattingRenderer to visualize the sequence in Unreal Engine, we align the motion capture skeleton with the 4DGS sequence.

Animate the 3D model: We constrain the 3D model to the skeleton. Further cleanup animation is likely necessary, as the motion capture data is not 1:1 accurate. Bake and visualize the animation sequence: For the scripts later in the pipeline to function properly, the animation first needs to baked, then each frame needs to be visualized as unique geometry.

Light with HDRI: The HDRI captured on shoot day is imported and set up in Unreal. Additional lights are added if necessary.

Capture each frame of animation: We use a bespoke script that automates the process – fundamentally, it generates cameras in a sphere around the subject (the number of cameras is dictated by the user), generates and exports a point cloud based on the geometry, renders the images, exports the camera position data, and moves on to the next frame.

Batch train each frame: Our last bespoke script takes all the data from the previous step and trains a gaussian splat of each frame using Jawset PostShot’s Splat3 training profile.

Weezer Snapchat Lenses

I had the privilege of shooting a 4D Gaussian Splat capture with the rock band Weezer as part of Snapchat’s Under The Ghost concert series. We built two lenses featuring 4DGS assets – I was given the opportunity to develop our “Get Weezered” lens completely end to end, and modeled, textured, and animated the 3D assets for “The Gold Album” lens.

Links to the lenses:
Get Weezered
The Gold Album

“Reckage” – 1 Bit Game Jam Submission

I was the artist for my team’s submission to the 1 Bit Game Jam, a one week game development competition that only allowed the use of 2 colors. A copy of the game is available upon request.