Key actions, tactical cooperation, and spatial relationships in sports activities are usually recorded through 2D videos, which limits users' ability to understand the game in three-dimensional space. This thesis presents a 4D sports scene reconstruction and replay pipeline based on motion capture data and 3D Gaussian Splatting representations. The system reconstructs dynamic human Gaussian avatars and the static environment separately, then integrates them in a unified interactive viewer for scene composition and rendering.
The motion capture system provides multi-view images and human skeletons, which are fitted to SMPL parameters. The images are segmented into static environment and dynamic human components. Environment Gaussians and Gaussian avatars are trained separately, then merged in the viewer for interactive free-viewpoint replay.
3D Gaussians are initialized on the SMPL canonical mesh. During training, the Gaussians are deformed into pose space using SMPL pose as reference, their attributes are predicted by hash-grid encoders, and rendered avatar images are optimized against the input views.
Background-only images are used to extract a point cloud and train static scene Gaussians. During optimization, the loss is computed only on the background region, producing a court representation that can be combined with dynamic avatars in the viewer.