Abstract
Shot boundary detection (SBD) is a fundamental task in video understanding, with the goal of automatically identifying the transition points between different shots in a video. Traditional methods typically focus solely on predicting boundary locations, lacking the ability to model the semantic content within shots and the relationships between them, making it difficult to meet the requirements of downstream tasks such as video generation and video editing for structured video understanding. Additionally, existing datasets generally suffer from imprecise annotations for fade transitions and a lack of samples for complex transitions, limiting further improvements in model performance.
To address these issues, this thesis proposes a structured shot boundary detection method based on the Shot-Query Transformer. We redefine the traditional shot boundary detection task as a structured relationship prediction problem, predicting not only the scope of shots but also modeling both intra-shot and inter-shot relationships simultaneously. The model adopts a DETR-style query-based Transformer architecture, achieving end-to-end shot set prediction through learnable shot queries, while using frame-level classification to enhance boundary localization accuracy. Furthermore, this thesis designs a fully synthetic data construction workflow to programmatically generate large-scale, accurately annotated transition data, and establishes a new evaluation benchmark, OmniShotCutBench, for comprehensively assessing model performance.
Experimental results demonstrate that our method outperforms existing mainstream approaches in multiple aspects, including boundary detection accuracy, complex transition recognition, and structured relationship prediction, thereby validating the effectiveness of structured modeling and synthetic supervision. This research provides new insights into the evolution of shot boundary detection from simple localization tasks toward comprehensive video structure understanding.
