VideoDetect/STORY-05.md

STORY-05: Classification & Confidence Aggregation

Epic

E1: Core Pipeline — I can classify face crops and aggregate video confidence.

Related Requirements

ID	Requirement
FR-03	Binary demographic classification of detected face crops: Outputs probability p ∈ [0,1] for target class
FR-04	Video-level confidence aggregation & threshold routing: Aggregates frame-level scores → video confidence C; Routes to MATCH, REVIEW, or SKIP
NFR-01	Throughput: ≥ 30 videos/hour/GPU
NFR-03	GPU Memory Safety: ≤ 18GB per GPU sustained
NFR-04	Determinism & Reproducibility: Config-seeded randomness, versioned models
TC-01	Hardware: 2× Tesla P40 24GB (compute capability 5.2, PCIe 3.0, no Tensor Cores)
TC-02	CUDA/Torch Compatibility: CUDA ≤ 11.8, PyTorch ≤ 2.1.0, FP32 inference only

Description

Implement demographic classification of detected face crops using a lightweight classifier (MobileNetV3 or equivalent). Apply temperature scaling for calibrated confidence scores. Aggregate frame-level confidence to video-level confidence using configurable aggregation logic. Route videos to MATCH, REVIEW, or SKIP based on threshold comparison.

Scope

In Scope

• MobileNetV3 (or equivalent) classification model via TensorRT FP32
• Temperature scaling for confidence calibration
• Frame-level confidence computation (softmax with temperature)
• Video-level confidence aggregation (max or weighted mean, configurable)
• Threshold-based routing (T_high, T_low)
• Routing decision assignment (MATCH, REVIEW, SKIP)
• Confidence score persistence to DB and output files

Out of Scope

• Frame sampling (covered in STORY-03)
• Face detection (covered in STORY-04)
• Results export format (covered in STORY-06)
• Review UI (covered in STORY-07)
• Active learning / model retraining (covered in STORY-08)

Deliverables

5.1 Classification Model

File: models/classifier/

Components:

• Source Model: MobileNetV3-small (or equivalent lightweight classifier)
  • Trained on demographic dataset (fairness considerations documented)
  • Binary classification: Black male subject (class 1) vs. not (class 0)
• ONNX Export: classifier.onnx
  • Input: [1, 3, 224, 224] (RGB, resized face crop)
  • Output: [1, 2] (logits for class 0 and class 1)
• TensorRT Engine: classifier.trt (FP32)
  • Builder config: max_batch_size=32, max_workspace_size=2GB
• Calibration Data: Temperature parameter T stored with model
• Model Metadata: model.json with architecture, training dataset, validation metrics

5.2 Classification Runner

File: src/classifier.py

Core components:

• Model Loader: Load TensorRT engine at startup

  class FaceClassifier:
      def __init__(self, engine_path, temperature=1.0, device='cuda'):
          self.engine = load_trt_engine(engine_path)
          self.context = self.engine.create_execution_context()
          self.temperature = temperature
          self.input_shape = (1, 3, 224, 224)
  

• Preprocessing: Convert face crop to model input
  • Resize to 224×224
  • Normalize (ImageNet statistics)
  • Convert to tensor (NCHW format)
• Inference: Run classification on face crops
  • Dynamic batching (shared with face detection batcher)
  • Execute TensorRT engine
  • Get raw logits output
• Temperature Scaling: Apply temperature to logits before softmax

  def calibrated_softmax(logits, temperature):
      scaled_logits = logits / temperature
      return softmax(scaled_logits, axis=-1)
  

• Frame-level Confidence: Extract probability for target class (class 1)

  p_i = calibrated_probs[:, 1]  # probability of target class
  

5.3 Confidence Aggregation

File: src/aggregator.py

Features:

• Aggregation Strategies (configurable):
  1. Max: C = max(p_i) — use highest confidence frame
  2. Weighted Mean: C = softmax(α·mean(p_i) + β·var(p_i))
  3. Top-K Mean: C = mean(top_k(p_i)) — average of top K confidences
• Default Strategy: Max (most conservative, aligns with FR-04)
• Parameter Configuration: α, β, K configurable in config.yaml
• Variance Calculation: Compute variance of frame-level confidences (for uncertainty estimation)

5.4 Threshold Routing

File: src/router.py

Features:

• Threshold Comparison:

  if C >= T_high:  # default 0.75
      routing = 'MATCH'
  elif C >= T_low:  # default 0.45
      routing = 'REVIEW'
  else:
      routing = 'SKIP'
  

• Configurable Thresholds: T_high and T_low in config.yaml
• Routing Decision Logging: Log routing decision with confidence score
• Edge Case Handling:
  • No faces detected → routing = SKIP (with confidence = 0.0)
  • All frames have same confidence → routing based on threshold comparison
  • Confidence exactly at threshold → use >= comparison (inclusive)

5.5 Configuration Updates

File: config.yaml (updates)

New fields:

classifier:
  model: mobilenetv3-small
  model_path: /models/classifier/classifier.trt
  input_size: 224
  temperature: 1.0  # calibration temperature
  default_strategy: max  # max, weighted_mean, top_k_mean

aggregation:
  strategy: max
  alpha: 1.0   # for weighted_mean
  beta: 0.1    # for weighted_mean
  top_k: 3     # for top_k_mean

routing:
  T_high: 0.75
  T_low: 0.45
  no_faces_decision: SKIP

Acceptance Criteria

Functional

• MobileNetV3 model loads and runs inference via TensorRT FP32
• Classification outputs calibrated probability p ∈ [0,1] for target class
• Temperature scaling is applied correctly (verified on test set)
• Frame-level confidence scores are deterministic (same input → same output)
• Max aggregation produces C = max(p_i) correctly
• Weighted mean aggregation produces correct result with configurable α, β
• Threshold routing assigns exactly one of: MATCH, REVIEW, SKIP
• Videos with no detected faces are routed to SKIP with confidence 0.0
• Confidence exactly at threshold uses >= comparison (inclusive)

Non-Functional

• Classification for 16 face crops (batched) completes in < 1 second on Tesla P40
• GPU memory usage stays ≤ 18GB per GPU during classification
• Aggregation computation is negligible (< 10ms per video)
• Temperature parameter is stored with model checkpoint
• All routing decisions are logged with confidence score and threshold values

Technical Constraints

• TensorRT engine is FP32 only
• No Tensor Cores used (CC 5.2 constraint)
• CUDA 11.8 compatible
• ONNX model exports without errors
• Confidence scores are reproducible (deterministic inference)
• Aggregation strategy is configurable without code changes

Dependencies

• Prerequisites: STORY-01 (Foundation), STORY-03 (Frame Sampling), STORY-04 (Face Detection — provides crops)
• Depends on: None (runs after face detection)
• Enables: STORY-06 (Results Persistence), STORY-07 (Review UI — provides routing decisions)

Risks & Mitigations

Risk	Mitigation
Temperature scaling parameters need tuning	Evaluate on held-out calibration set during training cycle
Max aggregation may be overly conservative	Offer weighted mean as alternative; allow per-job strategy selection
Model bias concerns with demographic classification	Document training dataset; audit fairness metrics; legal review required
TensorRT engine build time	Pre-build and cache engines; version control model files

Estimated Effort

• Sprint: 5-6 (second half)
• Story Points: 21
• Dependencies: STORY-01, STORY-03, STORY-04