# STORY-03: Frame Sampling

## Epic
**E1: Core Pipeline** — As an engineer, I can configure frame sampling interval and extract frames uniformly.

## Related Requirements
| ID | Requirement |
|----|---------|
| FR-01 | Configurable frame sampling interval (default: 1 frame per 30 seconds); Must support override per job/batch; Uniform temporal sampling preferred |
| NFR-02 | Latency per video: ≤ 45 seconds end-to-end (15-min avg video) |
| NFR-03 | GPU Memory Safety: ≤ 18GB per GPU sustained |
| NFR-05 | Fault Tolerance: Auto-retry on transient failures; skip & log on fatal errors |
| TC-03 | Storage I/O: Fast local NVMe/SSD for temp frame cache |

## Description
Implement frame extraction from processed videos using FFmpeg/OpenCV. Extract one frame per configurable interval (default: 30 seconds) uniformly across the video duration. Handle variable FPS, high-resolution frames, and manage scratch space efficiently.

## Scope

### In Scope
- Uniform temporal frame sampling via FFmpeg `-ss` timestamp extraction
- Configurable sampling interval (default: 30 seconds, override per job)
- Variable FPS handling (calculate correct timestamps)
- Frame extraction to JPEG format for storage efficiency
- Scratch space management (NVMe tmpfs with auto-cleanup)
- High-resolution frame handling (downscale 4K to fit VRAM constraints)
- Frame naming convention: `{video_id}_{timestamp}.jpg`
- Memory management (delete frames after processing)

### Out of Scope
- Face detection (covered in STORY-04)
- Classification (covered in STORY-05)
- Confidence scoring (covered in STORY-05)
- Results export (covered in STORY-06)
- Review UI (covered in STORY-07)

## Deliverables

### 3.1 Frame Sampler Module
**File:** `src/frame_sampler.py`

Core functionality:
- **Timestamp Calculator:** Compute uniform sampling timestamps
  ```python
  def calculate_timestamps(duration, interval):
      """Return list of timestamps for frame extraction."""
      count = max(1, int(duration / interval))
      step = duration / count
      return [i * step for i in range(count)]
  ```
- **FFmpeg Extraction:** Extract frames at computed timestamps
  ```bash
  ffmpeg -ss {timestamp} -i {video_path} -vframes 1 -q:v 2 -f jpeg {output_path}
  ```
- **Variable FPS Handling:** Adjust timestamps for videos with variable frame rates
- **Quality Control:** JPEG quality factor 2 (high quality, reasonable size)
- **Error Handling:** Per-frame error isolation — skip failed frames, log errors, continue

### 3.2 Resolution Handling
**File:** `src/frame_sampler.py` (resolution handling section)

Features:
- **Resolution Detection:** Use metadata from prober (STORY-02)
- **Downscale Logic:** Auto-downscale frames > 1080p to fit VRAM
  - 4K (3840x2160) → 1080p (1920x1080)
  - 2K (2560x1440) → 720p (1280x720)
  - ≤ 1080p → no change
- **FFmpeg Scale Filter:** Apply during extraction
  ```bash
  ffmpeg -ss {ts} -i {video} -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2" -vframes 1 -q:v 2 -f jpeg {output}
  ```
- **Aspect Ratio Preservation:** Letterbox/pillarbox to maintain original aspect ratio

### 3.3 Scratch Space Management
**File:** `src/scratch_manager.py`

Features:
- **Directory Structure:** `/scratch/{video_id}/frames/` for each video
- **Naming Convention:** `{video_id}_{timestamp_ms}.jpg`
- **Auto-Cleanup:** Delete all frames after video processing completes
- **Space Monitoring:** Alert if scratch usage exceeds 80% of tmpfs capacity
- **Cleanup on Error:** Ensure frames are cleaned up even if processing fails
- **Concurrent Access:** Per-video directory isolation (no conflicts between workers)

### 3.4 Configuration Updates
**File:** `config.yaml` (updates to STORY-01/02)

New fields:
```yaml
sampling:
  interval_seconds: 30
  override_per_job: true
  quality: 2  # JPEG quality (1-31, lower=better)
  format: jpeg

resolution:
  max_height: 1080
  preserve_aspect_ratio: true
  pad_to_square: false  # Will be done in face detection if needed

scratch:
  path: /scratch
  max_usage_percent: 80
  auto_cleanup: true
  cleanup_on_error: true
```

## Acceptance Criteria

### Functional
- [ ] Frames are extracted at uniform intervals (±1 frame tolerance) across video duration
- [ ] Default interval of 30 seconds produces correct number of frames for test videos
- [ ] Per-job interval override works (e.g., 10-second interval for specific videos)
- [ ] Variable FPS videos produce evenly spaced timestamps (not evenly spaced frames)
- [ ] 4K frames are downsampled to 1080p without crashing
- [ ] Aspect ratio is preserved during downscaling (no distortion)
- [ ] Frames are saved as high-quality JPEGs (quality factor 2)
- [ ] Frame naming follows convention: `{video_id}_{timestamp_ms}.jpg`
- [ ] Scratch space is cleaned up after video processing (no leftover frames)
- [ ] Failed frame extraction does not stop processing of other frames

### Non-Functional
- [ ] Frame extraction for a 15-minute video completes in < 10 seconds on NVMe
- [ ] Scratch space usage per video < 50MB (typical case)
- [ ] Memory usage during extraction < 500MB per worker
- [ ] FFmpeg process timeout enforced (30 seconds per frame)
- [ ] No frames left in scratch after processing completes (verified by directory check)

### Technical Constraints
- [ ] FFmpeg uses `-ss` for accurate timestamp seeking (not frame-by-frame)
- [ ] Downscaling uses FFmpeg scale filter (not post-processing resize)
- [ ] JPEG quality is consistent across all extracted frames
- [ ] Timestamps are in seconds with millisecond precision
- [ ] Scratch directory is created per-video and isolated from other workers

## Dependencies
- **Prerequisites:** STORY-01 (Foundation), STORY-02 (Ingestion — provides metadata)
- **Depends on:** None (runs after ingestion, before STORY-04)
- **Enables:** STORY-04 (Face Detection — provides frames), STORY-05 (Classification)

## Risks & Mitigations
| Risk | Mitigation |
|------|--|
| FFmpeg seeking is inaccurate for some codecs | Use `-accurate_seek` flag; fall back to frame-by-frame if needed |
| 4K downscaling increases processing time | Use FFmpeg scale filter (GPU-accelerated if available) |
| Scratch space fills up with many concurrent videos | Monitor usage; implement cleanup queue; alert at 80% |
| Variable FPS causes uneven frame distribution | Calculate timestamps based on duration, not frame count |

## Estimated Effort
- **Sprint:** 5 (first half)
- **Story Points:** 13
- **Dependencies:** STORY-01, STORY-02