After story 1
This commit is contained in:
+152
@@ -0,0 +1,152 @@
|
||||
# STORY-03: Frame Sampling
|
||||
|
||||
## Epic
|
||||
**E1: Core Pipeline** — As an engineer, I can configure frame sampling interval and extract frames uniformly.
|
||||
|
||||
## Related Requirements
|
||||
| ID | Requirement |
|
||||
|----|---------|
|
||||
| FR-01 | Configurable frame sampling interval (default: 1 frame per 30 seconds); Must support override per job/batch; Uniform temporal sampling preferred |
|
||||
| NFR-02 | Latency per video: ≤ 45 seconds end-to-end (15-min avg video) |
|
||||
| NFR-03 | GPU Memory Safety: ≤ 18GB per GPU sustained |
|
||||
| NFR-05 | Fault Tolerance: Auto-retry on transient failures; skip & log on fatal errors |
|
||||
| TC-03 | Storage I/O: Fast local NVMe/SSD for temp frame cache |
|
||||
|
||||
## Description
|
||||
Implement frame extraction from processed videos using FFmpeg/OpenCV. Extract one frame per configurable interval (default: 30 seconds) uniformly across the video duration. Handle variable FPS, high-resolution frames, and manage scratch space efficiently.
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
- Uniform temporal frame sampling via FFmpeg `-ss` timestamp extraction
|
||||
- Configurable sampling interval (default: 30 seconds, override per job)
|
||||
- Variable FPS handling (calculate correct timestamps)
|
||||
- Frame extraction to JPEG format for storage efficiency
|
||||
- Scratch space management (NVMe tmpfs with auto-cleanup)
|
||||
- High-resolution frame handling (downscale 4K to fit VRAM constraints)
|
||||
- Frame naming convention: `{video_id}_{timestamp}.jpg`
|
||||
- Memory management (delete frames after processing)
|
||||
|
||||
### Out of Scope
|
||||
- Face detection (covered in STORY-04)
|
||||
- Classification (covered in STORY-05)
|
||||
- Confidence scoring (covered in STORY-05)
|
||||
- Results export (covered in STORY-06)
|
||||
- Review UI (covered in STORY-07)
|
||||
|
||||
## Deliverables
|
||||
|
||||
### 3.1 Frame Sampler Module
|
||||
**File:** `src/frame_sampler.py`
|
||||
|
||||
Core functionality:
|
||||
- **Timestamp Calculator:** Compute uniform sampling timestamps
|
||||
```python
|
||||
def calculate_timestamps(duration, interval):
|
||||
"""Return list of timestamps for frame extraction."""
|
||||
count = max(1, int(duration / interval))
|
||||
step = duration / count
|
||||
return [i * step for i in range(count)]
|
||||
```
|
||||
- **FFmpeg Extraction:** Extract frames at computed timestamps
|
||||
```bash
|
||||
ffmpeg -ss {timestamp} -i {video_path} -vframes 1 -q:v 2 -f jpeg {output_path}
|
||||
```
|
||||
- **Variable FPS Handling:** Adjust timestamps for videos with variable frame rates
|
||||
- **Quality Control:** JPEG quality factor 2 (high quality, reasonable size)
|
||||
- **Error Handling:** Per-frame error isolation — skip failed frames, log errors, continue
|
||||
|
||||
### 3.2 Resolution Handling
|
||||
**File:** `src/frame_sampler.py` (resolution handling section)
|
||||
|
||||
Features:
|
||||
- **Resolution Detection:** Use metadata from prober (STORY-02)
|
||||
- **Downscale Logic:** Auto-downscale frames > 1080p to fit VRAM
|
||||
- 4K (3840x2160) → 1080p (1920x1080)
|
||||
- 2K (2560x1440) → 720p (1280x720)
|
||||
- ≤ 1080p → no change
|
||||
- **FFmpeg Scale Filter:** Apply during extraction
|
||||
```bash
|
||||
ffmpeg -ss {ts} -i {video} -vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2" -vframes 1 -q:v 2 -f jpeg {output}
|
||||
```
|
||||
- **Aspect Ratio Preservation:** Letterbox/pillarbox to maintain original aspect ratio
|
||||
|
||||
### 3.3 Scratch Space Management
|
||||
**File:** `src/scratch_manager.py`
|
||||
|
||||
Features:
|
||||
- **Directory Structure:** `/scratch/{video_id}/frames/` for each video
|
||||
- **Naming Convention:** `{video_id}_{timestamp_ms}.jpg`
|
||||
- **Auto-Cleanup:** Delete all frames after video processing completes
|
||||
- **Space Monitoring:** Alert if scratch usage exceeds 80% of tmpfs capacity
|
||||
- **Cleanup on Error:** Ensure frames are cleaned up even if processing fails
|
||||
- **Concurrent Access:** Per-video directory isolation (no conflicts between workers)
|
||||
|
||||
### 3.4 Configuration Updates
|
||||
**File:** `config.yaml` (updates to STORY-01/02)
|
||||
|
||||
New fields:
|
||||
```yaml
|
||||
sampling:
|
||||
interval_seconds: 30
|
||||
override_per_job: true
|
||||
quality: 2 # JPEG quality (1-31, lower=better)
|
||||
format: jpeg
|
||||
|
||||
resolution:
|
||||
max_height: 1080
|
||||
preserve_aspect_ratio: true
|
||||
pad_to_square: false # Will be done in face detection if needed
|
||||
|
||||
scratch:
|
||||
path: /scratch
|
||||
max_usage_percent: 80
|
||||
auto_cleanup: true
|
||||
cleanup_on_error: true
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
- [ ] Frames are extracted at uniform intervals (±1 frame tolerance) across video duration
|
||||
- [ ] Default interval of 30 seconds produces correct number of frames for test videos
|
||||
- [ ] Per-job interval override works (e.g., 10-second interval for specific videos)
|
||||
- [ ] Variable FPS videos produce evenly spaced timestamps (not evenly spaced frames)
|
||||
- [ ] 4K frames are downsampled to 1080p without crashing
|
||||
- [ ] Aspect ratio is preserved during downscaling (no distortion)
|
||||
- [ ] Frames are saved as high-quality JPEGs (quality factor 2)
|
||||
- [ ] Frame naming follows convention: `{video_id}_{timestamp_ms}.jpg`
|
||||
- [ ] Scratch space is cleaned up after video processing (no leftover frames)
|
||||
- [ ] Failed frame extraction does not stop processing of other frames
|
||||
|
||||
### Non-Functional
|
||||
- [ ] Frame extraction for a 15-minute video completes in < 10 seconds on NVMe
|
||||
- [ ] Scratch space usage per video < 50MB (typical case)
|
||||
- [ ] Memory usage during extraction < 500MB per worker
|
||||
- [ ] FFmpeg process timeout enforced (30 seconds per frame)
|
||||
- [ ] No frames left in scratch after processing completes (verified by directory check)
|
||||
|
||||
### Technical Constraints
|
||||
- [ ] FFmpeg uses `-ss` for accurate timestamp seeking (not frame-by-frame)
|
||||
- [ ] Downscaling uses FFmpeg scale filter (not post-processing resize)
|
||||
- [ ] JPEG quality is consistent across all extracted frames
|
||||
- [ ] Timestamps are in seconds with millisecond precision
|
||||
- [ ] Scratch directory is created per-video and isolated from other workers
|
||||
|
||||
## Dependencies
|
||||
- **Prerequisites:** STORY-01 (Foundation), STORY-02 (Ingestion — provides metadata)
|
||||
- **Depends on:** None (runs after ingestion, before STORY-04)
|
||||
- **Enables:** STORY-04 (Face Detection — provides frames), STORY-05 (Classification)
|
||||
|
||||
## Risks & Mitigations
|
||||
| Risk | Mitigation |
|
||||
|------|--|
|
||||
| FFmpeg seeking is inaccurate for some codecs | Use `-accurate_seek` flag; fall back to frame-by-frame if needed |
|
||||
| 4K downscaling increases processing time | Use FFmpeg scale filter (GPU-accelerated if available) |
|
||||
| Scratch space fills up with many concurrent videos | Monitor usage; implement cleanup queue; alert at 80% |
|
||||
| Variable FPS causes uneven frame distribution | Calculate timestamps based on duration, not frame count |
|
||||
|
||||
## Estimated Effort
|
||||
- **Sprint:** 5 (first half)
|
||||
- **Story Points:** 13
|
||||
- **Dependencies:** STORY-01, STORY-02
|
||||
Reference in New Issue
Block a user