perf: use cached face detection in map-faces live mode#1681
Open
laurigates wants to merge 1 commit intohacksider:mainfrom
Open
perf: use cached face detection in map-faces live mode#1681laurigates wants to merge 1 commit intohacksider:mainfrom
laurigates wants to merge 1 commit intohacksider:mainfrom
Conversation
Switch process_frame_v2() live path from get_many_faces() to get_faces_optimized() so map-faces live mode benefits from time-based detection caching, matching the simple-mode live path. get_faces_optimized() returns cached results when called within the detection interval (~33ms), avoiding redundant face detection on consecutive frames. This was already used in the simple (non-map) live mode but the map-faces live branch was still calling get_many_faces() directly. Inspired by hacksider#1617. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Reviewer's guide (collapsed on small PRs)Reviewer's GuideSwitches live map-faces processing in process_frame_v2() to use the cached face detection path so live mode benefits from time-based detection caching, aligning behavior with the simple live mode path. Sequence diagram for cached face detection in live map-faces processingsequenceDiagram
actor User
participant LiveStream as LiveStreamLoop
participant FaceSwapper as process_frame_v2
participant Detector as get_faces_optimized
participant Cache as DetectionCache
User ->> LiveStream: start_live_map_faces()
loop For_each_frame
LiveStream ->> FaceSwapper: process_frame_v2(frame)
FaceSwapper ->> Detector: get_faces_optimized(processed_frame)
Detector ->> Cache: lookup(processed_frame_timestamp)
alt cache_hit
Cache -->> Detector: cached_detected_faces
Detector -->> FaceSwapper: detected_faces
else cache_miss
Detector ->> Detector: run_face_detection()
Detector ->> Cache: store(results, timestamp)
Detector -->> FaceSwapper: detected_faces
end
FaceSwapper ->> FaceSwapper: map_faces_and_swap(detected_faces)
FaceSwapper -->> LiveStream: swapped_frame
end
LiveStream -->> User: display_swapped_video
Flow diagram for live path in process_frame_v2 using get_faces_optimizedflowchart TD
A[start_process_frame_v2] --> B{mode}
B -->|image_or_video| C[analyze_precomputed_faces]
C --> Z[end_process_frame_v2]
B -->|live_or_webcam| D[preprocess_frame]
D --> E[get_faces_optimized with processed_frame]
E --> F{detected_faces_present}
F -->|no| Z
F -->|yes and many_faces_flag| G[default_source_face]
G --> H[apply_face_swaps_to_all_detected_targets]
H --> Z
F -->|yes and not many_faces_flag| I[select_single_source_face]
I --> J[apply_face_swap_to_primary_target]
J --> Z[end_process_frame_v2]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switch
process_frame_v2()live path fromget_many_faces()toget_faces_optimized()so map-faces live mode benefits from time-based detection caching, matching the simple-mode live path.get_faces_optimized()returns cached results when called within the detection interval (~33ms), avoiding redundant face detection on consecutive frames. This was already used in the simple (non-map) live mode but the map-faces live branch was still callingget_many_faces()directly.Files changed
modules/processors/frame/face_swapper.py— 1 line:get_many_faces→get_faces_optimizedin the live stream branch ofprocess_frame_v2()Inspired by #1617.
Test plan
🤖 Generated with Claude Code
Summary by Sourcery
Enhancements: