jones/facefusion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Next (#318)

Browse Source 
* renaming and restructuring (#282)

* Renaming and restructuring

* Renaming and restructuring

* Renaming and restructuring

* Fix gender detection

* Implement distance to face debugger

* Implement distance to face debugger part2

* Implement distance to face debugger part3

* Mark as next

* Fix reference when face_debugger comes first

* Use official onnxruntime nightly

* CUDA on steroids

* CUDA on steroids

* Add some testing

* Set inswapper_128_fp16 as default

* Feat/block until post check (#292)

* Block until download is done

* Introduce post_check()

* Fix webcam

* Update dependencies

* Add --force-reinstall to installer

* Introduce config ini (#298)

* Introduce config ini

* Fix output video encoder

* Revert help listings back to commas, Move SSL hack to download.py

* Introduce output-video-preset which defaults to veryfast

* Mapping for nvenc encoders

* Rework on events and non-blocking UI

* Add fast bmp to temp_frame_formats

* Add fast bmp to temp_frame_formats

* Show total processing time on success

* Show total processing time on success

* Show total processing time on success

* Move are_images, is_image and is_video back to filesystem

* Fix some spacings

* Pissing everyone of by renaming stuff

* Fix seconds output

* feat/video output fps (#312)

* added output fps slider, removed 'keep fps' option (#311)

* added output fps slider, removed 'keep fps' option

* now uses passed fps instead of global fps for ffmpeg

* fps values are now floats instead of ints

* fix previous commit

* removed default value from fps slider

this is so we can implement a dynamic default value later

* Fix seconds output

* Some cleanup

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>

* Allow 0.01 steps for fps

* Make fps unregulated

* Make fps unregulated

* Remove distance from face debugger again (does not work)

* Fix gender age

* Fix gender age

* Hotfix benchmark suite

* Warp face normalize (#313)

* use normalized kp templates

* Update face_helper.py

* My 50 cents to warp_face()

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* face-swapper-weight (#315)

* Move prepare_crop_frame and normalize_crop_frame out of apply_swap

* Fix UI bug with different range

* feat/output video resolution (#316)

* Introduce detect_video_resolution, Rename detect_fps to detect_video_fps

* Add calc_video_resolution_range

* Make output resolution work, does not auto-select yet

* Make output resolution work, does not auto-select yet

* Try to keep the origin resolution

* Split code into more fragments

* Add pack/unpack resolution

* Move video_template_sizes to choices

* Improve create_video_resolutions

* Reword benchmark suite

* Optimal speed for benchmark

* Introduce different video memory strategies, rename max_memory to max… (#317)

* Introduce different video memory strategies, rename max_memory to max_system_memory

* Update readme

* Fix limit_system_memory call

* Apply video_memory_strategy to face debugger

* Limit face swapper weight to 3.0

* Remove face swapper weight due bad render outputs

* Show/dide logic for output video preset

* fix uint8 conversion

* Fix whitespace

* Finalize layout and update preview

* Fix multi renders on face debugger

* Restore less restrictive rendering of preview and stream

* Fix block mode for model downloads

* Add testing

* Cosmetic changes

* Enforce valid fps and resolution via CLI

* Empty config

* Cosmetics on args processing

* Memory workover (#319)

* Cosmetics on args processing

* Fix for MacOS

* Rename all max_ to _limit

* More fixes

* Update preview

* Fix whitespace

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
This commit is contained in:
Henry Ruhs
2024-01-19 19:57:48 +01:00
committed by GitHub
parent 3e93f99eeb
commit f6e56a3d8c
61 changed files with 1097 additions and 432 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
facefusion/face_helper.py
View File

@@ -10,47 +10,59 @@ TEMPLATES : Dict[Template, numpy.ndarray[Any, Any]] =\
{
'arcface_112_v1': numpy.array(
[
[ 39.7300, 51.1380 ],
[ 72.2700, 51.1380 ],
[ 56.0000, 68.4930 ],
[ 42.4630, 87.0100 ],
[ 69.5370, 87.0100 ]
[ 0.35473214, 0.45658929 ],
[ 0.64526786, 0.45658929 ],
[ 0.50000000, 0.61154464 ],
[ 0.37913393, 0.77687500 ],
[ 0.62086607, 0.77687500 ]
]),
'arcface_112_v2': numpy.array(
[
[ 38.2946, 51.6963 ],
[ 73.5318, 51.5014 ],
[ 56.0252, 71.7366 ],
[ 41.5493, 92.3655 ],
[ 70.7299, 92.2041 ]
[ 0.34191607, 0.46157411 ],
[ 0.65653393, 0.45983393 ],
[ 0.50022500, 0.64050536 ],
[ 0.37097589, 0.82469196 ],
[ 0.63151696, 0.82325089 ]
]),
'arcface_128_v2': numpy.array(
[
[ 46.2946, 51.6963 ],
[ 81.5318, 51.5014 ],
[ 64.0252, 71.7366 ],
[ 49.5493, 92.3655 ],
[ 78.7299, 92.2041 ]
[ 0.36167656, 0.40387734 ],
[ 0.63696719, 0.40235469 ],
[ 0.50019687, 0.56044219 ],
[ 0.38710391, 0.72160547 ],
[ 0.61507734, 0.72034453 ]
]),
'ffhq_512': numpy.array(
[
[ 192.98138, 239.94708 ],
[ 318.90277, 240.1936 ],
[ 256.63416, 314.01935 ],
[ 201.26117, 371.41043 ],
[ 313.08905, 371.15118 ]
[ 0.37691676, 0.46864664 ],
[ 0.62285697, 0.46912813 ],
[ 0.50123859, 0.61331904 ],
[ 0.39308822, 0.72541100 ],
[ 0.61150205, 0.72490465 ]
])
}
def warp_face(temp_frame : Frame, kps : Kps, template : Template, size : Size) -> Tuple[Frame, Matrix]:
normed_template = TEMPLATES.get(template) * size[1] / size[0]
def warp_face_by_kps(temp_frame : Frame, kps : Kps, template : Template, crop_size : Size) -> Tuple[Frame, Matrix]:
normed_template = TEMPLATES.get(template) * crop_size
affine_matrix = cv2.estimateAffinePartial2D(kps, normed_template, method = cv2.RANSAC, ransacReprojThreshold = 100)[0]
crop_frame = cv2.warpAffine(temp_frame, affine_matrix, (size[1], size[1]), borderMode = cv2.BORDER_REPLICATE)
crop_frame = cv2.warpAffine(temp_frame, affine_matrix, crop_size, borderMode = cv2.BORDER_REPLICATE, flags = cv2.INTER_AREA)
return crop_frame, affine_matrix
def paste_back(temp_frame : Frame, crop_frame: Frame, crop_mask : Mask, affine_matrix : Matrix) -> Frame:
def warp_face_by_bbox(temp_frame : Frame, bbox : Bbox, crop_size : Size) -> Tuple[Frame, Matrix]:
source_kps = numpy.array([[ bbox[0], bbox[1] ], [bbox[2], bbox[1] ], [bbox[0], bbox[3] ]], dtype = numpy.float32)
target_kps = numpy.array([[ 0, 0 ], [ crop_size[0], 0 ], [ 0, crop_size[1] ]], dtype = numpy.float32)
affine_matrix = cv2.getAffineTransform(source_kps, target_kps)
if bbox[2] - bbox[0] > crop_size[0] or bbox[3] - bbox[1] > crop_size[1]:
interpolation_method = cv2.INTER_AREA
else:
interpolation_method = cv2.INTER_LINEAR
crop_frame = cv2.warpAffine(temp_frame, affine_matrix, crop_size, flags = interpolation_method)
return crop_frame, affine_matrix
def paste_back(temp_frame : Frame, crop_frame : Frame, crop_mask : Mask, affine_matrix : Matrix) -> Frame:
inverse_matrix = cv2.invertAffineTransform(affine_matrix)
temp_frame_size = temp_frame.shape[:2][::-1]
inverse_crop_mask = cv2.warpAffine(crop_mask, inverse_matrix, temp_frame_size).clip(0, 1)