3.1.0 (#839)
* Replace audio whenever set via source * add H264_qsv&HEVC_qsv (#768) * Update ffmpeg.py * Update choices.py * Update typing.py * Fix spaces and newlines * Fix return type * Introduce hififace swapper * Disable stream for expression restorer * Webcam polishing part1 (#796) * Cosmetics on ignore comments * Testing for replace audio * Testing for restore audio * Testing for restore audio * Fix replace_audio() * Remove shortest and use fixed video duration * Remove shortest and use fixed video duration * Prevent duplicate entries to local PATH * Do hard exit on invalid args * Need for Python 3.10 * Fix state of face selector * Fix OpenVINO by aliasing GPU.0 to GPU * Fix OpenVINO by aliasing GPU.0 to GPU * Fix/age modifier styleganex 512 (#798) * fix * styleganex template * changes * changes * fix occlusion mask * add age modifier scale * change * change * hardcode * Cleanup * Use model_sizes and model_templates variables * No need for prepare when just 2 lines of code * Someone used spaces over tabs * Revert back [0][0] --------- Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> * Feat/update gradio5 (#799) * Update to Gradio 5 * Remove overrides for Gradio * Fix dark mode for Gradio * Polish errors * More styles for tabs and co * Make slider inputs and reset like a unit * Make slider inputs and reset like a unit * Adjust naming * Improved color matching (#800) * aura fix * fix import * move to vision.py * changes * changes * changes * changes * further reduction * add test * better test * change name * Minor cleanup * Minor cleanup * Minor cleanup * changes (#801) * Switch to official assets repo * Add __pycache__ to gitignore * Gradio pinned python-multipart to 0.0.12 * Update dependencies * Feat/temp path second try (#802) * Terminate base directory from temp helper * Partial adjust program codebase * Move arguments around * Make `-j` absolete * Resolve args * Fix job register keys * Adjust date test * Finalize temp path * Update onnxruntime * Update dependencies * Adjust color for checkboxes * Revert due terrible performance * Fix/enforce vp9 for webm (#805) * Simple fix to enforce vp9 for webm * Remove suggest methods from program helper * Cleanup ffmpeg.py a bit * Update onnxruntime (second try) * Update onnxruntime (second try) * Remove cudnn_conv_algo_search tweaks * Remove cudnn_conv_algo_search tweaks * changes * add both mask instead of multiply * adaptive color correction * changes * remove model size requirement * changes * add to facefusion.ini * changes * changes * changes * Add namespace for dfm creators * Release five frame enhancer models * Remove vendor from model name * Remove vendor from model name * changes * changes * changes * changes * Feat/download providers (#809) * Introduce download providers * update processors download method * add ui * Fix CI * Adjust UI component order, Use download resolver for benchmark * Remove is_download_done() * Introduce download provider set, Remove choices method from execution, cast all dict keys() via list() * Fix spacing --------- Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> * Fix model paths for 3.1.0 * Introduce bulk-run (#810) * Introduce bulk-run * Make bulk run bullet proof * Integration test for bulk-run * new alignment * Add safer global named resolve_file_pattern() (#811) * Allow bulk runner with target pattern only * changes * changes * Update Python to 3.12 for CI (#813) * changes * Improve NVIDIA device lookups * Rename template key to deepfacelive * Fix name * Improve resolve download * Rename bulk-run to batch-run * Make deep swapper inputs universal * Add more deepfacelive models * Use different morph value * Feat/simplify hashes sources download (#814) * Extract download directory path from assets path * Fix lint * Fix force-download command, Fix urls in frame enhancer * changes * fix warp_face_by_bounding_box dtype error * DFM Morph (#816) * changes * Improve wording, Replace [None], SideQuest: clean forward() of age modifier * SideQuest: clean forward() of face enhancer --------- Co-authored-by: henryruhs <info@henryruhs.com> * Fix preview refresh after slide * Add more deepfacelive models (#817) * Add more deepfacelive models * Add more deepfacelive models * Fix deep swapper sizes * Kill accent colors, Number input styles for Chrome * Simplify thumbnail-item looks * Fix first black screen * Introduce model helper * ci.yml: Add macOS on ARM64 to the testing (#818) * ci.yml: Add macOS on ARM64 to the testing * ci.yml: uses: AnimMouse/setup-ffmpeg@v1 * ci.yml: strategy: matrix: os: macos-latest, * - name: Set up FFmpeg * Update .github/workflows/ci.yml * Update ci.yml --------- Co-authored-by: Henry Ruhs <info@henryruhs.com> * Show/hide morph slider for deep swapper (#822) * remove dfl_head and update dfl_whole_face template * Add deep swapper models by Mats * Add deep swapper models by Druuzil * Add deep swapper models by Rumateus * Implement face enhancer weight for codeformer, Side Quest: has proces… (#823) * Implement face enhancer weight for codeformer, Side Quest: has processor checks * Fix typo * Fix face enhancer blend in UI * Use static model set creation * Add deep swapper models by Jen * Introduce create_static_model_set() everywhere (#824) * Move clear over to the UI (#825) * Fix model key * Undo restore_audio() * Switch to latest XSeg * Switch to latest XSeg * Switch to latest XSeg * Use resolve_download_url() everywhere, Vanish --skip-download flag * Fix resolve_download_url * Fix space * Kill resolve_execution_provider_keys() and move fallbacks where they belong * Kill resolve_execution_provider_keys() and move fallbacks where they belong * Remove as this does not work * Change TempFrameFormat order * Fix CoreML partially * Remove duplicates (Rumateus is the creator) * Add deep swapper models by Edel * Introduce download scopes (#826) * Introduce download scopes * Limit download scopes to force-download command * Change source-paths behaviour * Fix space * Update README * Rename create_log_level_program to create_misc_program * Fix wording * Fix wording * Update dependencies * Use tolerant for video_memory_strategy in benchmark * Feat/ffmpeg with progress (#827) * FFmpeg with progress bar * Fix typing * FFmpeg with progress bar part2 * Restore streaming wording * Change order in choices and typing * Introduce File using list_directory() (#830) * Feat/local deep swapper models (#832) * Local model support for deep swapper * Local model support for deep swapper part2 * Local model support for deep swapper part3 * Update yet another dfm by Druuzil * Refactor/choices and naming (#833) * Refactor choices, imports and naming * Refactor choices, imports and naming * Fix styles for tabs, Restore toast * Update yet another dfm by Druuzil * Feat/face masker models (#834) * Introduce face masker models * Introduce face masker models * Introduce face masker models * Register needed step keys * Provide different XSeg models * Simplify model context * Fix out of range for trim frame, Fix ffmpeg extraction count (#836) * Fix out of range for trim frame, Fix ffmpeg extraction count * Move restrict of trim frame to the core, Make sure all values are within the range * Fix and merge testing * Fix typing * Add region mask for deep swapper * Adjust wording * Move FACE_MASK_REGIONS to choices * Update dependencies * Feat/download provider fallback (#837) * Introduce download providers fallback, Use CURL everywhre * Fix CI * Use readlines() over readline() to avoid while * Use readlines() over readline() to avoid while * Use readlines() over readline() to avoid while * Use communicate() over wait() * Minor updates for testing * Stop webcam on source image change * Feat/webcam improvements (#838) * Detect available webcams * Fix CI, Move webcam id dropdown to the sidebar, Disable warnings * Fix CI * Remove signal on hard_exit() to prevent exceptions * Fix border color in toast timer * Prepare release * Update preview * Update preview * Hotfix progress bar --------- Co-authored-by: DDXDB <38449595+DDXDB@users.noreply.github.com> Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> Co-authored-by: Christian Clauss <cclauss@me.com>
This commit is contained in:
Henry Ruhs
@@ -1,114 +1,118 @@
|
||||
from functools import lru_cache
|
||||
from typing import Tuple
|
||||
|
||||
import cv2
|
||||
import numpy
|
||||
|
||||
from facefusion import inference_manager, state_manager
|
||||
from facefusion.download import conditional_download_hashes, conditional_download_sources
|
||||
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
|
||||
from facefusion.face_helper import create_rotated_matrix_and_size, estimate_matrix_by_face_landmark_5, transform_points, warp_face_by_translation
|
||||
from facefusion.filesystem import resolve_relative_path
|
||||
from facefusion.thread_helper import conditional_thread_semaphore
|
||||
from facefusion.typing import Angle, BoundingBox, DownloadSet, FaceLandmark5, FaceLandmark68, InferencePool, ModelSet, Prediction, Score, VisionFrame
|
||||
from facefusion.typing import Angle, BoundingBox, DownloadScope, DownloadSet, FaceLandmark5, FaceLandmark68, InferencePool, ModelSet, Prediction, Score, VisionFrame
|
||||
|
||||
MODEL_SET : ModelSet =\
|
||||
{
|
||||
'2dfan4':
|
||||
|
||||
@lru_cache(maxsize = None)
|
||||
def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
|
||||
return\
|
||||
{
|
||||
'hashes':
|
||||
'2dfan4':
|
||||
{
|
||||
'2dfan4':
|
||||
'hashes':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/2dfan4.hash',
|
||||
'path': resolve_relative_path('../.assets/models/2dfan4.hash')
|
||||
}
|
||||
'2dfan4':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', '2dfan4.hash'),
|
||||
'path': resolve_relative_path('../.assets/models/2dfan4.hash')
|
||||
}
|
||||
},
|
||||
'sources':
|
||||
{
|
||||
'2dfan4':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', '2dfan4.onnx'),
|
||||
'path': resolve_relative_path('../.assets/models/2dfan4.onnx')
|
||||
}
|
||||
},
|
||||
'size': (256, 256)
|
||||
},
|
||||
'sources':
|
||||
'peppa_wutz':
|
||||
{
|
||||
'2dfan4':
|
||||
'hashes':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/2dfan4.onnx',
|
||||
'path': resolve_relative_path('../.assets/models/2dfan4.onnx')
|
||||
}
|
||||
'peppa_wutz':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', 'peppa_wutz.hash'),
|
||||
'path': resolve_relative_path('../.assets/models/peppa_wutz.hash')
|
||||
}
|
||||
},
|
||||
'sources':
|
||||
{
|
||||
'peppa_wutz':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', 'peppa_wutz.onnx'),
|
||||
'path': resolve_relative_path('../.assets/models/peppa_wutz.onnx')
|
||||
}
|
||||
},
|
||||
'size': (256, 256)
|
||||
},
|
||||
'size': (256, 256)
|
||||
},
|
||||
'peppa_wutz':
|
||||
{
|
||||
'hashes':
|
||||
'fan_68_5':
|
||||
{
|
||||
'peppa_wutz':
|
||||
'hashes':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/peppa_wutz.hash',
|
||||
'path': resolve_relative_path('../.assets/models/peppa_wutz.hash')
|
||||
}
|
||||
},
|
||||
'sources':
|
||||
{
|
||||
'peppa_wutz':
|
||||
'fan_68_5':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', 'fan_68_5.hash'),
|
||||
'path': resolve_relative_path('../.assets/models/fan_68_5.hash')
|
||||
}
|
||||
},
|
||||
'sources':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/peppa_wutz.onnx',
|
||||
'path': resolve_relative_path('../.assets/models/peppa_wutz.onnx')
|
||||
}
|
||||
},
|
||||
'size': (256, 256)
|
||||
},
|
||||
'fan_68_5':
|
||||
{
|
||||
'hashes':
|
||||
{
|
||||
'fan_68_5':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/fan_68_5.hash',
|
||||
'path': resolve_relative_path('../.assets/models/fan_68_5.hash')
|
||||
}
|
||||
},
|
||||
'sources':
|
||||
{
|
||||
'fan_68_5':
|
||||
{
|
||||
'url': 'https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/fan_68_5.onnx',
|
||||
'path': resolve_relative_path('../.assets/models/fan_68_5.onnx')
|
||||
'fan_68_5':
|
||||
{
|
||||
'url': resolve_download_url('models-3.0.0', 'fan_68_5.onnx'),
|
||||
'path': resolve_relative_path('../.assets/models/fan_68_5.onnx')
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def get_inference_pool() -> InferencePool:
|
||||
_, model_sources = collect_model_downloads()
|
||||
model_context = __name__ + '.' + state_manager.get_item('face_landmarker_model')
|
||||
return inference_manager.get_inference_pool(model_context, model_sources)
|
||||
return inference_manager.get_inference_pool(__name__, model_sources)
|
||||
|
||||
|
||||
def clear_inference_pool() -> None:
|
||||
model_context = __name__ + '.' + state_manager.get_item('face_landmarker_model')
|
||||
inference_manager.clear_inference_pool(model_context)
|
||||
inference_manager.clear_inference_pool(__name__)
|
||||
|
||||
|
||||
def collect_model_downloads() -> Tuple[DownloadSet, DownloadSet]:
|
||||
model_set = create_static_model_set('full')
|
||||
model_hashes =\
|
||||
{
|
||||
'fan_68_5': MODEL_SET.get('fan_68_5').get('hashes').get('fan_68_5')
|
||||
'fan_68_5': model_set.get('fan_68_5').get('hashes').get('fan_68_5')
|
||||
}
|
||||
model_sources =\
|
||||
{
|
||||
'fan_68_5': MODEL_SET.get('fan_68_5').get('sources').get('fan_68_5')
|
||||
'fan_68_5': model_set.get('fan_68_5').get('sources').get('fan_68_5')
|
||||
}
|
||||
|
||||
if state_manager.get_item('face_landmarker_model') in [ 'many', '2dfan4' ]:
|
||||
model_hashes['2dfan4'] = MODEL_SET.get('2dfan4').get('hashes').get('2dfan4')
|
||||
model_sources['2dfan4'] = MODEL_SET.get('2dfan4').get('sources').get('2dfan4')
|
||||
model_hashes['2dfan4'] = model_set.get('2dfan4').get('hashes').get('2dfan4')
|
||||
model_sources['2dfan4'] = model_set.get('2dfan4').get('sources').get('2dfan4')
|
||||
|
||||
if state_manager.get_item('face_landmarker_model') in [ 'many', 'peppa_wutz' ]:
|
||||
model_hashes['peppa_wutz'] = MODEL_SET.get('peppa_wutz').get('hashes').get('peppa_wutz')
|
||||
model_sources['peppa_wutz'] = MODEL_SET.get('peppa_wutz').get('sources').get('peppa_wutz')
|
||||
model_hashes['peppa_wutz'] = model_set.get('peppa_wutz').get('hashes').get('peppa_wutz')
|
||||
model_sources['peppa_wutz'] = model_set.get('peppa_wutz').get('sources').get('peppa_wutz')
|
||||
|
||||
return model_hashes, model_sources
|
||||
|
||||
|
||||
def pre_check() -> bool:
|
||||
download_directory_path = resolve_relative_path('../.assets/models')
|
||||
model_hashes, model_sources = collect_model_downloads()
|
||||
|
||||
return conditional_download_hashes(download_directory_path, model_hashes) and conditional_download_sources(download_directory_path, model_sources)
|
||||
return conditional_download_hashes(model_hashes) and conditional_download_sources(model_sources)
|
||||
|
||||
|
||||
def detect_face_landmarks(vision_frame : VisionFrame, bounding_box : BoundingBox, face_angle : Angle) -> Tuple[FaceLandmark68, Score]:
|
||||
@@ -119,6 +123,7 @@ def detect_face_landmarks(vision_frame : VisionFrame, bounding_box : BoundingBox
|
||||
|
||||
if state_manager.get_item('face_landmarker_model') in [ 'many', '2dfan4' ]:
|
||||
face_landmark_2dfan4, face_landmark_score_2dfan4 = detect_with_2dfan4(vision_frame, bounding_box, face_angle)
|
||||
|
||||
if state_manager.get_item('face_landmarker_model') in [ 'many', 'peppa_wutz' ]:
|
||||
face_landmark_peppa_wutz, face_landmark_score_peppa_wutz = detect_with_peppa_wutz(vision_frame, bounding_box, face_angle)
|
||||
|
||||
@@ -128,7 +133,7 @@ def detect_face_landmarks(vision_frame : VisionFrame, bounding_box : BoundingBox
|
||||
|
||||
|
||||
def detect_with_2dfan4(temp_vision_frame: VisionFrame, bounding_box: BoundingBox, face_angle: Angle) -> Tuple[FaceLandmark68, Score]:
|
||||
model_size = MODEL_SET.get('2dfan4').get('size')
|
||||
model_size = create_static_model_set('full').get('2dfan4').get('size')
|
||||
scale = 195 / numpy.subtract(bounding_box[2:], bounding_box[:2]).max().clip(1, None)
|
||||
translation = (model_size[0] - numpy.add(bounding_box[2:], bounding_box[:2]) * scale) * 0.5
|
||||
rotated_matrix, rotated_size = create_rotated_matrix_and_size(face_angle, model_size)
|
||||
@@ -147,7 +152,7 @@ def detect_with_2dfan4(temp_vision_frame: VisionFrame, bounding_box: BoundingBox
|
||||
|
||||
|
||||
def detect_with_peppa_wutz(temp_vision_frame : VisionFrame, bounding_box : BoundingBox, face_angle : Angle) -> Tuple[FaceLandmark68, Score]:
|
||||
model_size = MODEL_SET.get('peppa_wutz').get('size')
|
||||
model_size = create_static_model_set('full').get('peppa_wutz').get('size')
|
||||
scale = 195 / numpy.subtract(bounding_box[2:], bounding_box[:2]).max().clip(1, None)
|
||||
translation = (model_size[0] - numpy.add(bounding_box[2:], bounding_box[:2]) * scale) * 0.5
|
||||
rotated_matrix, rotated_size = create_rotated_matrix_and_size(face_angle, model_size)
|
||||
@@ -167,7 +172,7 @@ def detect_with_peppa_wutz(temp_vision_frame : VisionFrame, bounding_box : Bound
|
||||
|
||||
def conditional_optimize_contrast(crop_vision_frame : VisionFrame) -> VisionFrame:
|
||||
crop_vision_frame = cv2.cvtColor(crop_vision_frame, cv2.COLOR_RGB2Lab)
|
||||
if numpy.mean(crop_vision_frame[:, :, 0]) < 30: # type:ignore[arg-type]
|
||||
if numpy.mean(crop_vision_frame[:, :, 0]) < 30: #type:ignore[arg-type]
|
||||
crop_vision_frame[:, :, 0] = cv2.createCLAHE(clipLimit = 2).apply(crop_vision_frame[:, :, 0])
|
||||
crop_vision_frame = cv2.cvtColor(crop_vision_frame, cv2.COLOR_Lab2RGB)
|
||||
return crop_vision_frame
|
||||
|
||||
Reference in New Issue
Block a user