3.1.0 (#839)

* Replace audio whenever set via source * add H264_qsv&HEVC_qsv (#768) * Update ffmpeg.py * Update choices.py * Update typing.py * Fix spaces and newlines * Fix return type * Introduce hififace swapper * Disable stream for expression restorer * Webcam polishing part1 (#796) * Cosmetics on ignore comments * Testing for replace audio * Testing for restore audio * Testing for restore audio * Fix replace_audio() * Remove shortest and use fixed video duration * Remove shortest and use fixed video duration * Prevent duplicate entries to local PATH * Do hard exit on invalid args * Need for Python 3.10 * Fix state of face selector * Fix OpenVINO by aliasing GPU.0 to GPU * Fix OpenVINO by aliasing GPU.0 to GPU * Fix/age modifier styleganex 512 (#798) * fix * styleganex template * changes * changes * fix occlusion mask * add age modifier scale * change * change * hardcode * Cleanup * Use model_sizes and model_templates variables * No need for prepare when just 2 lines of code * Someone used spaces over tabs * Revert back [0][0] --------- Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> * Feat/update gradio5 (#799) * Update to Gradio 5 * Remove overrides for Gradio * Fix dark mode for Gradio * Polish errors * More styles for tabs and co * Make slider inputs and reset like a unit * Make slider inputs and reset like a unit * Adjust naming * Improved color matching (#800) * aura fix * fix import * move to vision.py * changes * changes * changes * changes * further reduction * add test * better test * change name * Minor cleanup * Minor cleanup * Minor cleanup * changes (#801) * Switch to official assets repo * Add __pycache__ to gitignore * Gradio pinned python-multipart to 0.0.12 * Update dependencies * Feat/temp path second try (#802) * Terminate base directory from temp helper * Partial adjust program codebase * Move arguments around * Make `-j` absolete * Resolve args * Fix job register keys * Adjust date test * Finalize temp path * Update onnxruntime * Update dependencies * Adjust color for checkboxes * Revert due terrible performance * Fix/enforce vp9 for webm (#805) * Simple fix to enforce vp9 for webm * Remove suggest methods from program helper * Cleanup ffmpeg.py a bit * Update onnxruntime (second try) * Update onnxruntime (second try) * Remove cudnn_conv_algo_search tweaks * Remove cudnn_conv_algo_search tweaks * changes * add both mask instead of multiply * adaptive color correction * changes * remove model size requirement * changes * add to facefusion.ini * changes * changes * changes * Add namespace for dfm creators * Release five frame enhancer models * Remove vendor from model name * Remove vendor from model name * changes * changes * changes * changes * Feat/download providers (#809) * Introduce download providers * update processors download method * add ui * Fix CI * Adjust UI component order, Use download resolver for benchmark * Remove is_download_done() * Introduce download provider set, Remove choices method from execution, cast all dict keys() via list() * Fix spacing --------- Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> * Fix model paths for 3.1.0 * Introduce bulk-run (#810) * Introduce bulk-run * Make bulk run bullet proof * Integration test for bulk-run * new alignment * Add safer global named resolve_file_pattern() (#811) * Allow bulk runner with target pattern only * changes * changes * Update Python to 3.12 for CI (#813) * changes * Improve NVIDIA device lookups * Rename template key to deepfacelive * Fix name * Improve resolve download * Rename bulk-run to batch-run * Make deep swapper inputs universal * Add more deepfacelive models * Use different morph value * Feat/simplify hashes sources download (#814) * Extract download directory path from assets path * Fix lint * Fix force-download command, Fix urls in frame enhancer * changes * fix warp_face_by_bounding_box dtype error * DFM Morph (#816) * changes * Improve wording, Replace [None], SideQuest: clean forward() of age modifier * SideQuest: clean forward() of face enhancer --------- Co-authored-by: henryruhs <info@henryruhs.com> * Fix preview refresh after slide * Add more deepfacelive models (#817) * Add more deepfacelive models * Add more deepfacelive models * Fix deep swapper sizes * Kill accent colors, Number input styles for Chrome * Simplify thumbnail-item looks * Fix first black screen * Introduce model helper * ci.yml: Add macOS on ARM64 to the testing (#818) * ci.yml: Add macOS on ARM64 to the testing * ci.yml: uses: AnimMouse/setup-ffmpeg@v1 * ci.yml: strategy: matrix: os: macos-latest, * - name: Set up FFmpeg * Update .github/workflows/ci.yml * Update ci.yml --------- Co-authored-by: Henry Ruhs <info@henryruhs.com> * Show/hide morph slider for deep swapper (#822) * remove dfl_head and update dfl_whole_face template * Add deep swapper models by Mats * Add deep swapper models by Druuzil * Add deep swapper models by Rumateus * Implement face enhancer weight for codeformer, Side Quest: has proces… (#823) * Implement face enhancer weight for codeformer, Side Quest: has processor checks * Fix typo * Fix face enhancer blend in UI * Use static model set creation * Add deep swapper models by Jen * Introduce create_static_model_set() everywhere (#824) * Move clear over to the UI (#825) * Fix model key * Undo restore_audio() * Switch to latest XSeg * Switch to latest XSeg * Switch to latest XSeg * Use resolve_download_url() everywhere, Vanish --skip-download flag * Fix resolve_download_url * Fix space * Kill resolve_execution_provider_keys() and move fallbacks where they belong * Kill resolve_execution_provider_keys() and move fallbacks where they belong * Remove as this does not work * Change TempFrameFormat order * Fix CoreML partially * Remove duplicates (Rumateus is the creator) * Add deep swapper models by Edel * Introduce download scopes (#826) * Introduce download scopes * Limit download scopes to force-download command * Change source-paths behaviour * Fix space * Update README * Rename create_log_level_program to create_misc_program * Fix wording * Fix wording * Update dependencies * Use tolerant for video_memory_strategy in benchmark * Feat/ffmpeg with progress (#827) * FFmpeg with progress bar * Fix typing * FFmpeg with progress bar part2 * Restore streaming wording * Change order in choices and typing * Introduce File using list_directory() (#830) * Feat/local deep swapper models (#832) * Local model support for deep swapper * Local model support for deep swapper part2 * Local model support for deep swapper part3 * Update yet another dfm by Druuzil * Refactor/choices and naming (#833) * Refactor choices, imports and naming * Refactor choices, imports and naming * Fix styles for tabs, Restore toast * Update yet another dfm by Druuzil * Feat/face masker models (#834) * Introduce face masker models * Introduce face masker models * Introduce face masker models * Register needed step keys * Provide different XSeg models * Simplify model context * Fix out of range for trim frame, Fix ffmpeg extraction count (#836) * Fix out of range for trim frame, Fix ffmpeg extraction count * Move restrict of trim frame to the core, Make sure all values are within the range * Fix and merge testing * Fix typing * Add region mask for deep swapper * Adjust wording * Move FACE_MASK_REGIONS to choices * Update dependencies * Feat/download provider fallback (#837) * Introduce download providers fallback, Use CURL everywhre * Fix CI * Use readlines() over readline() to avoid while * Use readlines() over readline() to avoid while * Use readlines() over readline() to avoid while * Use communicate() over wait() * Minor updates for testing * Stop webcam on source image change * Feat/webcam improvements (#838) * Detect available webcams * Fix CI, Move webcam id dropdown to the sidebar, Disable warnings * Fix CI * Remove signal on hard_exit() to prevent exceptions * Fix border color in toast timer * Prepare release * Update preview * Update preview * Hotfix progress bar --------- Co-authored-by: DDXDB <38449595+DDXDB@users.noreply.github.com> Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> Co-authored-by: Christian Clauss <cclauss@me.com>
2024-12-24 12:46:56 +01:00
parent ec12f679bf
commit 7a09479fb5
102 changed files with 3699 additions and 2112 deletions
--- a/facefusion/ffmpeg.py
+++ b/facefusion/ffmpeg.py
@@ -5,22 +5,50 @@ import tempfile
 from typing import List, Optional

 import filetype
+from tqdm import tqdm

-from facefusion import logger, process_manager, state_manager
+from facefusion import logger, process_manager, state_manager, wording
 from facefusion.filesystem import remove_file
-from facefusion.temp_helper import get_temp_file_path, get_temp_frames_pattern
-from facefusion.typing import AudioBuffer, Fps, OutputVideoPreset
-from facefusion.vision import restrict_video_fps
+from facefusion.temp_helper import get_temp_file_path, get_temp_frame_paths, get_temp_frames_pattern
+from facefusion.typing import AudioBuffer, Fps, OutputVideoPreset, UpdateProgress
+from facefusion.vision import count_trim_frame_total, detect_video_duration, restrict_video_fps


-def run_ffmpeg(args : List[str]) -> subprocess.Popen[bytes]:
-	commands = [ shutil.which('ffmpeg'), '-hide_banner', '-loglevel', 'error' ]
+def run_ffmpeg_with_progress(args: List[str], update_progress : UpdateProgress) -> subprocess.Popen[bytes]:
+	log_level = state_manager.get_item('log_level')
+	commands = [ shutil.which('ffmpeg'), '-hide_banner', '-nostats', '-loglevel', 'error', '-progress', '-' ]
 	commands.extend(args)
 	process = subprocess.Popen(commands, stderr = subprocess.PIPE, stdout = subprocess.PIPE)

 	while process_manager.is_processing():
 		try:
-			if state_manager.get_item('log_level') == 'debug':
+
+			while __line__ := process.stdout.readline().decode().lower():
+				if 'frame=' in __line__:
+					_, frame_number = __line__.split('frame=')
+					update_progress(int(frame_number))
+
+			if log_level == 'debug':
+				log_debug(process)
+			process.wait(timeout = 0.5)
+		except subprocess.TimeoutExpired:
+			continue
+		return process
+
+	if process_manager.is_stopping():
+		process.terminate()
+	return process
+
+
+def run_ffmpeg(args : List[str]) -> subprocess.Popen[bytes]:
+	log_level = state_manager.get_item('log_level')
+	commands = [ shutil.which('ffmpeg'), '-hide_banner', '-nostats', '-loglevel', 'error' ]
+	commands.extend(args)
+	process = subprocess.Popen(commands, stderr = subprocess.PIPE, stdout = subprocess.PIPE)
+
+	while process_manager.is_processing():
+		try:
+			if log_level == 'debug':
 				log_debug(process)
 			process.wait(timeout = 0.5)
 		except subprocess.TimeoutExpired:
@@ -33,7 +61,7 @@ def run_ffmpeg(args : List[str]) -> subprocess.Popen[bytes]:


 def open_ffmpeg(args : List[str]) -> subprocess.Popen[bytes]:
-	commands = [ shutil.which('ffmpeg'), '-hide_banner', '-loglevel', 'quiet' ]
+	commands = [ shutil.which('ffmpeg'), '-loglevel', 'quiet' ]
 	commands.extend(args)
 	return subprocess.Popen(commands, stdin = subprocess.PIPE, stdout = subprocess.PIPE)

@@ -47,9 +75,8 @@ def log_debug(process : subprocess.Popen[bytes]) -> None:
 			logger.debug(error.strip(), __name__)


-def extract_frames(target_path : str, temp_video_resolution : str, temp_video_fps : Fps) -> bool:
-	trim_frame_start = state_manager.get_item('trim_frame_start')
-	trim_frame_end = state_manager.get_item('trim_frame_end')
+def extract_frames(target_path : str, temp_video_resolution : str, temp_video_fps : Fps, trim_frame_start : int, trim_frame_end : int) -> bool:
+	extract_frame_total = count_trim_frame_total(target_path, trim_frame_start, trim_frame_end)
 	temp_frames_pattern = get_temp_frames_pattern(target_path, '%08d')
 	commands = [ '-i', target_path, '-s', str(temp_video_resolution), '-q:v', '0' ]

@@ -62,34 +89,48 @@ def extract_frames(target_path : str, temp_video_resolution : str, temp_video_fp
 	else:
 		commands.extend([ '-vf', 'fps=' + str(temp_video_fps) ])
 	commands.extend([ '-vsync', '0', temp_frames_pattern ])
-	return run_ffmpeg(commands).returncode == 0
+
+	with tqdm(total = extract_frame_total, desc = wording.get('extracting'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
+		process = run_ffmpeg_with_progress(commands, lambda frame_number: progress.update(frame_number - progress.n))
+		return process.returncode == 0


-def merge_video(target_path : str, output_video_resolution : str, output_video_fps : Fps) -> bool:
+def merge_video(target_path : str, output_video_resolution : str, output_video_fps: Fps) -> bool:
+	output_video_encoder = state_manager.get_item('output_video_encoder')
+	output_video_quality = state_manager.get_item('output_video_quality')
+	output_video_preset = state_manager.get_item('output_video_preset')
+	merge_frame_total = len(get_temp_frame_paths(target_path))
 	temp_video_fps = restrict_video_fps(target_path, output_video_fps)
 	temp_file_path = get_temp_file_path(target_path)
 	temp_frames_pattern = get_temp_frames_pattern(target_path, '%08d')
-	commands = [ '-r', str(temp_video_fps), '-i', temp_frames_pattern, '-s', str(output_video_resolution), '-c:v', state_manager.get_item('output_video_encoder') ]
+	is_webm = filetype.guess_mime(target_path) == 'video/webm'

-	if state_manager.get_item('output_video_encoder') in [ 'libx264', 'libx265' ]:
-		output_video_compression = round(51 - (state_manager.get_item('output_video_quality') * 0.51))
-		commands.extend([ '-crf', str(output_video_compression), '-preset', state_manager.get_item('output_video_preset') ])
-	if state_manager.get_item('output_video_encoder') in [ 'libvpx-vp9' ]:
-		output_video_compression = round(63 - (state_manager.get_item('output_video_quality') * 0.63))
+	if is_webm:
+		output_video_encoder = 'libvpx-vp9'
+	commands = [ '-r', str(temp_video_fps), '-i', temp_frames_pattern, '-s', str(output_video_resolution), '-c:v', output_video_encoder ]
+	if output_video_encoder in [ 'libx264', 'libx265' ]:
+		output_video_compression = round(51 - (output_video_quality * 0.51))
+		commands.extend([ '-crf', str(output_video_compression), '-preset', output_video_preset ])
+	if output_video_encoder in [ 'libvpx-vp9' ]:
+		output_video_compression = round(63 - (output_video_quality * 0.63))
 		commands.extend([ '-crf', str(output_video_compression) ])
-	if state_manager.get_item('output_video_encoder') in [ 'h264_nvenc', 'hevc_nvenc' ]:
-		output_video_compression = round(51 - (state_manager.get_item('output_video_quality') * 0.51))
-		commands.extend([ '-cq', str(output_video_compression), '-preset', map_nvenc_preset(state_manager.get_item('output_video_preset')) ])
-	if state_manager.get_item('output_video_encoder') in [ 'h264_amf', 'hevc_amf' ]:
-		output_video_compression = round(51 - (state_manager.get_item('output_video_quality') * 0.51))
-		commands.extend([ '-qp_i', str(output_video_compression), '-qp_p', str(output_video_compression), '-quality', map_amf_preset(state_manager.get_item('output_video_preset')) ])
-	if state_manager.get_item('output_video_encoder') in [ 'h264_videotoolbox', 'hevc_videotoolbox' ]:
-		commands.extend([ '-q:v', str(state_manager.get_item('output_video_quality')) ])
+	if output_video_encoder in [ 'h264_nvenc', 'hevc_nvenc' ]:
+		output_video_compression = round(51 - (output_video_quality * 0.51))
+		commands.extend([ '-cq', str(output_video_compression), '-preset', map_nvenc_preset(output_video_preset) ])
+	if output_video_encoder in [ 'h264_amf', 'hevc_amf' ]:
+		output_video_compression = round(51 - (output_video_quality * 0.51))
+		commands.extend([ '-qp_i', str(output_video_compression), '-qp_p', str(output_video_compression), '-quality', map_amf_preset(output_video_preset) ])
+	if output_video_encoder in [ 'h264_videotoolbox', 'hevc_videotoolbox' ]:
+		commands.extend([ '-q:v', str(output_video_quality) ])
 	commands.extend([ '-vf', 'framerate=fps=' + str(output_video_fps), '-pix_fmt', 'yuv420p', '-colorspace', 'bt709', '-y', temp_file_path ])
-	return run_ffmpeg(commands).returncode == 0
+
+	with tqdm(total = merge_frame_total, desc = wording.get('merging'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
+		process = run_ffmpeg_with_progress(commands, lambda frame_number: progress.update(frame_number - progress.n))
+		return process.returncode == 0


 def concat_video(output_path : str, temp_output_paths : List[str]) -> bool:
+	output_audio_encoder = state_manager.get_item('output_audio_encoder')
 	concat_video_path = tempfile.mktemp()

 	with open(concat_video_path, 'w') as concat_video_file:
@@ -97,7 +138,7 @@ def concat_video(output_path : str, temp_output_paths : List[str]) -> bool:
 			concat_video_file.write('file \'' + os.path.abspath(temp_output_path) + '\'' + os.linesep)
 		concat_video_file.flush()
 		concat_video_file.close()
-	commands = [ '-f', 'concat', '-safe', '0', '-i', concat_video_file.name, '-c:v', 'copy', '-c:a', state_manager.get_item('output_audio_encoder'), '-y', os.path.abspath(output_path) ]
+	commands = [ '-f', 'concat', '-safe', '0', '-i', concat_video_file.name, '-c:v', 'copy', '-c:a', output_audio_encoder, '-y', os.path.abspath(output_path) ]
 	process = run_ffmpeg(commands)
 	process.communicate()
 	remove_file(concat_video_path)
@@ -112,8 +153,9 @@ def copy_image(target_path : str, temp_image_resolution : str) -> bool:


 def finalize_image(target_path : str, output_path : str, output_image_resolution : str) -> bool:
+	output_image_quality = state_manager.get_item('output_image_quality')
 	temp_file_path = get_temp_file_path(target_path)
-	output_image_compression = calc_image_compression(target_path, state_manager.get_item('output_image_quality'))
+	output_image_compression = calc_image_compression(target_path, output_image_quality)
 	commands = [ '-i', temp_file_path, '-s', str(output_image_resolution), '-q:v', str(output_image_compression), '-y', output_path ]
 	return run_ffmpeg(commands).returncode == 0

@@ -134,10 +176,10 @@ def read_audio_buffer(target_path : str, sample_rate : int, channel_total : int)
 	return None


-def restore_audio(target_path : str, output_path : str, output_video_fps : Fps) -> bool:
-	trim_frame_start = state_manager.get_item('trim_frame_start')
-	trim_frame_end = state_manager.get_item('trim_frame_end')
+def restore_audio(target_path : str, output_path : str, output_video_fps : Fps, trim_frame_start : int, trim_frame_end : int) -> bool:
+	output_audio_encoder = state_manager.get_item('output_audio_encoder')
 	temp_file_path = get_temp_file_path(target_path)
+	temp_video_duration = detect_video_duration(temp_file_path)
 	commands = [ '-i', temp_file_path ]

 	if isinstance(trim_frame_start, int):
@@ -146,13 +188,15 @@ def restore_audio(target_path : str, output_path : str, output_video_fps : Fps)
 	if isinstance(trim_frame_end, int):
 		end_time = trim_frame_end / output_video_fps
 		commands.extend([ '-to', str(end_time) ])
-	commands.extend([ '-i', target_path, '-c:v', 'copy', '-c:a', state_manager.get_item('output_audio_encoder'), '-map', '0:v:0', '-map', '1:a:0', '-shortest', '-y', output_path ])
+	commands.extend([ '-i', target_path, '-c:v', 'copy', '-c:a', output_audio_encoder, '-map', '0:v:0', '-map', '1:a:0', '-t', str(temp_video_duration), '-y', output_path ])
 	return run_ffmpeg(commands).returncode == 0


 def replace_audio(target_path : str, audio_path : str, output_path : str) -> bool:
+	output_audio_encoder = state_manager.get_item('output_audio_encoder')
 	temp_file_path = get_temp_file_path(target_path)
-	commands = [ '-i', temp_file_path, '-i', audio_path, '-c:a', state_manager.get_item('output_audio_encoder'), '-af', 'apad', '-shortest', '-y', output_path ]
+	temp_video_duration = detect_video_duration(temp_file_path)
+	commands = [ '-i', temp_file_path, '-i', audio_path, '-c:v', 'copy', '-c:a', output_audio_encoder, '-t', str(temp_video_duration), '-y', output_path ]
 	return run_ffmpeg(commands).returncode == 0


@@ -174,3 +218,13 @@ def map_amf_preset(output_video_preset : OutputVideoPreset) -> Optional[str]:
 	if output_video_preset in [ 'slow', 'slower', 'veryslow' ]:
 		return 'quality'
 	return None
+
+
+def map_qsv_preset(output_video_preset : OutputVideoPreset) -> Optional[str]:
+	if output_video_preset in [ 'ultrafast', 'superfast', 'veryfast', 'faster', 'fast' ]:
+		return 'fast'
+	if output_video_preset == 'medium':
+		return 'medium'
+	if output_video_preset in [ 'slow', 'slower', 'veryslow' ]:
+		return 'slow'
+	return None