3.0.0 (#748)
* Cleanup after age modifier PR * Cleanup after age modifier PR * Use OpenVino 2024.2.0 for installer * Prepare 3.0.0 for installer * Fix benchmark suite, Introduce sync_item() for state manager * Fix lint * Render slide preview also in lower res * Lower thread and queue count to avoid false usage * Fix spacing * Feat/jobs UI (#627) * Jobs UI part1 * Change naming * Jobs UI part2 * Jobs UI part3 * Jobs UI part4 * Jobs UI part4 * Jobs UI part5 * Jobs UI part6 * Jobs UI part7 * Jobs UI part8 * Jobs UI part9 * Jobs UI part10 * Jobs UI part11 * Jobs UI part12 * Fix rebase * Jobs UI part13 * Jobs UI part14 * Jobs UI part15 * changes (#626) * Remove useless ui registration * Remove useless ui registration * move job_list.py replace [0] with get_first() * optimize imports * fix date None problem add test job list * Jobs UI part16 * Jobs UI part17 * Jobs UI part18 * Jobs UI part19 * Jobs UI part20 * Jobs UI part21 * Jobs UI part22 * move job_list_options * Add label to job status checkbox group * changes * changes --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Update some dependencies * UI helper to convert 'none' * validate job (#628) * changes * changes * add test * changes * changes * Minor adjustments * Replace is_json with is_file * Handle empty and invalid json in job_list * Handle empty and invalid json in job_list * Handle empty and invalid json in job_list * Work on the job manager UI * Cosmetic changes on common helper * Just make it work for now * Just make it work for now * Just make it work for now * Streamline the step index lookups * Hide footer * Simplify instant runner * Simplify instant runner UI and job manager UI * Fix empty step choices * Fix empty step choices * Fix none values in UI * Rework on benchmark (add warmup) and job list * Improve ValueAndUnit * Add step 1 of x output * Cosmetic changes on the UI * Fix invalid job file names * Update preview * Introducing has_step() and sorting out insert behaviour * Introducing has_step() and sorting out insert behaviour * Add [ none ] to some job id dropdowns * Make updated dropdown values kinda perfect * Make updated dropdown values kinda perfect * Fix testing * Minor improvement on UI * Fix false config lookup * Remove TensorRT as our models are not made for it * Feat/cli commands second try rev2 (#640) * Refactor CLI to commands * Refactor CLI to commands part2 * Refactor CLI to commands part3 * Refactor CLI to commands part4 * Rename everything to facefusion.py * Refactor CLI to commands part5 * Refactor CLI to commands part6 * Adjust testing * Fix lint * Fix lint * Fix lint * Refactor CLI to commands part7 * Extend State typing * Fix false config lookup, adjust logical orders * Move away from passing program part1 * Move away from passing program part2 * Move away from passing program part3 * Fix lint * Move away from passing program part4 * ui-args update * ui-args update * ui-args update * temporary type fix * Move away from passing program part5 * remove unused * creates args.py * Move away from passing program part6 * Move away from passing program part7 --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Minor optimizations * Update commands in README * Fix job-retry command * Fix multi runs via UI * add more job keys * Cleanup codebase * One method to create inference session (#641) * One method to create inference session * Remove warnings, as there are none * Remember job id during processing * Fix face masker config block * Change wording * Prevent age modifier from using CoreML * add expression restorer (#642) * add expression restorer * fix import * fix lint * changes * changes * changes * Host the final model for expression restorer * Insert step on the given index * UI workover (#644) * UI workover part1 * Introduce ComponentOptions * Only set Media components to None when visibility changes * Clear static faces and reference faces between step processing * Minor changes * Minor changes * Fix testing * Enable test_sanitize_path_for_windows (#646) * Dynamic download during job processing (#647) * Fix face masker UI * Rename run-headless to headless-run * Feat/split frame processor UI (#649) * Split frame processor UI * Split frame processor UI part3, Refactor get_model_initializer * Split frame processor UI part4 * Feat/rename frame processors (#651) * Rename frame processors * Rename frame processors part2 * Fix imports Conflicts: facefusion/uis/layouts/benchmark.py facefusion/uis/layouts/default.py * Fix imports * Cosmetic changes * Fix multi threading for ROCm * Change temp frames pattern * Adjust terminal help * remove expression restorer (#653) * Expression restorer as processor (#655) * add expression restorer * changes * Cleanup code * Add TensorRT support back * Add TensorRT support back * Add TensorRT support back * changes (#656) * Change minor wording * Fix face enhancer slider * Add more typing * Fix expression-restorer when using trim (#659) * changes * changes * Rework/model and inference pool part2 (#660) * Rework on model and inference pool * Introduce inference sources and pools part1 * Introduce inference sources and pools part2 * Introduce inference sources and pools part3 * Introduce inference sources and pools part4 * Introduce inference sources and pools part5 * Introduce inference sources and pools part6 * Introduce inference sources and pools part6 * Introduce inference sources and pools part6 * Introduce inference sources and pools part7 * Introduce inference sources and pools part7 * Introduce inference sources and pools part8 * Introduce inference sources and pools part9 * Introduce inference sources and pools part10 * Introduce inference sources and pools part11 * Introduce inference sources and pools part11 * Introduce inference sources and pools part11 * Introduce inference sources and pools part12 * Reorganize the face masker UI * Fix trim in UI * Feat/hashed sources (#668) * Introduce source helper * Remove post_check() and just use process_manager * Remove post_check() part2 * Add hash based downloads * Add hash based downloads part2 * Add hash based downloads part3 * Add hash based downloads part4 * Add hash based downloads part5 * Add hash based downloads part6 * Add hash based downloads part7 * Add hash based downloads part7 * Add hash based downloads part8 * Remove print * Prepare 3.0.0 release * Fix UI * Release the check when really done * Update inputs for live portrait * Update to 3.0.0 releases, extend download postfix * Move files to the right place * Logging for the hash and source validation * Changing logic to handle corrupt sources * Fix typo * Use names over get_inputs(), Remove set_options() call * Age modifier now works for CoreML too * Update age_modifier.py * Add video encoder h264_videotoolbox and hevc_videotoolbox * Face editor add eye gaze & remove open factor sliders (#670) * changes * add eye gaze * changes * cleanup * add eyebrow control * changes * changes * Feat/terminal UI (#671) * Introduce terminal to the UI * Introduce terminal to the UI part2 * Introduce terminal to the UI part2 * Introduce terminal to the UI part2 * Calc range step to avoid weird values * Use Sequence for ranges * Use Sequence for ranges * changes (#673) * Use Sequence for ranges * Finalize terminal UI * Finalize terminal UI * Webcam cosmetics, Fix normalize fps to accept int * Cosmetic changes * Finalize terminal UI * Rename leftover typings * Fix wording * Fix rounding in metavar * Fix rounding in metavar * Rename to face classifier * Face editor lip moves (#677) * changes * changes * changes * Fix rounding in metavar * Rename to face classifier * changes * changes * update naming --------- Co-authored-by: henryruhs <info@henryruhs.com> * Fix wording * Feat/many landmarker + face analyser breakdown (#678) * Basic multi landmarker integration * Simplify some method names * Break into face_detector and face_landmarker * Fix cosmetics * Fix testing * Break into face_attributor and face_recognizer * Clear them all * Clear them all * Rename to face classifier * Rename to face classifier * Fix testing * Fix stuff * Add face landmarker model to UI * Add face landmarker model to UI part2 * Split the config * Split the UI * Improvement from code review * Improvement from code review * Validate args also for sub parsers * Remove clear of processors in process step * Allow finder control for the face editor * Fix lint * Improve testing performance * Remove unused file, Clear processors from the UI before job runs * Update the installer * Uniform set handler for swapper and detector in the UI * Fix example urls * Feat/inference manager (#684) * Introduce inference manager * Migrate all to inference manager * clean ini * Introduce app context based inference pools * Fix lint * Fix typing * Adjust layout * Less border radius * Rename app context names * Fix/live portrait directml (#691) * changes (#690) * Adjust naming * Use our assets release * Adjust naming --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Add caches to gitignore * Update dependencies and drop CUDA 11.8 support (#693) * Update dependencies and drop CUDA 11.8 support * Play save and keep numpy 1.x.x * Improve TensorRT optimization * changes * changes * changes * changes * changes * changes * changes * changes * changes * Reuse inference sessions (#696) * Fix force-download command * Refactor processors to forward() (#698) * Install tensorrt when selecting cuda * Minor changes * Use latest numpy * Fix limit system memory * Implement forward() for every inference (#699) * Implement forward() for every inference * Implement forward() for every inference * Implement forward() for every inference * Implement forward() for every inference * changes * changes * changes * changes * Feat/fairface (#710) * Replace gender_age model with fair face (#709) * changes * changes * changes * age dropdown to range-slider * Cleanup code * Cleanup code --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Extend installer to set library paths for cuda and tensorrt (#707) * Extend installer to set library paths for cuda and tensorrt * Add refresh of conda env * Remove invalid commands * Set the conda env according to operating system * Update for ROCm 6.2 * fix installer * Aktualisieren von installer.py * Add missing face selector keys * Try to keep original LD_LIBRARY_PATH * windows support installer * Final touch to the installer * Remove spaces * Simplidy collect_model_downloads() * Fix force download for once and forever * Housekeeping (#715) * changes * changes * changes * Fix performance part1 * Fix mixed states (#689) * Fix mixed states * Add missing sync for job args * Move UnionStateXXX to base typing * Undo * Remove UnionStateXXX * Fix app context performance lookup (#717) * Restore performance for inswapper * Mover upper() to the logger * Undo debugging * Move TensorRT installation to docs * Sort out log level typing, Add log level UI dropdown (#719) * Fix inference pool part1 * Validate conda library paths existence * Default face selector order to large-small * Fix inference pool context according to execution provider (#720) * Fix app context under Windows * CUDA and TensorRT update for the installer * Remove concept of static processor modules * Revert false commit * Change event order makes a difference * Fix multi model context in inference pool (#721) * Fix multi model context in inference pool * Fix multi model context in inference pool part2 * Use latest gradio to avoid fastapi bug * Rework on the Windows Installer * Use embedding converter (#724) * changes (#723) * Upload models to official assets repo --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Rework on the Windows Installer part2 * Resolve subprocess calls (#726) * Experiment * Resolve subprocess calls to cover edge cases like broken PATH * Adjust wording * Simplify code * Rework on the Windows Installer part3 * Rework on the Windows Installer part4 * Numpy fix for older onnxruntime * changes (#729) * Add space * Add MacOS installer * Use favicon * Fix disabled logger * Layout polishing (#731) * Update dependencies, Adjust many face landmarker logic * Cosmetics changes * Should be button * Introduce randomized action button * Fix update of lip syncer and expression restorer * Stop sharing inference session this prevents flushing VRAM * Fix test * Fix urls * Prepare release * Vanish inquirer * Sticky preview does not work on portrait images * Sticky preview only for landscape images and videos * remove gradio tunnel env * Change wording and deeplinks * increase peppa landmark score offset * Change wording * Graceful exit install.py * Just adding a required * Cannot use the exit_helper * Rename our model * Change color of face-landmark-68/5 * Limit liveportrait (#739) * changes * changes * changes * Cleanup * Cleanup --------- Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com> Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * limit expression restorer * change expression restorer 0-100 range * Use 256x icon * changes * changes * changes * changes * Limit face editor rotation (#745) * changes (#743) * Finish euler methods --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> * Use different coveralls badge * Move about wording * Shorten scope in the logger * changes * changes * Shorten scope in the logger * fix typo * Simplify the arcface converter names * Update preview --------- Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
This commit is contained in:
Henry Ruhs
0
facefusion/jobs/__init__.py
Normal file
0
facefusion/jobs/__init__.py
Normal file
15
facefusion/jobs/job_helper.py
Normal file
15
facefusion/jobs/job_helper.py
Normal file
@@ -0,0 +1,15 @@
|
||||
import os
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
|
||||
def get_step_output_path(job_id : str, step_index : int, output_path : str) -> Optional[str]:
|
||||
if output_path:
|
||||
output_directory_path, _ = os.path.split(output_path)
|
||||
output_file_name, output_file_extension = os.path.splitext(_)
|
||||
return os.path.join(output_directory_path, output_file_name + '-' + job_id + '-' + str(step_index) + output_file_extension)
|
||||
return None
|
||||
|
||||
|
||||
def suggest_job_id(job_prefix : str = 'job') -> str:
|
||||
return job_prefix + '-' + datetime.now().strftime('%Y-%m-%d-%H-%M-%S')
|
||||
34
facefusion/jobs/job_list.py
Normal file
34
facefusion/jobs/job_list.py
Normal file
@@ -0,0 +1,34 @@
|
||||
from datetime import datetime
|
||||
from typing import Optional, Tuple
|
||||
|
||||
from facefusion.date_helper import describe_time_ago
|
||||
from facefusion.jobs import job_manager
|
||||
from facefusion.typing import JobStatus, TableContents, TableHeaders
|
||||
|
||||
|
||||
def compose_job_list(job_status : JobStatus) -> Tuple[TableHeaders, TableContents]:
|
||||
jobs = job_manager.find_jobs(job_status)
|
||||
job_headers : TableHeaders = [ 'job id', 'steps', 'date created', 'date updated', 'job status' ]
|
||||
job_contents : TableContents = []
|
||||
|
||||
for index, job_id in enumerate(jobs):
|
||||
if job_manager.validate_job(job_id):
|
||||
job = jobs[job_id]
|
||||
step_total = job_manager.count_step_total(job_id)
|
||||
date_created = prepare_describe_datetime(job.get('date_created'))
|
||||
date_updated = prepare_describe_datetime(job.get('date_updated'))
|
||||
job_contents.append(
|
||||
[
|
||||
job_id,
|
||||
step_total,
|
||||
date_created,
|
||||
date_updated,
|
||||
job_status
|
||||
])
|
||||
return job_headers, job_contents
|
||||
|
||||
|
||||
def prepare_describe_datetime(date_time : Optional[str]) -> Optional[str]:
|
||||
if date_time:
|
||||
return describe_time_ago(datetime.fromisoformat(date_time))
|
||||
return None
|
||||
263
facefusion/jobs/job_manager.py
Normal file
263
facefusion/jobs/job_manager.py
Normal file
@@ -0,0 +1,263 @@
|
||||
import glob
|
||||
import os
|
||||
from copy import copy
|
||||
from typing import List, Optional
|
||||
|
||||
from facefusion.choices import job_statuses
|
||||
from facefusion.date_helper import get_current_date_time
|
||||
from facefusion.filesystem import create_directory, is_directory, is_file, move_file, remove_directory, remove_file
|
||||
from facefusion.jobs.job_helper import get_step_output_path
|
||||
from facefusion.json import read_json, write_json
|
||||
from facefusion.temp_helper import create_base_directory
|
||||
from facefusion.typing import Args, Job, JobSet, JobStatus, JobStep, JobStepStatus
|
||||
|
||||
JOBS_PATH : Optional[str] = None
|
||||
|
||||
|
||||
def init_jobs(jobs_path : str) -> bool:
|
||||
global JOBS_PATH
|
||||
|
||||
JOBS_PATH = jobs_path
|
||||
job_status_paths = [ os.path.join(JOBS_PATH, job_status) for job_status in job_statuses ]
|
||||
|
||||
create_base_directory()
|
||||
for job_status_path in job_status_paths:
|
||||
create_directory(job_status_path)
|
||||
return all(is_directory(status_path) for status_path in job_status_paths)
|
||||
|
||||
|
||||
def clear_jobs(jobs_path : str) -> bool:
|
||||
return remove_directory(jobs_path)
|
||||
|
||||
|
||||
def create_job(job_id : str) -> bool:
|
||||
job : Job =\
|
||||
{
|
||||
'version': '1',
|
||||
'date_created': get_current_date_time().isoformat(),
|
||||
'date_updated': None,
|
||||
'steps': []
|
||||
}
|
||||
|
||||
return create_job_file(job_id, job)
|
||||
|
||||
|
||||
def submit_job(job_id : str) -> bool:
|
||||
drafted_job_ids = find_job_ids('drafted')
|
||||
steps = get_steps(job_id)
|
||||
|
||||
if job_id in drafted_job_ids and steps:
|
||||
return set_steps_status(job_id, 'queued') and move_job_file(job_id, 'queued')
|
||||
return False
|
||||
|
||||
|
||||
def submit_jobs() -> bool:
|
||||
drafted_job_ids = find_job_ids('drafted')
|
||||
|
||||
if drafted_job_ids:
|
||||
for job_id in drafted_job_ids:
|
||||
if not submit_job(job_id):
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def delete_job(job_id : str) -> bool:
|
||||
return delete_job_file(job_id)
|
||||
|
||||
|
||||
def delete_jobs() -> bool:
|
||||
job_ids = find_job_ids('drafted') + find_job_ids('queued') + find_job_ids('failed') + find_job_ids('completed')
|
||||
|
||||
if job_ids:
|
||||
for job_id in job_ids:
|
||||
if not delete_job(job_id):
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def find_jobs(job_status : JobStatus) -> JobSet:
|
||||
job_ids = find_job_ids(job_status)
|
||||
jobs : JobSet = {}
|
||||
|
||||
for job_id in job_ids:
|
||||
jobs[job_id] = read_job_file(job_id)
|
||||
return jobs
|
||||
|
||||
|
||||
def find_job_ids(job_status : JobStatus) -> List[str]:
|
||||
job_pattern = os.path.join(JOBS_PATH, job_status, '*.json')
|
||||
job_files = glob.glob(job_pattern)
|
||||
job_files.sort(key = os.path.getmtime)
|
||||
job_ids = []
|
||||
|
||||
for job_file in job_files:
|
||||
job_id, _ = os.path.splitext(os.path.basename(job_file))
|
||||
job_ids.append(job_id)
|
||||
return job_ids
|
||||
|
||||
|
||||
def validate_job(job_id : str) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
return bool(job and 'version' in job and 'date_created' in job and 'date_updated' in job and 'steps' in job)
|
||||
|
||||
|
||||
def has_step(job_id : str, step_index : int) -> bool:
|
||||
step_total = count_step_total(job_id)
|
||||
return step_index in range(step_total)
|
||||
|
||||
|
||||
def add_step(job_id : str, step_args : Args) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
|
||||
if job:
|
||||
job.get('steps').append(
|
||||
{
|
||||
'args': step_args,
|
||||
'status': 'drafted'
|
||||
})
|
||||
return update_job_file(job_id, job)
|
||||
return False
|
||||
|
||||
|
||||
def remix_step(job_id : str, step_index : int, step_args : Args) -> bool:
|
||||
steps = get_steps(job_id)
|
||||
step_args = copy(step_args)
|
||||
|
||||
if step_index and step_index < 0:
|
||||
step_index = count_step_total(job_id) - 1
|
||||
|
||||
if has_step(job_id, step_index):
|
||||
output_path = steps[step_index].get('args').get('output_path')
|
||||
step_args['target_path'] = get_step_output_path(job_id, step_index, output_path)
|
||||
return add_step(job_id, step_args)
|
||||
return False
|
||||
|
||||
|
||||
def insert_step(job_id : str, step_index : int, step_args : Args) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
step_args = copy(step_args)
|
||||
|
||||
if step_index and step_index < 0:
|
||||
step_index = count_step_total(job_id) - 1
|
||||
|
||||
if job and has_step(job_id, step_index):
|
||||
job.get('steps').insert(step_index,
|
||||
{
|
||||
'args': step_args,
|
||||
'status': 'drafted'
|
||||
})
|
||||
return update_job_file(job_id, job)
|
||||
return False
|
||||
|
||||
|
||||
def remove_step(job_id : str, step_index : int) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
|
||||
if step_index and step_index < 0:
|
||||
step_index = count_step_total(job_id) - 1
|
||||
|
||||
if job and has_step(job_id, step_index):
|
||||
job.get('steps').pop(step_index)
|
||||
return update_job_file(job_id, job)
|
||||
return False
|
||||
|
||||
|
||||
def get_steps(job_id : str) -> List[JobStep]:
|
||||
job = read_job_file(job_id)
|
||||
|
||||
if job:
|
||||
return job.get('steps')
|
||||
return []
|
||||
|
||||
|
||||
def count_step_total(job_id : str) -> int:
|
||||
steps = get_steps(job_id)
|
||||
|
||||
if steps:
|
||||
return len(steps)
|
||||
return 0
|
||||
|
||||
|
||||
def set_step_status(job_id : str, step_index : int, step_status : JobStepStatus) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
|
||||
if job:
|
||||
steps = job.get('steps')
|
||||
|
||||
if has_step(job_id, step_index):
|
||||
steps[step_index]['status'] = step_status
|
||||
return update_job_file(job_id, job)
|
||||
return False
|
||||
|
||||
|
||||
def set_steps_status(job_id : str, step_status : JobStepStatus) -> bool:
|
||||
job = read_job_file(job_id)
|
||||
|
||||
if job:
|
||||
for step in job.get('steps'):
|
||||
step['status'] = step_status
|
||||
return update_job_file(job_id, job)
|
||||
return False
|
||||
|
||||
|
||||
def read_job_file(job_id : str) -> Optional[Job]:
|
||||
job_path = find_job_path(job_id)
|
||||
return read_json(job_path) #type:ignore[return-value]
|
||||
|
||||
|
||||
def create_job_file(job_id : str, job : Job) -> bool:
|
||||
job_path = find_job_path(job_id)
|
||||
|
||||
if not is_file(job_path):
|
||||
job_create_path = suggest_job_path(job_id, 'drafted')
|
||||
return write_json(job_create_path, job) #type:ignore[arg-type]
|
||||
return False
|
||||
|
||||
|
||||
def update_job_file(job_id : str, job : Job) -> bool:
|
||||
job_path = find_job_path(job_id)
|
||||
|
||||
if is_file(job_path):
|
||||
job['date_updated'] = get_current_date_time().isoformat()
|
||||
return write_json(job_path, job) #type:ignore[arg-type]
|
||||
return False
|
||||
|
||||
|
||||
def move_job_file(job_id : str, job_status : JobStatus) -> bool:
|
||||
job_path = find_job_path(job_id)
|
||||
job_move_path = suggest_job_path(job_id, job_status)
|
||||
return move_file(job_path, job_move_path)
|
||||
|
||||
|
||||
def delete_job_file(job_id : str) -> bool:
|
||||
job_path = find_job_path(job_id)
|
||||
return remove_file(job_path)
|
||||
|
||||
|
||||
def suggest_job_path(job_id : str, job_status : JobStatus) -> Optional[str]:
|
||||
job_file_name = get_job_file_name(job_id)
|
||||
|
||||
if job_file_name:
|
||||
return os.path.join(JOBS_PATH, job_status, job_file_name)
|
||||
return None
|
||||
|
||||
|
||||
def find_job_path(job_id : str) -> Optional[str]:
|
||||
job_file_name = get_job_file_name(job_id)
|
||||
|
||||
if job_file_name:
|
||||
for job_status in job_statuses:
|
||||
job_pattern = os.path.join(JOBS_PATH, job_status, job_file_name)
|
||||
job_paths = glob.glob(job_pattern)
|
||||
|
||||
for job_path in job_paths:
|
||||
return job_path
|
||||
return None
|
||||
|
||||
|
||||
def get_job_file_name(job_id : str) -> Optional[str]:
|
||||
if job_id:
|
||||
return job_id + '.json'
|
||||
return None
|
||||
106
facefusion/jobs/job_runner.py
Normal file
106
facefusion/jobs/job_runner.py
Normal file
@@ -0,0 +1,106 @@
|
||||
from facefusion.ffmpeg import concat_video
|
||||
from facefusion.filesystem import is_image, is_video, move_file, remove_file
|
||||
from facefusion.jobs import job_helper, job_manager
|
||||
from facefusion.typing import JobOutputSet, JobStep, ProcessStep
|
||||
|
||||
|
||||
def run_job(job_id : str, process_step : ProcessStep) -> bool:
|
||||
queued_job_ids = job_manager.find_job_ids('queued')
|
||||
|
||||
if job_id in queued_job_ids:
|
||||
if run_steps(job_id, process_step) and finalize_steps(job_id):
|
||||
clean_steps(job_id)
|
||||
return job_manager.move_job_file(job_id, 'completed')
|
||||
clean_steps(job_id)
|
||||
job_manager.move_job_file(job_id, 'failed')
|
||||
return False
|
||||
|
||||
|
||||
def run_jobs(process_step : ProcessStep) -> bool:
|
||||
queued_job_ids = job_manager.find_job_ids('queued')
|
||||
|
||||
if queued_job_ids:
|
||||
for job_id in queued_job_ids:
|
||||
if not run_job(job_id, process_step):
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def retry_job(job_id : str, process_step : ProcessStep) -> bool:
|
||||
failed_job_ids = job_manager.find_job_ids('failed')
|
||||
|
||||
if job_id in failed_job_ids:
|
||||
return job_manager.set_steps_status(job_id, 'queued') and job_manager.move_job_file(job_id, 'queued') and run_job(job_id, process_step)
|
||||
return False
|
||||
|
||||
|
||||
def retry_jobs(process_step : ProcessStep) -> bool:
|
||||
failed_job_ids = job_manager.find_job_ids('failed')
|
||||
|
||||
if failed_job_ids:
|
||||
for job_id in failed_job_ids:
|
||||
if not retry_job(job_id, process_step):
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def run_step(job_id : str, step_index : int, step : JobStep, process_step : ProcessStep) -> bool:
|
||||
step_args = step.get('args')
|
||||
|
||||
if job_manager.set_step_status(job_id, step_index, 'started') and process_step(job_id, step_index, step_args):
|
||||
output_path = step_args.get('output_path')
|
||||
step_output_path = job_helper.get_step_output_path(job_id, step_index, output_path)
|
||||
|
||||
return move_file(output_path, step_output_path) and job_manager.set_step_status(job_id, step_index, 'completed')
|
||||
job_manager.set_step_status(job_id, step_index, 'failed')
|
||||
return False
|
||||
|
||||
|
||||
def run_steps(job_id : str, process_step : ProcessStep) -> bool:
|
||||
steps = job_manager.get_steps(job_id)
|
||||
|
||||
if steps:
|
||||
for index, step in enumerate(steps):
|
||||
if not run_step(job_id, index, step, process_step):
|
||||
return False
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def finalize_steps(job_id : str) -> bool:
|
||||
output_set = collect_output_set(job_id)
|
||||
|
||||
for output_path, temp_output_paths in output_set.items():
|
||||
if all(map(is_video, temp_output_paths)):
|
||||
if not concat_video(output_path, temp_output_paths):
|
||||
return False
|
||||
if any(map(is_image, temp_output_paths)):
|
||||
for temp_output_path in temp_output_paths:
|
||||
if not move_file(temp_output_path, output_path):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def clean_steps(job_id: str) -> bool:
|
||||
output_set = collect_output_set(job_id)
|
||||
|
||||
for temp_output_paths in output_set.values():
|
||||
for temp_output_path in temp_output_paths:
|
||||
if not remove_file(temp_output_path):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def collect_output_set(job_id : str) -> JobOutputSet:
|
||||
steps = job_manager.get_steps(job_id)
|
||||
output_set : JobOutputSet = {}
|
||||
|
||||
for index, step in enumerate(steps):
|
||||
output_path = step.get('args').get('output_path')
|
||||
|
||||
if output_path:
|
||||
step_output_path = job_manager.get_step_output_path(job_id, index, output_path)
|
||||
output_set.setdefault(output_path, []).append(step_output_path)
|
||||
return output_set
|
||||
27
facefusion/jobs/job_store.py
Normal file
27
facefusion/jobs/job_store.py
Normal file
@@ -0,0 +1,27 @@
|
||||
from typing import List
|
||||
|
||||
from facefusion.typing import JobStore
|
||||
|
||||
JOB_STORE : JobStore =\
|
||||
{
|
||||
'job_keys': [],
|
||||
'step_keys': []
|
||||
}
|
||||
|
||||
|
||||
def get_job_keys() -> List[str]:
|
||||
return JOB_STORE.get('job_keys')
|
||||
|
||||
|
||||
def get_step_keys() -> List[str]:
|
||||
return JOB_STORE.get('step_keys')
|
||||
|
||||
|
||||
def register_job_keys(step_keys : List[str]) -> None:
|
||||
for step_key in step_keys:
|
||||
JOB_STORE['job_keys'].append(step_key)
|
||||
|
||||
|
||||
def register_step_keys(job_keys : List[str]) -> None:
|
||||
for job_key in job_keys:
|
||||
JOB_STORE['step_keys'].append(job_key)
|
||||
Reference in New Issue
Block a user