utils
¶
Classes¶
fastvideo.utils.FlexibleArgumentParser
¶
Bases: ArgumentParser
ArgumentParser that allows both underscore and dash in names.
Source code in fastvideo/utils.py
fastvideo.utils.SortedHelpFormatter
¶
Bases: HelpFormatter
SortedHelpFormatter that sorts arguments by their option strings.
Functions¶
fastvideo.utils.align_to
¶
fastvideo.utils.current_stream
¶
replace torch.cuda.current_stream() with fastvideo.utils.current_stream().
it turns out that torch.cuda.current_stream() is quite expensive,
as it will construct a new stream object at each call.
here we patch torch.cuda.set_stream to keep track of the current stream
directly, so that we can avoid calling torch.cuda.current_stream().
the underlying hypothesis is that we do not call torch._C._cuda_setStream
from C/C++ code.
Source code in fastvideo/utils.py
fastvideo.utils.decorate_logs
¶
decorate_logs(process_name: str | None = None) -> None
Adds a process-specific prefix to each line of output written to stdout and stderr.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
process_name
|
str | None
|
Optional; the name of the process to use in the prefix. If not provided, the current process name from the multiprocessing context is used. |
None
|
Source code in fastvideo/utils.py
fastvideo.utils.dict_to_3d_list
¶
dict_to_3d_list(mask_strategy: dict[str, Any] | None = None, t_max: int | None = None, l_max: int | None = None, h_max: int | None = None) -> list[list[list[Tensor | None]]]
Convert a dictionary of mask indices to a 3D list of tensors. Args: mask_strategy: keys are "t_l_h", values are torch.Tensor masks. t_max, l_max, h_max: if provided (all three), force the output shape to (t_max, l_max, h_max). If all three are None, infer shape from the data.
Source code in fastvideo/utils.py
fastvideo.utils.find_hccl_library
¶
find_hccl_library() -> str
We either use the library file specified by the HCCL_SO_PATH
environment variable, or we find the library file brought by PyTorch.
After importing torch, libhccl.so can be
found by ctypes automatically.
Source code in fastvideo/utils.py
fastvideo.utils.find_nccl_library
¶
find_nccl_library() -> str
We either use the library file specified by the FASTVIDEO_NCCL_SO_PATH
environment variable, or we find the library file brought by PyTorch.
After importing torch, libnccl.so.2 or librccl.so.1 can be
found by ctypes automatically.
Source code in fastvideo/utils.py
fastvideo.utils.get_compute_dtype
¶
Get the current compute dtype from mixed precision policy.
Returns:
| Type | Description |
|---|---|
dtype
|
torch.dtype: The compute dtype to use, defaults to get_default_dtype() if no policy set |
Source code in fastvideo/utils.py
fastvideo.utils.get_mixed_precision_state
¶
Get the current mixed precision state.
Source code in fastvideo/utils.py
fastvideo.utils.get_mp_context
¶
Get a multiprocessing context with a particular method (spawn or fork). By default we follow the value of the FASTVIDEO_WORKER_MULTIPROC_METHOD to determine the multiprocessing method (default is fork). However, under certain conditions, we may enforce spawn and override the value of FASTVIDEO_WORKER_MULTIPROC_METHOD.
Source code in fastvideo/utils.py
fastvideo.utils.import_pynvml
¶
Historical comments:
libnvml.so is the library behind nvidia-smi, and
pynvml is a Python wrapper around it. We use it to get GPU
status without initializing CUDA context in the current process.
Historically, there are two packages that provide pynvml:
- nvidia-ml-py (https://pypi.org/project/nvidia-ml-py/): The official
wrapper. It is a dependency of FastVideo, and is installed when users
install FastVideo. It provides a Python module named pynvml.
- pynvml (https://pypi.org/project/pynvml/): An unofficial wrapper.
Prior to version 12.0, it also provides a Python module pynvml,
and therefore conflicts with the official one which is a standalone Python file.
This causes errors when both of them are installed.
Starting from version 12.0, it migrates to a new module
named pynvml_utils to avoid the conflict.
It is so confusing that many packages in the community use the
unofficial one by mistake, and we have to handle this case.
For example, nvcr.io/nvidia/pytorch:24.12-py3 uses the unofficial
one, and it will cause errors, see the issue
https://github.com/vllm-project/vllm/issues/12847 for example.
After all the troubles, we decide to copy the official pynvml
module to our codebase, and use it directly.
Source code in fastvideo/utils.py
fastvideo.utils.log_torch_cuda_memory
¶
log_torch_cuda_memory(tag: str | None = None, *, log_fn: Callable[[str], None] | None = None, log_file_path: str | PathLike[str] | None = 'memory_trace.txt') -> None
Log CUDA memory statistics via logger and append to a trace file.
Source code in fastvideo/utils.py
fastvideo.utils.maybe_download_lora
¶
maybe_download_lora(model_name_or_path: str, local_dir: str | None = None, download: bool = True) -> str
Check if the model path is a Hugging Face Hub model ID and download it if needed. Args: model_name_or_path: Local path or Hugging Face Hub model ID local_dir: Local directory to save the model download: Whether to download the model from Hugging Face Hub
Returns:
| Type | Description |
|---|---|
str
|
Local path to the model |
Source code in fastvideo/utils.py
fastvideo.utils.maybe_download_model
¶
maybe_download_model(model_name_or_path: str, local_dir: str | None = None, download: bool = True) -> str
Check if the model path is a Hugging Face Hub model ID and download it if needed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name_or_path
|
str
|
Local path or Hugging Face Hub model ID |
required |
local_dir
|
str | None
|
Local directory to save the model |
None
|
download
|
bool
|
Whether to download the model from Hugging Face Hub |
True
|
Returns:
| Type | Description |
|---|---|
str
|
Local path to the model |
Source code in fastvideo/utils.py
fastvideo.utils.maybe_download_model_index
¶
Download and extract just the model_index.json for a Hugging Face model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name_or_path
|
str
|
Path or HF Hub model ID |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
The parsed model_index.json as a dictionary |
Source code in fastvideo/utils.py
fastvideo.utils.resolve_obj_by_qualname
¶
Resolve an object by its fully qualified name.
fastvideo.utils.run_method
¶
run_method(obj: Any, method: str | bytes | Callable, args: tuple[Any], kwargs: dict[str, Any]) -> Any
Run a method of an object with the given arguments and keyword arguments. If the method is string, it will be converted to a method using getattr. If the method is serialized bytes and will be deserialized using cloudpickle. If the method is a callable, it will be called directly.
Source code in fastvideo/utils.py
fastvideo.utils.set_mixed_precision_policy
¶
set_mixed_precision_policy(param_dtype: dtype, reduce_dtype: dtype, output_dtype: dtype | None = None, mp_policy: MixedPrecisionPolicy | None = None)
Set mixed precision policy globally.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
param_dtype
|
dtype
|
Parameter dtype used for training |
required |
reduce_dtype
|
dtype
|
Reduction dtype used for gradients |
required |
output_dtype
|
dtype | None
|
Optional output dtype |
None
|
Source code in fastvideo/utils.py
fastvideo.utils.verify_model_config_and_directory
¶
Verify that the model directory contains a valid diffusers configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_path
|
str
|
Path to the model directory |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
The loaded model configuration as a dictionary |
Source code in fastvideo/utils.py
fastvideo.utils.warn_for_unimplemented_methods
¶
A replacement for abc.ABC.
When we use abc.ABC, subclasses will fail to instantiate
if they do not implement all abstract methods.
Here, we only require raise NotImplementedError in the
base class, and log a warning if the method is not implemented
in the subclass.