fastvideo.v1.utils

Module Contents

Classes

FlexibleArgumentParser	ArgumentParser that allows both underscore and dash in names.
MixedPrecisionState
SortedHelpFormatter	SortedHelpFormatter that sorts arguments by their option strings.
StoreBoolean
TypeBasedDispatcher

Functions

align_to	align height, width according to alignment
current_stream	replace torch.cuda.current_stream() with fastvideo.v1.utils.current_stream(). it turns out that torch.cuda.current_stream() is quite expensive, as it will construct a new stream object at each call. here we patch torch.cuda.set_stream to keep track of the current stream directly, so that we can avoid calling torch.cuda.current_stream().
dict_to_3d_list	Convert a dictionary of mask indices to a 3D list of tensors.
find_nccl_library	We either use the library file specified by the VLLM_NCCL_SO_PATH environment variable, or we find the library file brought by PyTorch. After importing torch, libnccl.so.2 or librccl.so.1 can be found by ctypes automatically.
get_compute_dtype	Get the current compute dtype from mixed precision policy.
get_exception_traceback
get_lock
get_mixed_precision_state	Get the current mixed precision state.
import_pynvml	Historical comments:
kill_itself_when_parent_died
maybe_download_lora	Check if the model path is a Hugging Face Hub model ID and download it if needed.
maybe_download_model	Check if the model path is a Hugging Face Hub model ID and download it if needed.
maybe_download_model_index	Download and extract just the model_index.json for a Hugging Face model.
remote_breakpoint
resolve_obj_by_qualname	Resolve an object by its fully qualified name.
run_method	Run a method of an object with the given arguments and keyword arguments. If the method is string, it will be converted to a method using getattr. If the method is serialized bytes and will be deserialized using cloudpickle. If the method is a callable, it will be called directly.
set_mixed_precision_policy	Set mixed precision policy globally.
shallow_asdict
update_environment_variables
verify_model_config_and_directory	Verify that the model directory contains a valid diffusers configuration.
warn_for_unimplemented_methods	A replacement for abc.ABC. When we use abc.ABC, subclasses will fail to instantiate if they do not implement all abstract methods. Here, we only require raise NotImplementedError in the base class, and log a warning if the method is not implemented in the subclass.

Data

PRECISION_TO_TYPE
STR_ATTN_CONFIG_ENV_VAR
STR_BACKEND_ENV_VAR
T
logger
prev_set_stream

API

class fastvideo.v1.utils.FlexibleArgumentParser(*args, **kwargs)

Bases: argparse.ArgumentParser

ArgumentParser that allows both underscore and dash in names.

Initialization

parse_args(args=None, namespace=None) → argparse.Namespace

class fastvideo.v1.utils.MixedPrecisionState

compute_dtype: Optional[torch.dtype]

None

master_dtype: Optional[torch.dtype]

None

output_dtype: Optional[torch.dtype]

None

param_dtype: Optional[torch.dtype]

None

reduce_dtype: Optional[torch.dtype]

None

fastvideo.v1.utils.PRECISION_TO_TYPE

None

fastvideo.v1.utils.STR_ATTN_CONFIG_ENV_VAR: str

‘FASTVIDEO_ATTENTION_CONFIG’

fastvideo.v1.utils.STR_BACKEND_ENV_VAR: str

‘FASTVIDEO_ATTENTION_BACKEND’

class fastvideo.v1.utils.SortedHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: argparse.HelpFormatter

SortedHelpFormatter that sorts arguments by their option strings.

Initialization

add_arguments(actions)

class fastvideo.v1.utils.StoreBoolean(option_strings, dest, default=False, required=False, help=None)

Bases: argparse.Action

fastvideo.v1.utils.T

‘TypeVar(…)’

class fastvideo.v1.utils.TypeBasedDispatcher(mapping: List[Tuple[Type, Callable]])

Initialization

fastvideo.v1.utils.align_to(value: int, alignment: int) → int

align height, width according to alignment

Parameters:

• value (int) – height or width

• alignment (int) – target alignment factor

Returns:

the aligned value

Return type:

int

fastvideo.v1.utils.current_stream() → torch.cuda.Stream

replace torch.cuda.current_stream() with fastvideo.v1.utils.current_stream(). it turns out that torch.cuda.current_stream() is quite expensive, as it will construct a new stream object at each call. here we patch torch.cuda.set_stream to keep track of the current stream directly, so that we can avoid calling torch.cuda.current_stream().

the underlying hypothesis is that we do not call torch._C._cuda_setStream from C/C++ code.

fastvideo.v1.utils.dict_to_3d_list(mask_strategy: Optional[Dict[str, Any]] = None, t_max: Optional[int] = None, l_max: Optional[int] = None, h_max: Optional[int] = None) → List[List[List[Optional[torch.Tensor]]]]

Convert a dictionary of mask indices to a 3D list of tensors.

Parameters:

• mask_strategy – keys are “t_l_h”, values are torch.Tensor masks.

• t_max – if provided (all three), force the output shape to (t_max, l_max, h_max). If all three are None, infer shape from the data.

• l_max – if provided (all three), force the output shape to (t_max, l_max, h_max). If all three are None, infer shape from the data.

• h_max – if provided (all three), force the output shape to (t_max, l_max, h_max). If all three are None, infer shape from the data.

fastvideo.v1.utils.find_nccl_library() → str

We either use the library file specified by the VLLM_NCCL_SO_PATH environment variable, or we find the library file brought by PyTorch. After importing torch, libnccl.so.2 or librccl.so.1 can be found by ctypes automatically.

fastvideo.v1.utils.get_compute_dtype() → torch.dtype

Get the current compute dtype from mixed precision policy.

Returns:

The compute dtype to use, defaults to get_default_dtype() if no policy set

Return type:

torch.dtype

fastvideo.v1.utils.get_exception_traceback() → str

fastvideo.v1.utils.get_lock(model_name_or_path: str)

fastvideo.v1.utils.get_mixed_precision_state() → fastvideo.v1.utils.MixedPrecisionState

Get the current mixed precision state.

fastvideo.v1.utils.import_pynvml()

Historical comments:

libnvml.so is the library behind nvidia-smi, and pynvml is a Python wrapper around it. We use it to get GPU status without initializing CUDA context in the current process. Historically, there are two packages that provide pynvml:

• nvidia-ml-py (https://pypi.org/project/nvidia-ml-py/): The official wrapper. It is a dependency of FastVideo, and is installed when users install FastVideo. It provides a Python module named pynvml.

• pynvml (https://pypi.org/project/pynvml/): An unofficial wrapper. Prior to version 12.0, it also provides a Python module pynvml, and therefore conflicts with the official one which is a standalone Python file. This causes errors when both of them are installed. Starting from version 12.0, it migrates to a new module named pynvml_utils to avoid the conflict. It is so confusing that many packages in the community use the unofficial one by mistake, and we have to handle this case. For example, nvcr.io/nvidia/pytorch:24.12-py3 uses the unofficial one, and it will cause errors, see the issue https://github.com/vllm-project/vllm/issues/12847 for example. After all the troubles, we decide to copy the official pynvml module to our codebase, and use it directly.

fastvideo.v1.utils.kill_itself_when_parent_died() → None

fastvideo.v1.utils.logger

‘init_logger(…)’

fastvideo.v1.utils.maybe_download_lora(model_name_or_path: str, local_dir: Optional[str] = None, download: bool = True) → str

Check if the model path is a Hugging Face Hub model ID and download it if needed.

Parameters:

• model_name_or_path – Local path or Hugging Face Hub model ID

• local_dir – Local directory to save the model

• download – Whether to download the model from Hugging Face Hub

Returns:

Local path to the model

fastvideo.v1.utils.maybe_download_model(model_name_or_path: str, local_dir: Optional[str] = None, download: bool = True) → str

Check if the model path is a Hugging Face Hub model ID and download it if needed.

Parameters:

• model_name_or_path – Local path or Hugging Face Hub model ID

• local_dir – Local directory to save the model

• download – Whether to download the model from Hugging Face Hub

Returns:

Local path to the model

fastvideo.v1.utils.maybe_download_model_index(model_name_or_path: str) → Dict[str, Any]

Download and extract just the model_index.json for a Hugging Face model.

Parameters:

model_name_or_path – Path or HF Hub model ID

Returns:

The parsed model_index.json as a dictionary

fastvideo.v1.utils.prev_set_stream

None

fastvideo.v1.utils.remote_breakpoint() → None

fastvideo.v1.utils.resolve_obj_by_qualname(qualname: str) → Any

Resolve an object by its fully qualified name.

fastvideo.v1.utils.run_method(obj: Any, method: Union[str, bytes, Callable], args: tuple[Any], kwargs: dict[str, Any]) → Any

Run a method of an object with the given arguments and keyword arguments. If the method is string, it will be converted to a method using getattr. If the method is serialized bytes and will be deserialized using cloudpickle. If the method is a callable, it will be called directly.

fastvideo.v1.utils.set_mixed_precision_policy(master_dtype: torch.dtype, param_dtype: torch.dtype, reduce_dtype: torch.dtype, output_dtype: Optional[torch.dtype] = None)

Set mixed precision policy globally.

Parameters:

• param_dtype – Parameter dtype used for training

• reduce_dtype – Reduction dtype used for gradients

• output_dtype – Optional output dtype

fastvideo.v1.utils.shallow_asdict(obj) → Dict[str, Any]

fastvideo.v1.utils.update_environment_variables(envs: Dict[str, str])

fastvideo.v1.utils.verify_model_config_and_directory(model_path: str) → Dict[str, Any]

Verify that the model directory contains a valid diffusers configuration.

Parameters:

model_path – Path to the model directory

Returns:

The loaded model configuration as a dictionary

fastvideo.v1.utils.warn_for_unimplemented_methods(cls: Type[fastvideo.v1.utils.T]) → Type[fastvideo.v1.utils.T]

A replacement for abc.ABC. When we use abc.ABC, subclasses will fail to instantiate if they do not implement all abstract methods. Here, we only require raise NotImplementedError in the base class, and log a warning if the method is not implemented in the subclass.