fastvideo.v1.dataset.utils

fastvideo.v1.dataset.utils#

Module Contents#

Functions#

collate_latents_embs_masks

collate_rows_from_parquet_schema

Collate rows from parquet files based on the provided schema. Dynamically processes tensor fields based on schema and returns batched data.

get_torch_tensors_from_row_dict

Get the latents and prompts from a row dictionary.

pad

Pad or crop an embedding [L, D] to exactly padding_length tokens. Return:

API#

fastvideo.v1.dataset.utils.collate_latents_embs_masks(batch_to_process, text_padding_length, keys, cfg_rate=0.0, rng=None) → tuple[torch.Tensor, torch.Tensor, torch.Tensor, list[str]][source]#
fastvideo.v1.dataset.utils.collate_rows_from_parquet_schema(rows, parquet_schema, text_padding_length, cfg_rate=0.0, rng=None) → dict[str, Any][source]#

Collate rows from parquet files based on the provided schema. Dynamically processes tensor fields based on schema and returns batched data.

Parameters:

  • rows – List of row dictionaries from parquet files

  • parquet_schema – PyArrow schema defining the structure of the data

Returns:

Dict containing batched tensors and metadata

fastvideo.v1.dataset.utils.get_torch_tensors_from_row_dict(row_dict, keys, cfg_rate, rng=None) → dict[str, Any][source]#

Get the latents and prompts from a row dictionary.

fastvideo.v1.dataset.utils.pad(t: torch.Tensor, padding_length: int) → torch.Tensor[source]#

Pad or crop an embedding [L, D] to exactly padding_length tokens. Return:

  • [L, D] tensor in pinned CPU memory

  • [L] attention mask in pinned CPU memory

Contents