fastvideo.pipelines.preprocess.preprocess_pipeline_text

`fastvideo.pipelines.preprocess.preprocess_pipeline_text`#

Text-only Data Preprocessing pipeline implementation.

This module contains an implementation of the Text-only Data Preprocessing pipeline using the modular pipeline architecture, based on the ODE Trajectory preprocessing.

Module Contents#

Classes#

PreprocessPipeline_Text

Text-only preprocessing pipeline implementation.

Data#

`EntryClass`
`logger`

API#

fastvideo.pipelines.preprocess.preprocess_pipeline_text.EntryClass#: None

class fastvideo.pipelines.preprocess.preprocess_pipeline_text.PreprocessPipeline_Text(model_path: str, fastvideo_args: fastvideo.fastvideo_args.FastVideoArgs | fastvideo.fastvideo_args.TrainingArgs, required_config_modules: list[str] | None = None, loaded_modules: dict[str, torch.nn.Module] | None = None)[source]#

Bases: fastvideo.pipelines.preprocess.preprocess_pipeline_base.BasePreprocessPipeline

Text-only preprocessing pipeline implementation.

Initialization

Initialize the pipeline. After init, the pipeline should be ready to use. The pipeline should be stateless and not hold any batch state.

create_pipeline_stages(fastvideo_args: fastvideo.fastvideo_args.FastVideoArgs)[source]#: Set up pipeline stages with proper dependency injection.

forward(batch: fastvideo.pipelines.pipeline_batch_info.ForwardBatch, fastvideo_args: fastvideo.fastvideo_args.FastVideoArgs, args)[source]#

get_pyarrow_schema()[source]#: Return the PyArrow schema for text-only pipeline.

num_processed_samples: int[source]#: 0

pbar: Any[source]#: None

preprocess_dataloader: torchdata.stateful_dataloader.StatefulDataLoader[source]#: None

preprocess_loader_iter: collections.abc.Iterator[dict[str, Any]][source]#: None

preprocess_text_only(fastvideo_args: fastvideo.fastvideo_args.FastVideoArgs, args)[source]#: Preprocess text-only data.

fastvideo.pipelines.preprocess.preprocess_pipeline_text.logger[source]#: ‘init_logger(…)’