Video Decoders

Decoders type

class eztorch.datasets.decoders.DecoderType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Frame decoders

Used to decode videos stored as frames.

class eztorch.datasets.decoders.GeneralFrameVideo(num_threads_io=0, num_threads_decode=0, transform=None)

GeneralFrameVideo is an abstract class for accessing clips stored as frames.

Parameters:

• num_threads_io (int, optional) – Controls whether parallelizable io operations are performed across multiple threads.

  Default: 0

• num_threads_decode (int, optional) – Controls whether parallelizable decode operations are performed across multiple threads.

  Default: 0

• transform (Optional[Callable], optional) – The transform to apply to the frames.

  Default: None

class eztorch.datasets.decoders.FrameVideo(video_path, duration, fps, frame_filter=None, time_difference_prob=0.0, video_frame_to_path_fn=<function get_video_to_frame_path_fn.<locals>.fn>, video_frame_paths=None, num_threads_io=0, num_threads_decode=0, transform=None, decode_float=False)

FrameVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is stored as an image.

Parameters:

• video_path (str) – The path of the video.

• duration (float) – The duration of the video in seconds.

• fps (int) – The target fps for the video. This is needed to link the frames to a second timestamp in the video.

• frame_filter (Optional[Callable[[list[int]], tuple[ndarray[Any, dtype[TypeVar(_ScalarType_co, bound= generic, covariant=True)]], ndarray[Any, dtype[TypeVar(_ScalarType_co, bound= generic, covariant=True)]]]]], optional) – Function to subsample frames in a clip before loading. If None, no subsampling is performed

  Default: None

• video_frame_to_path_fn (Optional[Callable[[str, int], int]], optional) – A function that maps from the video path and a frame index integer to the file path where the frame is located.

  Default: <function get_video_to_frame_path_fn.<locals>.fn>

• video_frame_paths (list[str] | None, optional) – List of frame paths for each index of a video.

  Default: None

• num_threads_io (int, optional) – Controls whether parallelizable io operations are performed across multiple threads.

  Default: 0

• num_threads_decode (int, optional) – Controls whether parallelizable decode operations are performed across multiple threads.

  Default: 0

• transform (Optional[Callable], optional) – The transform to apply to the frames.

  Default: None

• decode_float (bool, optional) – Whether to decode the clip as float.

  Default: False

class eztorch.datasets.decoders.FrameSoccerNetVideo(video_path, half_path, duration, fps_video, fps, num_frames, transform=None, video_frame_to_path_fn=<function get_video_to_frame_path_fn.<locals>.fn>, time_difference_prob=0.0, num_threads_io=0, num_threads_decode=0, num_decode=1, num_decode_time_diff=2, mask_ratio=0.0, mask_tube=2, min_clip_duration=0, decode_float=False)

FrameSoccerNetVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is stored as an image.

Parameters:

• video_path (str | Path) – The path of the video.

• half_path (str | Path) – The path of the half.

• duration (float) – The duration of the video in seconds.

• fps_video (int) – The fps of the video.

• fps (int) – The target fps for the video. This is needed to link the frames to a second timestamp in the video.

• num_frames (int) – The number of frames of the video.

• transform (Optional[Callable], optional) – The transform to apply to the frames.

  Default: None

• video_frame_to_path_fn (Callable[[str, int], int], optional) – A function that maps from the video path and a frame index integer to the file path where the frame is located.

  Default: <function get_video_to_frame_path_fn.<locals>.fn>

• time_difference_prob (float, optional) – Probability to apply time difference.

  Default: 0.0

• num_threads_io (int, optional) – Controls whether parallelizable io operations are performed across multiple threads.

  Default: 0

• num_threads_decode (int, optional) – Controls whether parallelizable decode operations are performed across multiple threads.

  Default: 0

• num_decode (int, optional) – Number of decode to perform. If > 1, the videos decoded are stored in a list.

  Default: 1

• num_decode_time_diff (int, optional) – Number of time difference decode to perform. If > 1, the videos decoded are stored in a list.

  Default: 2

• mask_ratio (float, optional) – Masking ratio for the video.

  Default: 0.0

• mask_ratio – Sequence tube size for masking the video.

• min_clip_duration (float, optional) – The minimum duration of a clip.

  Default: 0

• decode_float (bool, optional) – Whether to decode the clip as float.

  Default: False

class eztorch.datasets.decoders.FrameSpotVideo(video_path, num_frames, transform=None, video_frame_to_path_fn=<function get_video_to_frame_path_fn.<locals>.fn>, num_threads_io=0, num_threads_decode=0, num_decode=1, mask_ratio=0.0, mask_tube=2, min_clip_duration=0, decode_float=False)

FrameSpotVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is stored as an image.

Parameters:

• video_path (str | Path) – The path of the video.

• num_frames (int) – The number of frames of the video.

• transform (Optional[Callable], optional) – The transform to apply to the frames.

  Default: None

• video_frame_to_path_fn (Callable[[str, int], int], optional) – A function that maps from the video path and a frame index integer to the file path where the frame is located.

  Default: <function get_video_to_frame_path_fn.<locals>.fn>

• num_threads_io (int, optional) – Controls whether parallelizable io operations are performed across multiple threads.

  Default: 0

• num_threads_decode (int, optional) – Controls whether parallelizable decode operations are performed across multiple threads.

  Default: 0

• num_decode (int, optional) – Number of decode to perform. If > 1, the videos decoded are stored in a list.

  Default: 1

• mask_ratio (float, optional) – Masking ratio for the video.

  Default: 0.0

• mask_ratio – Sequence tube size for masking the video.

• min_clip_duration (float, optional) – The minimum duration of a clip.

  Default: 0

• decode_float (bool, optional) – Whether to decode the clip as float.

  Default: False

Dumb decoders

Used to simulate video decoding, return randomly generated tensors.

class eztorch.datasets.decoders.DumbSoccerNetVideo(video_path, half_path, duration, fps_video, fps, num_frames, min_clip_duration, num_decode, **kwargs)

DumbSoccerNetVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is randomly generated.

Parameters:

• video_path (str | Path) – The path of the video.

• half_path (str | Path) – The path of the half.

• duration (float) – The duration of the video in seconds.

• fps_video (int) – The fps of the video.

• fps (int) – The target fps for the video. This is needed to link the frames to a second timestamp in the video.

• num_frames (int) – The number of frames of the video.

• min_clip_duration (float) – The minimum duration of a clip.

• num_decode (int) – Number of duplicate output clip.

class eztorch.datasets.decoders.DumbSpotVideo(video_path, fps, num_frames, num_decode=1, min_clip_duration=0, **kwargs)

DumbSpotVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is randomly generated.

Parameters:

• video_path (str | Path) – The path of the video.

• fps (int) – The target fps for the video. This is needed to link the frames to a second timestamp in the video.

• num_frames (int) – The number of frames of the video.

• min_clip_duration (float, optional) – The minimum duration of a clip.

  Default: 0

• num_decode (int, optional) – Number of duplicate output clip.

  Default: 1