Skip to content

Add R(2+1)D, S3D, and MViT V2 as feature extractors#306

Merged
ekosman merged 2 commits intomainfrom
copilot/add-feature-extractors
Feb 21, 2026
Merged

Add R(2+1)D, S3D, and MViT V2 as feature extractors#306
ekosman merged 2 commits intomainfrom
copilot/add-feature-extractors

Conversation

Copy link
Contributor

Copilot AI commented Feb 21, 2026

The codebase only supported older video feature extractors (C3D, MFNET, 3D-ResNet). This adds three modern, torchvision-backed alternatives with Kinetics-400 pretrained weights — no separate weight downloads required.

New extractors

Key Model Output dim Input size
r2plus1d R(2+1)D-18 — decomposed spatiotemporal convolutions 512 112×112
s3d S3D — separable 3D Inception-style 1024 224×224
mvit MViT V2 Small — multiscale video transformer 768 224×224

Changes

  • network/r2plus1d.py, network/s3d.py, network/mvit.py — thin wrapper classes that strip the classification head and expose the backbone as a feature extractor
  • utils/load_model.py — adds loading logic for the three new models; feature_extractor_path is now str | None — omitting it triggers torchvision's auto-download of pretrained weights (only c3d, mfnet, r3d101, r3d152 still require an explicit path)
  • utils/utils.py — adds correct normalization and resize/crop transforms per model
  • utils/types.py — extends FeatureExtractor union
  • feature_extractor.py, AD_live_prediction.py — new choices exposed in CLI args

Usage

# Uses Kinetics-400 pretrained weights automatically — no path needed
python feature_extractor.py --model_type mvit --dataset_path /data/videos --save_dir features/

# Or supply custom weights
python feature_extractor.py --model_type r2plus1d --pretrained_3d /weights/my_r2plus1d.pth ...

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Co-authored-by: ekosman <10260165+ekosman@users.noreply.github.com>
Copilot AI changed the title [WIP] Add more feature extractors using up-to-date models Add R(2+1)D, S3D, and MViT V2 as feature extractors Feb 21, 2026
Copilot AI requested a review from ekosman February 21, 2026 23:33
@ekosman ekosman marked this pull request as ready for review February 21, 2026 23:33
@ekosman ekosman merged commit a4f55f1 into main Feb 21, 2026
4 checks passed
@ekosman ekosman deleted the copilot/add-feature-extractors branch February 21, 2026 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants