feat: av1 by SomeBody16 · Pull Request #4500 · webex/webex-js-sdk

SomeBody16 · 2025-09-25T13:26:50Z

COMPLETES SPARK-728169

This pull request addresses

The need to add AV1 codec support to the Webex JS SDK multistream functionality. AV1 (AOMedia Video 1) is a modern, open-source video codec that provides superior compression efficiency compared to H.264, enabling better video quality at lower bitrates. This enhancement allows the SDK to leverage AV1 codec capabilities for improved video streaming performance in multistream scenarios.

by making the following changes

Architecture Improvements

Introduced Codec Helper Pattern: Created a factory-based architecture for handling different video codecs (AV1 and H.264) through dedicated helper classes
- MediaCodecHelper - Abstract base class defining the codec helper interface
- MediaCodecHelperAV1 - AV1-specific implementation with maxPicSize and frame size calculations
- MediaCodecHelperH264 - H.264-specific implementation maintaining existing behavior
- MediaCodecHelperFactory - Factory class for creating appropriate codec helpers

Core Functionality

AV1 Codec Support: Added comprehensive AV1 codec parameter handling including:
- Level index, tier, maxWidth, maxHeight, maxPicSize, and maxDecodeRate parameters
- Picture size to frame size conversion (rounded to 16x16 macroblock units)
- Resolution-specific AV1 codec parameters for 180p, 360p, 540p, 720p, and 1080p
- Integration with @webex/internal-media-core AV1Codec class
Enhanced Media Request Manager: Refactored mediaRequestManager.ts to:
- Support codec-specific media request degradation
- Handle maxPicSize updates dynamically
- Improve resolution handling across different codecs
- Reduce complexity by delegating codec-specific logic to helper classes
Remote Media Enhancements: Updated remote media handling to:
- Support new remote video resolutions with codec-aware constraints
- Handle maxPicSize changes in media requests
- Improved receive slot management with codec-specific parameters

Code Quality

Type Safety: Added comprehensive TypeScript types for AV1 codec info and parameters
Constants Organization: Consolidated codec-related constants in a dedicated constants file
Test Coverage: Added 600+ lines of new unit tests covering:
- Media request manager with AV1 codec scenarios
- Remote media group functionality
- Remote media manager behavior
- Utility functions for codec handling

Package Updates

Updated @webex/media-helpers and @webex/plugin-meetings package versions
Cleaned up unused dependencies in yarn.lock

Change Type

New feature (non-breaking change which adds functionality)
Internal code refactor

The following scenarios were tested

Automated Tests

✅ All existing unit tests pass
✅ New unit tests added for:
- AV1 codec helper functionality (codec info generation, media request degradation, bitrate calculations)
- Media request manager with AV1 codec support (maxPicSize handling, resolution degradation)
- Remote media group with AV1 parameters
- Remote media manager with codec-specific behavior
- Utility functions for picSize calculations and frame size conversions

Manual Testing

✅ Verified AV1 codec parameter selection based on maxPicSize constraints
✅ Tested media request degradation with different resolution levels (180p-1080p)
✅ Validated maxPicSize updates propagate correctly through the system
✅ Confirmed backward compatibility with existing H.264 codec flows
✅ Tested codec factory correctly instantiates appropriate helper classes

Integration Scenarios

✅ Media request creation with AV1 codec info
✅ Dynamic resolution changes during active multistream sessions
✅ MaxPicSize enforcement when receiving video streams
✅ WCME codec info generation for AV1 streams

The GAI Coding Policy And Copyright Annotation Best Practices

I certified that

I have read and followed contributing guidelines
I discussed changes with code owners prior to submitting this pull request
I have not skipped any automated checks
All existing and new tests passed
I have updated the documentation accordingly

Summary Statistics

Files Changed: 23 files
Lines Added: ~2,087
Lines Removed: ~686
Net Impact: +1,401 lines
New Test Coverage: 1,000+ lines of unit tests

Make sure to have followed the contributing guidelines before submitting.

…ucing media codec helpers

…utions and maxPicSize

…improve resolution handling

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts

packages/@webex/plugin-meetings/src/meeting/index.ts

packages/@webex/plugin-meetings/src/meetings/util.ts

packages/@webex/plugin-meetings/src/multistream/codec/constants.ts

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.factory.ts

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.factory.ts

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.av1.ts

packages/@webex/plugin-meetings/src/multistream/receiveSlot.ts

packages/@webex/plugin-meetings/src/multistream/remoteMedia.ts

antsukanova · 2025-10-31T10:55:58Z

packages/@webex/plugin-meetings/src/multistream/remoteMedia.ts

+        codecInfo: mediaCodecHelper.getCodecInfo({
+          maxFs: this.getEffectiveMaxFs(),
+          maxPicSize: this.getEffectiveMaxPicSize(),
+        }),


It should not be possible that maxFs and maxPicSize will be passed at the same time, right? Thinking in the way that I feel, here we need to think about more abstract passing data according to what codec we are currently working with.

Something like

codecInfo = helper.getCodecInfo(this.buildCodecConstraints()); private buildCodecConstraints(): CodecConstraints { const codec = this.options.preferredCodec; if (codec === 'av1') { const maxPicSize = this.getEffectiveMaxPicSize(); return maxPicSize ? {maxPicSize} : {}; } // default h264 or do if for h264 const maxFs = this.getEffectiveMaxFs(); return maxFs ? {maxFs} : {}; }

Here is tricky because at one side I want to delegate all codec checks to mediaCodecHelper but at the same time here we need different info for different codecs

Maybe solution would be instead of calling two getEffectiveX() methods to pass references to it?

const codecInfo = mediaCodecHelper.getCodecInfo({ maxFs: () => this.getEffectiveMaxFs(), maxPicSize: () => this.getEffectiveMaxPicSize(), })

So won't calculate useless stuff but av1/h264 will call method that they want to use?

packages/@webex/plugin-meetings/src/multistream/remoteMediaManager.ts

packages/@webex/plugin-meetings/src/multistream/remoteMedia.ts

…ilability checks

… in media codec helper

…ling to use string literals

… H.264 and AV1

SomeBody16 · 2025-11-03T14:32:09Z

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts

+export type {
+  /** @deprecated use CodecInfo from @webex/plugin-meetings/src/codec/types instead */
+  CodecInfo,
+} from './codec/types';


Because this types are used by children repositories, we need to have backward compatibility for them.

I'm not sure if this is the correct way to deprecate things for the SDK. Should we take care to update this or at least create a task for such changes? Could the SDK team address these deprecations?

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.ts

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.av1.ts

packages/@webex/plugin-meetings/src/multistream/remoteMediaManager.ts

…axPicSize

…ion handling

antsukanova · 2025-11-25T09:39:54Z

packages/@webex/plugin-meetings/src/multistream/codec/constants.ts

+  },
+  '1080p': {
+    maxPicSize: 2_359_296,
+    levelIdx: 9,


So far, I believe @k-wasniowski said that we won't support the different levelIdx, ot I'm missing something?

antsukanova · 2025-11-25T09:43:27Z

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.av1.ts

+
+    return [
+      WcmeCodecInfo.fromAv1(
+        45,


could we have any constant for this param here?

antsukanova · 2025-11-25T09:44:48Z

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.av1.ts

+      resolution = PANE_SIZE_TO_RESOLUTION[paneSize];
+    } else {
+      LoggerProxy.logger.warn(
+        `MediaCodecHelperAV1#getMaxPicSize --> unsupported paneSize: ${paneSize}, using "medium" instead`


nit: you can use in log PANE_SIZE_TO_RESOLUTION.medium

antsukanova · 2025-11-25T11:19:21Z

packages/@webex/plugin-meetings/src/multistream/codec/mediaCodecHelper.h264.ts

+    if (mr.codecInfo?.codec !== 'h264') {
+      return 0;
+    }


I'm not sure if returning 0 silently is good idea here for an incorrect codec. It would be good to log info that unexpected mr.codecInfo?.codec is passed here.

antsukanova · 2025-11-25T11:26:31Z

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts

+export type {
+  /** @deprecated use CodecInfo from @webex/plugin-meetings/src/codec/types instead */
+  CodecInfo,
+} from './codec/types';


I'm not sure if this is the correct way to deprecate things for the SDK. Should we take care to update this or at least create a task for such changes? Could the SDK team address these deprecations?

antsukanova · 2025-11-25T11:27:46Z

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts


-type ClientRequestsMap = {[key: MediaRequestId]: MediaRequest};
-
+// eslint-disable-next-line import/prefer-default-export


Why do you need this comment for ESLint? It wasn't here before

antsukanova · 2025-11-25T11:45:22Z

packages/@webex/plugin-meetings/src/multistream/codec/types.ts

+  getCodecInfo(options: {
+    getMaxFs?: () => number;
+    getMaxPicSize?: () => number;
+  }): CodecInfo | undefined;


I feel we need to improve this part here...The shared signature in MediaCodecHelper forces every helper to accept both getMaxFs and getMaxPicSize, even though only one makes sense per codec.
Can we have here generic helper interface for getCodecInfo and export concrete types MediaCodecHelperH264?

something like

interface MediaCodecHelper<TOptions, TCodecInfo> {getCodecInfo(options: TOptions): TCodecInfo | undefined; ... }

antsukanova · 2025-11-25T11:49:23Z

packages/@webex/plugin-meetings/src/multistream/remoteMedia.ts

+        codecInfo: mediaCodecHelper.getCodecInfo({
+          getMaxFs: () => this.getEffectiveMaxFs(),
+          getMaxPicSize: () => this.getEffectiveMaxPicSize(),
+        }),


Can we have here after generic helper interface for getCodecInfo impromevents something like:

const codecInfo = mediaCodecHelper.codec === 'h264' ? mediaCodecHelper.getCodecInfo({getMaxFs: () => this.getEffectiveMaxFs()}) : mediaCodecHelper.getCodecInfo({getMaxPicSize: () => this.getEffectiveMaxPicSize()});

feat: av1 attempt

0e4c10d

SomeBody16 changed the title ~~feat: av1 attempt~~ feat: av1 Sep 25, 2025

fnowakow added 8 commits September 26, 2025 13:09

fix: old media request

8227743

feat: preferred av1 codec

a6d44d9

feat: enhance media request handling for AV1 codec support

a3b2525

feat: add maxPicSize handling and utility functions for media requests

c247499

refactor: remove unused import from constants file

3f74f74

refactor: consolidate codec handling by removing constants and introd…

2008cf1

…ucing media codec helpers

feat: update media request handling to support new remote video resol…

19f1a54

…utions and maxPicSize

feat: enhance media request manager to handle maxPicSize updates and …

a442b1f

…improve resolution handling

SomeBody16 commented Oct 29, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts Show resolved Hide resolved

fix: update getEffectiveMaxPicSize to correctly return maxPicSize if set

9a04467

k-wasniowski reviewed Oct 30, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/meeting/index.ts Outdated Show resolved Hide resolved

k-wasniowski reviewed Oct 30, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/meetings/util.ts Outdated Show resolved Hide resolved

antsukanova reviewed Oct 30, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/multistream/codec/constants.ts Outdated Show resolved Hide resolved

antsukanova reviewed Oct 30, 2025

View reviewed changes

antsukanova reviewed Oct 31, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts Outdated Show resolved Hide resolved

packages/@webex/plugin-meetings/src/multistream/mediaRequestManager.ts Show resolved Hide resolved

antsukanova reviewed Oct 31, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/multistream/remoteMediaManager.ts Show resolved Hide resolved

szotrabh reviewed Oct 31, 2025

View reviewed changes

packages/@webex/plugin-meetings/src/multistream/remoteMedia.ts Outdated Show resolved Hide resolved

fnowakow added 9 commits November 3, 2025 12:06

fix: revert useless moved type

b7a78a1

refactor: update codec handling to use web-capabilities for codec ava…

a7387cb

…ilability checks

refactor: media codec helper improvements

73148b1

refactor: rename bitrate function to improve clarity and update usage…

4eafad3

… in media codec helper

refactor: rename picSizeToFrameSize to pixelsToMacroblocks

3d5b1bb

refactor: update function names in receiveSlot for clarity

5a6ff63

refactor: replace string literals with enum for remote video resolutions

4137ec1

refactor: deprecate old types and update remote video resolution hand…

9d7222c

…ling to use string literals

refactor: update codec availability checks to use WebCapabilities for…

c69a904

… H.264 and AV1

SomeBody16 commented Nov 3, 2025

View reviewed changes

antsukanova reviewed Nov 6, 2025

View reviewed changes

fnowakow added 6 commits November 12, 2025 13:31

fix: addressed PR comments

628569e

refactor: update codecInfo method to accept functions for maxFs and m…

486a5b6

…axPicSize

refactor: update tests

1cd822d

refactor: enhance media codec helpers and update remote media resolut…

59d0ef8

…ion handling

test: add unit tests for AV1 and H264 media codec helpers

1c80746

test: add stubs for video codec capability checks in meeting tests

758b704

antsukanova reviewed Nov 25, 2025

View reviewed changes


		type ClientRequestsMap = {[key: MediaRequestId]: MediaRequest};

		// eslint-disable-next-line import/prefer-default-export

Conversation

SomeBody16 commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

COMPLETES SPARK-728169

This pull request addresses

by making the following changes

Architecture Improvements

Core Functionality

Code Quality

Package Updates

Change Type

The following scenarios were tested

Automated Tests

Manual Testing

Integration Scenarios

The GAI Coding Policy And Copyright Annotation Best Practices

I certified that

Summary Statistics

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

antsukanova Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Comments

SomeBody16 commented Sep 25, 2025 •

edited

Loading

antsukanova Oct 31, 2025 •

edited

Loading