diff --git a/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Negotiation_of_VSCs.svg b/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Negotiation_of_VSCs.svg
new file mode 100644
index 0000000000..914e74957c
--- /dev/null
+++ b/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Negotiation_of_VSCs.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Resolution_Switching.svg b/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Resolution_Switching.svg
new file mode 100644
index 0000000000..12e1f4bda4
--- /dev/null
+++ b/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/Resolution_Switching.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/sequence_diagram.png b/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/sequence_diagram.png
deleted file mode 100644
index 5bd854464a..0000000000
Binary files a/assets/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle/sequence_diagram.png and /dev/null differ
diff --git a/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle.md b/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle.md
index 05d3d242fd..b72b2deb4e 100644
--- a/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle.md
+++ b/proposals/0296-Update-video-streaming-capabilities-during-ignition-cycle.md
@@ -2,65 +2,210 @@
* Proposal: [SDL-0296](0296-Update-video-streaming-capabilities-during-ignition-cycle.md)
* Author: [Dmytro Boltovskyi](https://github.com/dboltovskyi)
-* Status: **Returned for Revisions**
+* Status: **In Review**
* Impacted Platforms: [Core / iOS / Java Suite]
## Introduction
-The main purpose of this proposal is to make it possible to update video streaming capabilities during the ignition cycle.
+The main purpose of this proposal is to make it possible to update video streaming capabilities (VSC) during the ignition cycle.
## Motivation
-Within the implementation of the [Pixel density and Scale](https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0179-pixel-density-and-scale.md) proposal, new video streaming capabilities parameters `diagonalScreenSize`, `pixelPerInch` and `scale` were introduced.
+Currently there are numerous VSC parameters which may be provided by the HMI through SDL Core to mobile applications.
+These may also include the parameters described in the [Pixel density and Scale](https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0179-pixel-density-and-scale.md) proposal.
+These parameters may be used by projection or navigation mobile applications to properly adjust video streaming data.
+Unfortunately, once these parameters are defined during the HMI initialization process, they can't be updated until the next ignition cycle.
-Usually, these parameters are passed to SDL Core by the HMI in `UI.GetCapabilities` response during HMI initialization process. That means if parameters are defined in a current ignition cycle they cannot be updated until the next cycle.
-However, the HMI is able to change the values of these parameters dynamically, and currently, there is no way to notify SDL Core and Mobile application about the new values.
+This prevents the implementation of use cases related to dynamic resolution switching, such as:
+ - Picture-in-Picture
+ - Preview
+ - Split screen
+ - Collapsed view
+
+The idea of the proposal is to add such possibilities into the SDL ecosystem. The offered solution also allows adjusting the size of UI elements depending on a current screen mode.
## Proposed solution
### SDL Core changes
-In order to make it possible to update the described parameters during a single ignition cycle, existing `OnSystemCapabilityUpdated` notification can be used.
-Currently, `REMOTE_CONTROL` and `APP_SERVICES` capabilities can already be updated through it, so the idea is to extend this list by `VIDEO_STREAMING` capabilities.
+#### Negotiation of VSCs
+
+Available and supported VSCs need to be negotiated between the HMI and mobile application.
-SDL Core gets initial values through `UI.GetCapabilities` request/response.
-After that, these values can be updated by the HMI through `OnSystemCapabilityUpdated` notification.
+1. HMI provides a list of supported by system VSCs within the new `additionalVideoStreamingCapabilities` parameter of `VideoStreamingCapability` struct in `UI.GetCapabilities` response:
-`OnSystemCapabilityUpdated` notification will be transferred to Mobile application if it has set `subscribe` to `true` using `GetSystemCapability` for `systemCapabilityType` = `VIDEO_STREAMING`.
+```xml
+
+
+
+
+
+```
-HMI needs to be notified if a particular application is subscribed on `SystemCapability` updates.
+2. Mobile application obtains these VSCs through `GetSystemCapability` request of `VIDEO_STREAMING` type.
+Within this request mobile application (SDL library) can also subscribe to receive future VSC updates.
-#### HMI API
+3. Mobile application provides supported by application VSCs within new `OnAppCapabilityUpdated` notification.
```xml
-
-
-
- The type of system capability which app is subscribed/unsubscribed to.
-
-
-
-
- Defines whether the application is subscribed to get updates for the defined systemCapabilityType.
-
-
-
-
- ID of the application that relates to this subscription status change.
-
-
+
+ A notification to inform SDL Core that a specific app capability has changed.
+
+ The app capability that has been updated
+
+
+
+
+ Used as a descriptor of what data to expect in this struct. The corresponding param to this enum should be included and the only other param included.
+
+
+ Describes supported capabilities for video streaming
+
+
+
+
+ Contains information about this app's video streaming capabilities.
+
+ Supported streaming resolutions
+
+
+
+
+ Enumerations of all available app capability types
+
+
+
+
+
+
+
+
```
-In case application sends `GetSystemCapability` with the defined `subscribe` parameter, SDL has to send `OnSystemCapabilitySubscription` notification to HMI.
+Details on how a mobile application selects supported VSCs are described in the [Mobile libraries changes](0296-Update-video-streaming-capabilities-during-ignition-cycle.md#mobile-libraries-changes-ios-and-android) section.
+
+4. These VSCs are transferred by SDL Core to the HMI within the same new notification.
+
+
+
+#### Resolution Switching
+
+In order to notify a mobile application of the current VSC, the HMI may use existing `OnSystemCapabilityUpdated` notification of `VIDEO_STREAMING` type.
+
+Mobile application will receive such updates if it has been subscribed to them previously.
+
+There are no changes expected on a protocol level. Only the existing service Stop/Start sequences would be utilized.
+
+
+
+### Mobile libraries changes (iOS and Android)
+
+#### Negotiation of VSCs
+
+Mobile application (SDL library) has to provide back to the HMI filtered list of supported VSCs based on the complete list provided by the HMI.
+
+In order to simplify this mechanism for application developers, there is an idea to add VSC constraints into the Public API.
+
+To generalize both Android and iOS platforms, these are the public methods (or properties) that need to be implemented by an application developer:
+ - min/max `aspectRatio`
+ - min/max `resolution`
+ - min `diagonalScreenSize`
+
+While `diagonalScreenSize` is mandatory, `aspectRatio` and `resolution` are optional. Combining `diagonalScreenSize` with `aspectRatio` or `resolution`, the flow of choosing supported VSCs would look like the following:
+1. Start iterating through all VSCs provided by HMI
+2. If `diagonalScreenSize` provided by HMI is less than developer's constraint, skip this VSC and go to next iteration, otherwise check `aspectRatio` constraint
+3. If `aspectRatio` suits constraint provided - add this VSC to supported VCS list, otherwise check `resolution` constraint
+4. If `resolution` suits constraint provided - add this VSC to supported VSC list, otherwise go to next iteration
+
+***Examples:***
+
+**iOS**
+
+In `SDLStreamingMediaConfiguration`:
+
+```objectivec
+@property (strong, nonatomic, nullable) SDLSupportedStreamingRange *supportedLandscapeStreamingRange;
+@property (strong, nonatomic, nullable) SDLSupportedStreamingRange *supportedPortraitStreamingRange;
+
+@interface SDLSupportedStreamingRange : NSObject
+// The minimum supported normalized aspect ratio, Min value is 1. (0 matches any ratio)
+@property (nonatomic, assign) float minimumAspectRatio;
+// The maximum supported normalized aspect ratio, Min value is 1. (0 matches any ratio)
+@property (nonatomic, assign) float maximumAspectRatio;
+// The minimum supported diagonal screen size in inches, defaults to 0 (0 matches any size)
+@property (nonatomic, assign) float minimumDiagonal;
+// The minimum resolution to support
+@property (nonatomic, assign, nullable) SDLImageResolution *minimumResolution;
+// The maximum resolution to support
+@property (nonatomic, assign, nullable) SDLImageResolution *maximumResolution;
+// Check if the argument is within the [.minimumResolution, .maximumResolution] range
+- (BOOL)isImageResolutionInRange:(SDLImageResolution*)imageResolution;
+// Check if the argument is within the [.minimumAspectRatio, .maximumAspectRatio] range
+- (BOOL)isAspectRatioInRange:(float)aspectRatio;
+<……>
+@end
+```
+
+**Android**
+
+In `VideoStreamManager` add new `startRemoteDisplayStream`:
+
+```java
+public void startRemoteDisplayStream(Context context, Class remoteDisplayClass,
+ VideoStreamingParameters parameters, final boolean encrypted, SupportedStreamingRange streamingRange)
+```
+
+In this way, developers will be able to pass constraints related to specific application directly into `SDLManager`
+where `SupportedStreamingRange`:
+
+```java
+public class SupportedStreamingRange {
+ private Resolution minSupportedResolution;
+ private Resolution maxSupportedResolution;
+ private Double maxScreenDiagonal;
+ private AspectRatio aspectRatio;
+}
+```
+In `SdlRemoteDisplay` implementation by mobile application developers:
+
+Later, when alternative supported resolutions are retrieved from `HMI`, `SupportedStreamingRange` will be used to unpack data:
+
+```java
+private List getSupportedCapabilities(
+ Resolution minResolution,
+ Resolution maxResolution,
+ Double constraintDiagonalMax,
+ AspectRatio ratioRange
+)
+```
+
+#### Resolution Switching
+
+Mobile applications should be able to update the streaming content window to the new VSCs received in `OnSystemCapabilityUpdated`.
+
+In order to do this, application developers would be notified with a callback `onViewResized(width, height)/videoStreamingSizeDidUpdate:(CGSize)displaySize`. It would be called once new data on resolution is retrieved from the HMI and the internal process in the app library of proper component change is finished. This callback is passed to implemented by developers classes responsible for markup, where UI reorganization could be handled.
+
+SDL Manager wouldn't check and validate the VSC that the HMI sends to the app to switch to.
+
+### HMI Integration Guidelines
+
+The HMI Integration Guidelines would need to include the following recommendations for how various screen modes have to be handled.
+
+If the change is a PIP-type change, scale the current video stream to the size you desire without going through this flow. Touches should never be passed to the app. Either a system menu/buttons should be displayed when selected, or the selection should bring the user immediately back to the full-screen app.
+
+If the change is a split-screen type change (and cannot be handled by scaling the original video), then go through this flow. If the app does not support your custom split-resolution, either don't allow the user to put the app in split screen, or scale the video stream to a size that fits the window and use "black bars" to fill the rest of the window. Touches may be passed to the app in certain limited cases.
+
+ * The window must have a width of at least 3" and a height of at least 3".
-#### Sequence diagram
+ * The touches must be offset for the developer so that 0,0 continues to be in the top-left corner of the app window. If these requirements are not met, either a system menu / buttons should be displayed when selected, or the selection should bring the user immediately back to the full-screen app.
-
+### Old applications behavior
-### Mobile libraries changes (Java Suite and iOS)
+In the case where a particular application doesn't support this feature at all, it wouldn't send `OnAppCapabilityUpdated` notification to SDL Core.
+Hence the HMI would treat this application as one which doesn't support dynamic resolution switching and would allow only full-screen mode for it.
-Mobile applications should be able to update the streaming content window to the new video streaming capabilities received in `OnSystemCapabilityUpdated`.
+Nevertheless [alternatives solution](0296-Update-video-streaming-capabilities-during-ignition-cycle.md#alternatives-considered) may be implemented on HMI to handle such applications.
## Potential downsides
@@ -68,10 +213,12 @@ The author was unable to identify any potential downsides.
## Impact on existing code
-* SDL core needs to be updated to support described logic and new HMI API.
-* Java Suite and iOS proxy libraries would require minimal updates since most of the related functionality have been implemented under [Pixel density and Scale](https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0179-pixel-density-and-scale.md) proposal already.
-* HMI may need to be updated to handle new `OnSystemCapabilitySubscription` message from SDL.
+* SDL Core needs to be updated to support described logic and new APIs.
+* Java Suite and iOS app libraries would require updates related to dynamic resolution switching and new APIs.
+* HMI needs to be updated to handle only supported VSCs received from mobile applications.
## Alternatives considered
-The author was unable to identify any alternative solutions.
+Implement scaling of video frame completely on the HMI. This approach has the following drawbacks:
+ - aspect ratio remains the same
+ - size of the touchable UI elements cannot be adjusted for small resolutions and hence the elements become hard to use