All FFmpeg changes #1

jaypadia-frame · 2023-11-29T17:45:34Z

New branch with all the FFmpeg changes for the libproresdecoder.a integration.

The files in the auxiliary folder are the ones that don't go into the repo, but are needed for compilation. They are

libproresdecoder.a (the .a library from apple, which we don't have here)
proresdecoder.h (include header from apple)
proresdecoder.pc (package config)

New branch with all the FFmpeg changes for the libproresdecoder.a integration. The files in the auxiliary folder are the ones that don't go into the repo, but are needed for compilation. They are - libproresdecoder.a (the .a library from apple, which we don't have here) - proresdecoder.h (include header from apple) - proresdecoder.pc (package config)

jaypadia-frame · 2023-11-29T17:46:10Z

auxiliary/proresdecoder.h

@@ -0,0 +1,148 @@
+/*!


This is the include header we received along with the .a library.

jaypadia-frame · 2023-11-29T17:47:19Z

auxiliary/proresdecoder.h

+    kPRFormat_b64a  = 'b64a'            //!< 4:4:4:4 Full-range (0-65535) ARGB  16-bit big endian per component
+} PRPixelFormat;
+#define PR_PIXEL_FORMAT_DEFINED
+#endif


There compression types are described here: https://developer.apple.com/library/archive/technotes/tn2162/_index.html#//apple_ref/doc/uid/DTS40013070-CH1-TNTAG8

jaypadia-frame · 2023-11-29T17:47:43Z

auxiliary/proresdecoder.h

+    kPRHalfSize     = kPRDownscale_1x2,
+    kPRQuarterSize  = kPRDownscale_2x2
+
+} PRDownscaleMode;


We shall always assume kPRDownscale_1x1 as the others may not apply

jaypadia-frame · 2023-11-29T17:48:57Z

configure

                           for DNN based filters like dnn_processing [no]
  --enable-libopus         enable Opus de/encoding via libopus [no]
  --enable-libplacebo      enable libplacebo library [no]
+  --enable-libproresdecoder enable Apple ProRes decoder library [no]


This is the enable / disable option for FFmpeg compliation. We haven't set it for a default enable

jaypadia-frame · 2023-11-29T17:50:53Z

configure

    }
 }
 enabled libplacebo        && require_pkg_config libplacebo "libplacebo >= 4.192.0" libplacebo/vulkan.h pl_vulkan_create
+enabled libproresdecoder  && require_pkg_config libproresdecoder proresdecoder proresdecoder.h PROpenDecoder


require_pkg_config: package config is present
libproresdecoder: library file name
proresdecoder proresdecoder.h: include header to look for
PROpenDecoder: actual frame decode function name

This follows the template for other / existing codecs.
I had digged into the configure functions that access these params (or rather where these params are passed into & processed by; don't remember the full details about the impact and use of each of these).

jaypadia-frame · 2023-11-29T17:53:36Z

libavcodec/allcodecs.c

 extern const FFCodec ff_ppm_encoder;
 extern const FFCodec ff_ppm_decoder;
 extern const FFCodec ff_prores_encoder;
-extern const FFCodec ff_prores_decoder;


The prores decoder was moved to the end in order to deprioritize this decoder.

jaypadia-frame · 2023-11-29T17:54:01Z

libavcodec/allcodecs.c

 extern const FFCodec ff_libopenjpeg_decoder;
 extern const FFCodec ff_libopus_encoder;
 extern const FFCodec ff_libopus_decoder;
+extern const FFCodec ff_libproresdecoder_decoder;


New decoder - libproresdecoder

jaypadia-frame · 2023-11-29T17:55:48Z

libavcodec/libproresdecoder.h

+{
+    return 0;
+}
+#endif


The lines 45 - 51 should be deleted.

jaypadia-frame · 2023-11-29T17:56:52Z

libavcodec/libproresdecoder.c

+    .priv_data_size = sizeof(ProResDecodeCodec),
+    .init           = initAppleProResDecoder,
+    .close          = closeAppleProResDecoder,
+    FF_CODEC_DECODE_CB(decodeAppleProResFrame),


The 3 main functions to this interface are the ones that init, close, decode

jaypadia-frame · 2023-11-29T17:57:19Z

libavcodec/libproresdecoder.c

+};
+
+const FFCodec ff_libproresdecoder_decoder = {
+    .p.name         = "libproresdecoder",


This is the name of the decoder in the command line.

jaypadia-frame · 2023-11-29T17:59:54Z

auxiliary/proresdecoder.h

+ * @return A reference to the instantiated decoder, or NULL on failure.
+ */
+PRDecoderRef
+PROpenDecoder(int numThreads, void (*threadStartupCallback)());


==
Important NOTE
We haven't yet explored the numThreads opportunity provided by the decoder itself.
We are only use the FFmpeg threading per frame feature.

jaypadia-frame · 2023-11-29T18:03:38Z

auxiliary/proresdecoder.h

+        int compressedFrameSize,
+        PRPixelBuffer* destinationBuffer,
+        PRDownscaleMode downscaleMode,
+        bool ignoreAlpha);


This needs the decoder reference, pointer to and the size of the compressed video frame, output buffer, downscale mode (always set to 1x1) and whether to consider the alpha (we always look for it) - that helps us define the context in our local header - libavcodec/libproresdecoder.h::ProResDecodeCodec

jaypadia-frame · 2023-11-29T18:04:56Z

libavcodec/libproresdecoder.c

+
+    dstdecodectx->m_width = srcdecodectx->m_width; 
+    dstdecodectx->m_height = srcdecodectx->m_height;
+    dstdecodectx->pix_fmt = srcdecodectx->pix_fmt;


just copying over everything that I could I think of that would need to be replicated.

jaypadia-frame · 2023-11-29T18:07:23Z

libavcodec/libproresdecoder.c

+    proresdecodecctx->m_height = avctx->height; 
+    // For now the downscale mode is universally 1x1
+    proresdecodecctx->downscale_mode = kPRDownscale_1x1;
+


For the reading of the tags and setting up accordingly, we follow the same approach as the FFmpeg prores decoder does: https://github.com/jaypadia-frame/FFmpeg/blob/master/libavcodec/proresdec2.c#L134

jaypadia-frame · 2023-11-29T18:09:18Z

libavcodec/libproresdecoder.c

+    return 0;
+};
+
+int decodeAppleProResFrame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) {


The interface here is the same as other decoders' decode frame function.

jaypadia-frame · 2023-11-29T18:10:35Z

libavcodec/libproresdecoder.c

+
+int decodePacket(ProResDecodeCodec* decoderctx, AVPacket* pkt, AVFrame* framePtr, bool drain) {
+    if (drain) {
+        // return AVError_create(AVErrorCode_kErrorEOF);


These return AVError_create(AVErrorCode_kErrorEOF); were referenced from Harry's team's changes. They should be cleaned up, I guess. We should have readable return strings that enumerate these return value.
TODO for Jay.

jaypadia-frame · 2023-11-29T18:12:28Z

libavcodec/libproresdecoder.c

+    memset(framePtr->data[0], 0, framePtr->linesize[0] * decoderctx->m_height); // Y
+    memset(framePtr->data[1], 0, framePtr->linesize[1] * decoderctx->m_height); // U
+    memset(framePtr->data[2], 0, framePtr->linesize[2] * decoderctx->m_height); // V 
+


The code from here on below converts the packed YUV outputs in Apple's formats (the ones listed in that include header) to more generic (or rather standardized(?)) FFmpeg supported YUV formats.
Ques: Should. this code be another function?

jaypadia-frame · 2023-11-29T18:13:36Z

libavcodec/libproresdecoder.c

+            }
+        }
+    }
+    free(dataPtr);


I believe I had forgotten this earlier which caused the memory leak. 🤦 🤦 🤦

jaypadia-frame · 2023-11-29T18:14:36Z

libavcodec/libproresdecoder.c

+        return -2; // error code to be updated
+    }
+
+    int rowBytes = PRBytesPerRowNeededInPixelBuffer(decoderctx->m_width, decoderctx->pix_fmt, decoderctx->downscale_mode);


Good to have this as we don't have to do this Maths. Although, with the formats explained by Apple in the link above, we could have done that.

jaypadia-frame commented Nov 29, 2023

View reviewed changes

auxiliary/proresdecoder.h

@@ -0,0 +1,148 @@

/*!

Copy link

Owner Author

jaypadia-frame Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the include header we received along with the .a library.

jaypadia-frame commented Nov 29, 2023

View reviewed changes

libavcodec/libproresdecoder.h

{

return 0;

}

#endif

Copy link

Owner Author

jaypadia-frame Nov 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lines 45 - 51 should be deleted.

jaypadia-frame commented Nov 29, 2023

View reviewed changes

All FFmpeg changes #1

Are you sure you want to change the base?

All FFmpeg changes #1

Uh oh!

Conversation

jaypadia-frame commented Nov 29, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaypadia-frame Nov 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaypadia-frame Nov 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jaypadia-frame Nov 29, 2023 •

edited

Loading

jaypadia-frame Nov 29, 2023 •

edited

Loading