mac-au: Add an AUv3 host as filter

[pkviet]

Description

This adds an AUv3 host to obs-studio.
The host is loaded as a filter.
It is compatible with both v2 and v3 AudioUnits which provide audio filters on macOS.
Sidechain is supported.
Resizing of the AU GUI is also supported (that was tricky).
The AU host follows version 3 specifications, which add a bridge allowing to load AU version 2.

Some screenshots:
AUv2 from Apple

AUv3

Sidechain

Sidechain and GUI

Motivation and Context

AU filters are native to macOS. They're a very good alternative to VST on macOS.
macOS ships with a series of good AU filters.

I coded a VST3 host and decided to look at the competition (AU & CLAP).
It seemed easy to write an AU host so for fun, I did it. The architecture is the same as that of obs-vst3.

How Has This Been Tested?

Tested on macOS 26.1.
AU both v2 & v3 work fine, with or without GUI.
Tested the GUIs resize fine.

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[akagumo]

I could help test, how can I help?

[pkviet]

I could help test, how can I help?

In the checks tab, there are test builds (named 'artefacts') which are available (requires a github account).

[akagumo]

I have downloaded and tested some. is this the correct place to report info?
I have Mac Studio M4 Max running 15.7.4. So far Soundtoy, Arturia, iZotope, Valhalla work fine. IK TR5 plugins crash immediately.
Do you need any logs?

[pkviet]

I have downloaded and tested some. is this the correct place to report info?
I have Mac Studio M4 Max running 15.7.4. So far Soundtoy, Arturia, iZotope, Valhalla work fine. IK TR5 plugins crash immediately.
Do you need any logs?

Yes thanks. Report the working AUs.
And for those crashing, a crash log would be useful.
I can only test free AU though. I will try to test the crashing AUs to check whether i can repro

[akagumo]

the normal logs does not show any crash, system crash logs doesn't show anything for obs.
is there are debug mode to turn on and if so, where?

[pkviet]

the normal logs is not show crash, system crash logs doesn't show anything for obs.
is there are debug mode to turn on and if so, where?

Mmh odd. If obs crashes, macOS should deliver a crash report. You don’t have that?
On the vendor website, i see only tr6 not 5. There is a free bundle for 6 that i will test.

[akagumo]

sometimes macOS doesn't record the crash unfortunately. so no logs in the console.

I took a break and came back and IK plugins are not crashing now.
the crashing of the IK AU plugins might have been due to having two vst2 plugins on that were not installed.
I might have installed the newer version of that plugin which is only vst3 now, so no vst2 version.
I think I turned them off before break. testing more plugins now.
will be great when vst3 and AU are finalized into official release of OBS!

[akagumo]

I just don't understand why having a vst2 plugin added and then not installed causes problems.
I have a problem with Touch Portal I am testing that the developer has stated might be related to vst2 plugins.
the TP issue had nothing to do with plugins, but just TP unable to send commands via web socket after a couple hours of streaming.