Threading model pains

[SamWindell]

I wanted to start an additional discussion regarding thread-check in the CLAP wrapper.

Oh boy, it's painful trying to make this all work robustly with CLAP, VST3, AU and misbehaving hosts.

CLAP versus the other formats seems to have slightly different threading models.

I'll just focus on the main-thread (AKA UI thread) for now.

CLAP model

CLAP spec says this about the main-thread: "This is the thread in which most of the interaction between the plugin and host happens. This will be the same OS thread throughout the lifetime of the plug-in."

That means its the same ID, right?

Link

VST3 model

VST3 just says this:

• All initialisation/de-initialisation are done in the UI Thread (sometime referred to as main thread).
• All function exported by the plug-in are called by the host in the UI Thread with the exception of:
  Vst:: IAudioProcessor::process: which could be called in an Audio Thread (sometime refered as realtime/processing thread), avoid any memory allocation!
  Vst:: IAudioProcessor::setProcessing: which could be called in an Audio Thread, avoid any memory allocation!
• All function exported by the host are called by the plug-in in the UI Thread.

Link

AU model

Not sure - documentation seems sparse. It seems they have the optional ability to mutex-protect some functions? Do we need this? SetMutex used in a guard before lots of callbacks such as here. Does that suggest the threading model is the wild west and mutex protecting everything is needed?

The reality

Using CLAP-wrapper's thread-check extension, I got these results in certain situations:

• AU: Logic Pro 11.0.1 can call CLAP destroy() not on main thread
• VST3: Cakewalk, REAPER, Bitwig can call activate()/deactivate() not on main thread
• VST3: ACID Pro can call state->save() not on main thread

There are likely more cases than this.

The CLAP-wrapper thread-check extension uses 2 methods to try and give you the right result if you ask it is_main_thread: check against a thread ID, and check against thread_local 'overrides'.

For me, this is proof enough that VST3/AU hosts do not follow CLAP's model: that it's a single consistent thread.

Are hosts instead following a different model? That the main-thread can change IDs so long as there is only one main-thread at at time. A kind of 'logical' main thread. It's not clear. Maybe they are just calling the functions from different threads with no regard for if it's main-thread or not (hopefully not).

For now, I cannot trust the CLAP-wrapper thread-check extension and so I do not use it when the code detects it's in a VST3/AU wrapper.

Tests using thread-safety detection

I have been experimenting with a thread-safety detection system. Every CLAP main-thread callback enters and leaves a 'detection zone'. https://github.com/floe-audio/Floe/blob/1f7f4c7a7bacf90b4d41b6db5815765f6c02f65b/src/os/threading.cpp#L11-L32

Initial results:

Note these errors simply represent that at the point of trying to enter the 'logical main thread', another thread already got there. So either that callback or this callback is incorrect.

Unless I've got something wrong, these confirm the presence of race-conditions.

gui.destroy #0: multiple main threads

CLAP-as-AUv2 in Pluginval

  0x1103eed23  -[Floe3352205692_nsview dealloc] (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/detail/auv2/wrappedview.asinclude.mm:154)
  0x10f4a2497  ClapGuiDestroy (???)
  0x10f475773  Check (???)

params.get_value #0: multiple main threads

CLAP-as-AUv2 in Pluginval

  0x10f873a7b  ausdk::AUMethodGetProperty (???)
  0x10f85bdf3  ausdk::AUBase::DispatchGetProperty (???)
  0x1103a4a73  GetParameterInfo (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:465)
  0x10f4e2783  ClapParamsGetValue (???)
  0x10f475773  Check (???)

params.get_value #0: multiple main threads

CLAP-as-AUv2 in Logic Pro 10.7.2

Possibly due to either get_value or state.load called from parallel 'main threads'
EDIT: Logic Pro is calling GetParameterInfo from the audio thread. get_value is called from a thread called com.apple.audio. CLAP-wrapper then within GetParameterInfo calls get_value - this could be avoided I think.

params.get_value #0: multiple main threads

CLAP-as-AUv2 in GarageBand 10.4.12

Possibly due to either get_value or state.load called from parallel 'main threads'

on_main_thread #0: multiple main threads

CLAP-as-AUv2 in Pluginval

 0x1102ff257  timerCallback (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/detail/os/macos.mm:69)
  0x1102ff12b  executeDefered (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/detail/os/macos.mm:62)
  0x11030f9f3  onIdle (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:1122)
  0x10f3e325b  ClapOnMainThread (???)
  0x10f3dd773  Check (???)

activate #0: multiple main threads

CLAP-as-AUv2 in Pluginval

  0x10f7db31b  ausdk::AUMethodInitialize (???)
  0x10f7c3073  ausdk::AUBase::DoInitialize (???)
  0x11030b567  Initialize (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:177)
  0x11030b793  activateCLAP (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:927)
  0x1102f1717  activate (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/clap_proxy.cpp:320)
  0x10f3e08e7  ClapActivate (???)
  0x10f3dd773  Check (???)

deactivate #0: multiple main threads

CLAP-as-AUv2 in Pluginval

  0x10f7db4b3  ausdk::AUMethodUninitialize (???)
  0x10f7c2be7  ausdk::AUBase::DoCleanup (???)
  0x11030d1e3  Cleanup (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:561)
  0x11030d56f  deactivateCLAP (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/wrapasauv2.cpp:940)
  0x1102f1747  deactivate (.zig-cache-global/p/N-V-__8AAI0aEAAgIOKJQmqSWnD9V2gBob6pqElBItNJFfsE/src/clap_proxy.cpp:326)
  0x10f3e10e7  ClapDeactivate (???)
  0x10f3dd773  Check (???)

---

These are AU only. Current theory: is CLAP-wrappers on_main_thread() callbacks really the main thread? If not, then that could be hogging the 'logical main thread' and therefore those errors arise? Not sure.

Discussion

1. How could we support thread-check when the threading models are different?
2. Does anyone have any clearer idea of what exactly VST3 or AU use as their threading model?
3. What do we need to do to ensure that people can write a CLAP plugin following only the CLAP spec and it guaranteed no threading issues when wrapped as VST3/AU?

[SamWindell]

Just a quick edit for anyone who happens to read this.

Fixing this solved lots of issues: #395. The get_value call is frequent and therefore we would get contention-failures in lots of places because the logging system I use only captures whatever the 2nd call is that would contend with the main thread.

However, I still get concurrent host calls in the in these hosts: pluginval and "My VST3 HostApplication". "My VST3 HostApplication" is the name that a JUCE-based host uses - see JUCE source code. As it's a non-specific name, it's unclear whether it's host used by real producers and composers or if it's a hobby-scale host (which we don't care so much about supporting). For this reason. I wrap all my main-thread calls in the mutex. It's relatively inexpensive and then I just don't need to worry about this problem any more.