Add support for windows application loopback audio capture to specify/exclude process id

[nitedani]

Windows supports capturing/excluding the audio of a specific process/process tree.
I would like to set the ProcessLoopbackParams that is passed to the windows api.
Related:
https://docs.microsoft.com/en-us/samples/microsoft/windows-classic-samples/applicationloopbackaudio-sample/
https://github.com/microsoft/Windows-classic-samples/tree/main/Samples/ApplicationLoopback

[mackron]

This request is reasonable. The documentation you linked to uses ActivateAudioInterfaceAsync() which is only used in the UWP build in miniaudio which had me worried that I wouldn't be able to do it with the regular desktop build. However, it looks like it is indeed possible.

Note that development has been slow on miniaudio lately so I can't give a timeframe on when I'll get to this.

---

Note to self when implementing this. The following functions need to be updated to take the PROPVARIANT object:

• ma_IMMDevice_Activate() in ma_context_get_IAudioClient_Desktop__wasapi()
• ActivateAudioInterfaceAsync() in ma_context_get_IAudioClient_UWP__wasapi()

Issue from the NAudio project on how to use the PROPVARIANT object: naudio/NAudio#878

[mackron]

I've gone ahead and added experimental support for this. This requires Windows 10 build 20348, but unfortunately I've only got access to build 19044 so I'm not able to reliably test this.

To use this, configure it in the device config:

deviceConfig.wasapi.loopbackProcessID = 1234;
deviceConfig.wasapi.loopbackProcessExclude = true; /* true = exclude; false = include. Defaults to false. */

If anybody is able to give that a test I'd appreciate it.

[DanielMcPherson]

I have tested this on Windows 11 Version 10.0.22000 Build 22000. Unfortunately, it does not seem to be working. I played audio from multiple sources simultaneously (two web browsers and Windows Media Player). Using the Microsoft example @nitedani linked to, I can use the Media Player's PID to record just Media Player's audio, or to record everything except Media Player. The dev branch of miniaudio records all three audio sources, not including or excluding the single PID.

[mackron]

That's unfortunate. I might need the community's help with this one since I lack OS support to test this properly on my side.

[mackron]

I've pushed a potential fix for this. I was forgetting to set the virtual device ID which is required for process-specific loopback. I've also made it so process-specific loopback will be ignored if an explicit device ID is requested because that conflicts with the requirement for the virtual device ID.

Another thing to note is that initialization will fail if attempting to specify a process ID when running on an unsupported version of Windows. I was debating whether or not fall back to regular loopback mode, but I was thinking that if a program has requested a specific process ID that it's probably important to the program, and it's probably best to explicitly let them know that it didn't work rather than just silently falling back.

[DanielMcPherson]

I'm getting an ma_device_init_ex failure on Windows 11. It happens whether I specify a real PID that is playing audio, or just 1234, and whether I set loopbackProcessExclude to true or false. Is there something in the config setup that I'm missing?

    deviceConfig = ma_device_config_init(ma_device_type_loopback);
    deviceConfig.wasapi.loopbackProcessID = 1234;
    deviceConfig.wasapi.loopbackProcessExclude = true; /* true = exclude; false = include. Defaults to false. */
    deviceConfig.capture.pDeviceID = NULL; /* Use default device for this example. Set this to the ID of a _playback_ device if you want to capture from a specific device. */
    deviceConfig.capture.format = encoder.config.format;
    deviceConfig.capture.channels = encoder.config.channels;
    deviceConfig.sampleRate = encoder.config.sampleRate;
    deviceConfig.dataCallback = data_callback;
    deviceConfig.pUserData = &encoder;

    result = ma_device_init_ex(backends, sizeof(backends) / sizeof(backends[0]), NULL, &deviceConfig, &device);
    if (result != MA_SUCCESS) {
        printf("Failed to initialize loopback device.\n");
        return -2;
    }

[mackron]

That configuration is fine. I think I might know what's going on, and if I'm right, it's going to get very messy and complicated due to compatibility with Windows Vista and 7. The regular desktop Win32 build uses the IMMDevice API to connect to an audio endpoint with WASAPI. To connect to a device you need to retrieve a handle to it via IMMDeviceEnumerator::GetDevice() (https://docs.microsoft.com/en-us/windows/win32/api/mmdeviceapi/nf-mmdeviceapi-immdeviceenumerator-getdevice). This is supported from Vista and the beginning of WASAPI. I think the problem is that the IMMDeviceEnumerator class only works with actual audio endpoints and not virtual devices as is required for the use of process-specific loopback.

The alternative is to use ActivateAudioInterfaceAsync(), which I suspect will work, and is what's used in the Microsoft examples, but the problem is that I think that's only supported starting with Windows 8. I have code in place to use ActivateAudioInterfaceAsync() for the UWP build, but if I enable that globally, the WASAPI backend won't work with Windows Vista and 7 which is not an acceptable trade off.

I've pushed a change to the dev branch that forces the UWP build for the normal desktop build. This is just a hacked together experiment for now, but it will result in ActivateAudioInterfaceAsync() being used instead of IMMDevice. I'm not sure if that'll get process loopback stuff working, and it will lack proper device enumeration (miniaudio currently assumes default devices with the UWP build). Use like this:

#define MA_FORCE_UWP // <-- Add this
#define MINIAUDIO_IMPLEMENTATION
#include "miniaudio.h"

[DanielMcPherson]

Unfortunately, this is not working for me. I get a compile error for miniaudio.h line 20835
'HRESULT (LPCWSTR,const IID *,PROPVARIANT *,ma_IActivateAudioInterfaceCompletionHandler *,ma_IActivateAudioInterfaceAsyncOperation **)': cannot convert argument 2 from 'const IID' to 'const IID *'

I can fix the compile error by changing MA_IID_IAudioClient to &MA_IID_IAudioClient, but ma_device_init_ex still fails, returning error -2.

[mackron]

Thanks. I forgot to test the C++ build. I've pushed another potential fix for that init error you're getting. One of these days we'll get it working!

[DanielMcPherson]

Latest dev version fixes the compile error. However, I'm getting error -100 from ma_device_init_ex.

In ma_device_init_internal__wasapi line 21220

hr = ma_IAudioClient_QueryInterface(pData->pAudioClient, &MA_IID_IAudioClient2, (void**)&pAudioClient2);

is returning E_NOINTERFACE No such interface supported.

Here's my setup code:

    encoderConfig = ma_encoder_config_init(ma_encoding_format_wav, ma_format_s16, 2, 44100);
    if (ma_encoder_init_file(argv[1], &encoderConfig, &encoder) != MA_SUCCESS) {
        printf("Failed to initialize output file.\n");
        return -1;
    }
    deviceConfig = ma_device_config_init(ma_device_type_loopback);
    deviceConfig.wasapi.loopbackProcessID = 1234;
    deviceConfig.wasapi.loopbackProcessExclude = true; /* true = exclude; false = include. Defaults to false. */
    deviceConfig.capture.pDeviceID = NULL; /* Use default device for this example. Set this to the ID of a _playback_ device if you want to capture from a specific device. */
    deviceConfig.capture.format = encoder.config.format;
    deviceConfig.capture.channels = encoder.config.channels;
    deviceConfig.sampleRate = encoder.config.sampleRate;
    deviceConfig.dataCallback = data_callback;
    deviceConfig.pUserData = &encoder;

    result = ma_device_init_ex(backends, sizeof(backends) / sizeof(backends[0]), NULL, &deviceConfig, &device);