[FEAT] Feature Request: Support for the Unity Engine

[EitanWong]

Requirements

• I have searched the existing issues to ensure this feature has not already been requested.
• I have clearly described the problem and the proposed solution.
• I have considered alternative solutions.

1. Is your feature request related to a problem? Please describe.

When the SoundFlow.dll (compiled for .NET 8.0) is imported into a Unity project, the following two critical assembly resolution errors occur, preventing the library from loading:

Assemby 'Assets/Plugins/UniAudioKit/Runtime/Plugins/lib/net8.0/SoundFlow.dll' will not be loaded due to errors:
Unable to resolve reference 'System.Runtime.Intrinsics'. Is the assembly missing or incompatible with the current platform?
Reference validation can be disabled in the Plugin Inspectlor.
Unable to resolve reference 'System.Text.Json'. Is the assembly missing or incompatible with the current platform?
Reference validation can be disabled in the Plugin Inspector.

These errors are caused by direct dependencies on two APIs that are not fully compatible with Unity's .NET runtime environment:

System.Runtime.Intrinsics: Used extensively in SoundFlow.Utils.MathHelper for hardware-accelerated (SIMD) calculations for FFT and windowing functions.
System.Text.Json: Used for JSON serialization tasks. While modern Unity versions have improved support, it is not universally available or is often less preferred than established solutions within the ecosystem.

2. Describe the Solution You'd Like

The proposed solution is to refactor the library to abstract these specific dependencies behind interfaces, allowing different implementations to be injected based on the host platform.

1. Abstract System.Runtime.Intrinsics:

• Action: Create an IMathOperations interface for methods that currently use SSE/AVX (e.g., Fft, HammingWindow).

• Default Implementation: A NetCoreIntrinsicsProvider class will contain the existing, high-performance SSE/AVX code.

• Unity Implementation: A UnityBurstProvider class will implement the same interface using Unity's Burst compiler and Unity.Burst.Intrinsics for equivalent, high-performance SIMD operations. A fallback to the existing scalar logic can be used for non-Burst scenarios.

2. Abstract System.Text.Json:

• Action: Create an IJsonSerializer interface with Serialize(T obj) and Deserialize(string json) methods.

• Default Implementation: A SystemTextJsonProvider class will use System.Text.Json to handle serialization for standard .NET environments.

• Unity Implementation: A UnityJsonProvider class will implement the interface using a Unity-compatible solution, such as UnityEngine.JsonUtility (for simple cases) or a third-party library like Newtonsoft.Json (which is standard in many Unity projects).

This dual-interface approach resolves all compatibility errors while making the entire library more modular, maintainable, and extensible.

3. Describe Alternatives You've Considered

The only alternative is a manual fork and rewrite of the library's internals, which is unsustainable. It would prevent users from receiving updates and would fragment the codebase. The proposed interface-based solution is the clean way to support multiple platforms.

4. Proposed API (if applicable)

The core idea is to use dependency injection to provide the correct implementation at runtime.

Global Configuration:

// In a central configuration class for SoundFlow
public static class SoundFlowConfig
{
    // Set the provider at app startup. Defaults to .NET 8 versions.
    public static IMathOperations MathProvider { get; set; } = new NetCoreIntrinsicsProvider();
    public static IJsonSerializer JsonSerializer { get; set; } = new SystemTextJsonProvider();
}

Example JSON Interface:

// The new abstraction layer for serialization
public interface IJsonSerializer
{
    string Serialize<T>(T value);
    T Deserialize<T>(string json);
}

// Default implementation
public class SystemTextJsonProvider : IJsonSerializer
{
    public string Serialize<T>(T value) => System.Text.Json.JsonSerializer.Serialize(value);
    public T Deserialize<T>(string json) => System.Text.Json.JsonSerializer.Deserialize<T>(json);
}

// Unity-specific implementation
public class UnityJsonProvider : IJsonSerializer
{
    public string Serialize<T>(T value) => UnityEngine.JsonUtility.ToJson(value);
    public T Deserialize<T>(string json) => UnityEngine.JsonUtility.FromJson<T>(json);
}

This API design is clean, flexible, and completely decouples the core library logic from the platform-specific implementations.

5. Benefits

• Full Unity Compatibility: Directly solves the blocking issues, making SoundFlow available to a vast new user base.

• Enhanced Extensibility: Developers are no longer locked into System.Text.Json. They could, for instance, create their own NewtonsoftJsonProvider if they prefer its features, even in a standard .NET project.

• Future-Proof Architecture: This modular design makes it trivial to add support for other platforms (e.g., MAUI, Godot) in the future by simply adding a new implementation of the interfaces.

• Performance without Compromise: The use of Burst for math operations ensures that Unity developers still get the best possible performance.

6. Potential Drawbacks/Challenges

• Initial Refactoring Effort: There is an upfront cost to refactoring MathHelper and any classes that use System.Text.Json.

• Feature Parity: If the library uses advanced features of System.Text.Json (e.g., custom converters, reference handling), care must be taken to ensure the chosen Unity-compatible serializer (JsonUtility or Newtonsoft.Json) can support them.

• Build/Distribution Strategy: A clear strategy for packaging the core library and the platform-specific provider packages will be needed (e.g., via separate NuGet packages or Unity packages).

7. Additional Context

The provided code for MathHelper is a clear example of where the System.Runtime.Intrinsics abstraction is needed. A similar pattern of direct dependency likely exists for System.Text.Json and would benefit equally from this proposed refactoring. Applying this strategy to both dependencies is the key to creating a truly portable and more robust SoundFlow library.

[LSXPrime]

You know, after more than six years of deep diving into Unity, I've honestly never hit a wall with its built-in AudioSource components and effects. Because of that, I really struggle to see a compelling use for something extra like SoundFlow in Unity, unless maybe you're building some kind of dedicated audio editing suite. Even then, frankly, Unity might introduce more overhead than it's worth for that specific task.

My real motivation for building SoundFlow was actually to fill a gap I saw in the broader .NET ecosystem – specifically, getting truly cross-platform audio without drowning in dependency hell. But when it comes to Unity, its audio ecosystem already feels pretty solid. You've got the native components, tons of external packages, and even professional tools like FMOD available.

So, that's why I'm genuinely curious, what unique value or functionality do you envision SoundFlow bringing to the Unity development environment?

[EitanWong]

What I might need is the ability to process microphone recordings using WebRTC audio processing, which Unity doesn't natively implement. I'd like to play around with Sound Flow more to make a better recording tool plugin, including comprehensive audio processing capabilities.

[LSXPrime]

I'm not saying this won't be implemented, but if it's a critical need, you can integrate the WebRTC APM extension directly from source file AudioProcessingModule.cs without depending on the full SoundFlow library. This would primarily involve removing the ISoundModifier interface and adapting the AudioEngine.OnAudioProcessed += HandleAudioEngineProcessedForAec; callback to work directly with Unity's Microphone.Start(...) API. Otherwise, I'll address this after feature requests #26 and #60 have been completed.

[EitanWong]

Got it, thanks for the pointer. I'll give it a shot and try to implement it with AudioProcessingModule.cs.

[xue-fei]

Continuously monitor this issue, Unity's DLL dependencies are like hell……

[LSXPrime]

While I was preparing to address that, I remembered that Unity's latest version was using C# 7.3 or 8.0 back then. I've now found they've updated to C# 9.0. My library is using C# 12.0, and I am planning to upgrade to C# 14.0 when a stable LTS version of .NET 10.0 is released. Therefore, I have no plans to downgrade the language version, which has presented a roadblock for me.

Is there a way to run this without disrupting the language version and syntax? Would providing a compiled DLL version tackle that? I haven't worked with compiled libraries in Unity before; I've always loved to keep things in their source shape.

[xue-fei]

While I was preparing to address that, I remembered that Unity's latest version was using C# 7.3 or 8.0 back then. I've now found they've updated to C# 9.0. My library is using C# 12.0, and I am planning to upgrade to C# 14.0 when a stable LTS version of .NET 10.0 is released. Therefore, I have no plans to downgrade the language version, which has presented a roadblock for me.

Is there a way to run this without disrupting the language version and syntax? Would providing a compiled DLL version tackle that? I haven't worked with compiled libraries in Unity before; I've always loved to keep things in their source shape.

Um, your workload is a bit too much. Moreover, the problem is not entirely due to the downgrade of the C# version. Unity's dll dependency is a difficult problem to solve. I modified some codes from your project and put it into Unity to achieve noise reduction for wav files. It sounds good.
https://github.com/xue-fei/soundflow-unity/blob/master/Assets/soundflow-unity/Test.cs

[xue-fei]

I migrated part of your project code to Unity. After testing, the noise reduction and recording to files work normally, but there is no sound when playing WAV files. I tried to switch the output device, but it still doesn't work. I don't know why...

https://github.com/xue-fei/soundflow-unity/blob/master/Assets/soundflow-unity/Samples/SimplePlayer/UnitySimplePlayer.cs

25.7.7 Unity now is ok……

[LSXPrime]

I migrated part of your project code to Unity. After testing, the noise reduction and recording to files work normally, but there is no sound when playing WAV files. I tried to switch the output device, but it still doesn't work. I don't know why...

https://github.com/xue-fei/soundflow-unity/blob/master/Assets/soundflow-unity/Samples/SimplePlayer/UnitySimplePlayer.cs

25.7.7 Unity now is ok……

Unity working now? what was the issue, a native or managed code?

[xue-fei]

I migrated part of your project code to Unity. After testing, the noise reduction and recording to files work normally, but there is no sound when playing WAV files. I tried to switch the output device, but it still doesn't work. I don't know why...
https://github.com/xue-fei/soundflow-unity/blob/master/Assets/soundflow-unity/Samples/SimplePlayer/UnitySimplePlayer.cs
25.7.7 Unity now is ok……

Unity working now? what was the issue, a native or managed code?

There is no problem running in Unity. How can I obtain the audio data after real-time noise reduction or echo cancellation?

[LSXPrime]

There is no problem running in Unity. How can I obtain the audio data after real-time noise reduction or echo cancellation?

Without modifying the library code, simply create an analyzer and add it to the sound player. If that's not possible, then modify the APM modifier to fire an event with the cleaned audio.

For example:

using SoundFlow.Abstracts;
using SoundFlow.Interfaces;
using System;

namespace SoundFlow.Analyzers;

public class CallbackAnalyzer : AudioAnalyzer
{
    /// <inheritdoc />
    public override string Name { get; set; } = "Callback Analyzer";

    /// <summary>
    /// Event that is raised when audio data has been analyzed.
    /// Subscribers will receive a read-only span of the audio buffer.
    /// </summary>
    public event Action<ReadOnlySpan<float>>? AudioAvailable;

    /// <summary>
    /// Initializes a new instance of the <see cref="CallbackAnalyzer"/> class.
    /// Note: This analyzer does not use the IVisualizer, so it is ignored.
    /// </summary>
    public CallbackAnalyzer() : base(null)
    {
    }

    /// <summary>
    /// Raises the AudioAvailable event, passing the audio buffer to any subscribers.
    /// </summary>
    /// <param name="buffer">The audio buffer to be passed to subscribers.</param>
    protected override void Analyze(Span<float> buffer)
    {
        // Raise the event, notifying any subscribers and passing them the data.
        // We pass it as a ReadOnlySpan to prevent subscribers from modifying the original buffer.
        AudioAvailable?.Invoke(buffer);
    }
}