[FEATURE REQ] Support streaming TTS responses.

[bmdub]

Describe the feature or improvement you are requesting

AudioClient.GenerateSpeechAsync() will only return after the entire generated speech has been received by the client.

To get true streaming, one must set PipelineMessage.BufferResponse = false (for this particular use-case only.)

One solution is to add the following method to AudioClient.cs:

public virtual async IAsyncEnumerable<BinaryData> GenerateSpeechStreamingAsync(string text, GeneratedSpeechVoice voice, SpeechGenerationOptions options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    Argument.AssertNotNull(text, nameof(text));

    options ??= new();
    CreateSpeechGenerationOptions(text, voice, ref options);

    using BinaryContent content = options.ToBinaryContent();
    var requestOptions = cancellationToken.ToRequestOptions();
    using PipelineMessage message = CreateCreateSpeechRequest(content, requestOptions);
    message.BufferResponse = false;
    var result = ClientResult.FromResponse(await Pipeline.ProcessMessageAsync(message, requestOptions).ConfigureAwait(false));
    var stream = result.GetRawResponse().ContentStream;
    var buffer = ArrayPool<byte>.Shared.Rent(1024 * 16);
    try
    {
        int bytesRead;
        while ((bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length, cancellationToken).ConfigureAwait(false)) != 0)
        {
            yield return BinaryData.FromBytes(buffer.AsMemory(0, bytesRead));
        }
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(buffer);
    }
}

A better solution may be to simply return the Stream and have the user handle the reading/buffering.

If you're interested in a PR, lmk. (I had a previous PR #426 which was sub-optimal, altering a generated code file.)

Additional context

No response

[jsquire]

Hi @bmdub. Thank you for reaching out and for your suggestion. This is an interesting one, where I'm not sure if a new API member would be the right design here, whether this would potentially be better represented as another option member in SpeechGenerationOptions, or this is a form that we would advocate using the protocol overload for. @KrzysztofCwalina would be the authoritative voice on the API shape.

In the meantime, you can apply this change either by:

• Writing a custom policy which explicitly disables buffering.
• Dropping down to the protocol level and setting BufferResponse to false on your RequestOptions.

The latter would look similar to your proposed method, something like:

var unbuffered = await client.GenerateSpeechAsync(
    BinaryContent.Create(BinaryData.FromObjectAsJson(new
    {
        input = "This is a test",
        model = "gpt-4.1",
        voice = "fable",
        instructions = "Talk like a pirate"
    })),
    new RequestOptions { BufferResponse = false });

var stream = unbuffered.GetRawResponse().ContentStream;
var buffer = ArrayPool<byte>.Shared.Rent(1024 * 16);

try
{
    int bytesRead;
    while ((bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false)) != 0)
    {
       var data = BinaryData.FromBytes(buffer.AsMemory(0, bytesRead));

       // Do something with the data.  This is just an placeholder example.
       await ProcessAudioChunkAsync(data).ConfigureAwait(false);
    }
}
finally
{
    ArrayPool<byte>.Shared.Return(buffer);
}