Example for Closed Captioning and muting output audio

[gblikas]

Please read this first

• Have you read the docs? Agents SDK docs
• Have you searched for related issues? Others may have had similar requests

Describe the feature

We need an example on how to properly use the agents sdk to mute output audio and provide CC via the "output_text_delta". It seems like something in the session is blocking the muting of output audio, and the event "listeners" for "output_text_delta" don't fire.

The example should be minimal, but the SDK changes needed should be complete.

Additional Context

{
  "dependencies": {
   "@openai/agents-realtime": "^0.1.2"
  }
}

const transport = new OpenAIRealtimeWebRTC({
          mediaStream,
          audioElement: audioEl,
          model: 'gpt-realtime',
        });

        const agent = new RealtimeAgent({
          name: 'Voice Q&A',
          instructions: buildInstructions(),
        });

        const session = new RealtimeSession(agent, {
            model: 'gpt-realtime',
            config: {
              turnDetection: {
                type: 'semantic_vad',
                eagerness: 'medium',
                createResponse: true,
                interruptResponse: true,
              },
            },
          });

[seratch]

This may not be a direct solution for your use case, but one thing we may want to try is to send response.cancel event from your app to OpenAI server to cancel the on-going response from the server. https://platform.openai.com/docs/api-reference/realtime-client-events/response/cancel

[gblikas]

@seratch , thanks for the quick reply.

This may not be a direct solution for your use case, but one thing we may want to try is to send response.cancel event from your app to OpenAI server to cancel the on-going response from the server. https://platform.openai.com/docs/api-reference/realtime-client-events/response/cancel

I'll give that a try. However, the fact that we cannot do audio.volume = 1 is destructive. I'm assuming the same reason we cannot access audio.volume is the same reason we don't have proper access to the "output_text_delta"?...

[gblikas]

@seratch any information on the live captions?