Tutorial: Speech-to-Text Integration — Blazor

Set up IsabyteAISpeechToText and IsabyteAIRealtimeTranscriber, pick a backend (local Whisper, OpenAI, Azure), and wire transcribed text into form fields or chat workflows.

Level: Intermediate · Time: ~20 minutes

Two components, two use cases

Pick the right one

If your user uploads a meeting recording and waits for a transcript, use IsabyteAISpeechToText. If they speak and the UI reacts as they talk, use IsabyteAIRealtimeTranscriber.

Step 1: Choose a backend

The "backend" is the actual recognition engine. There's no registration step: each component picks it at runtime from its parameters. The two components do not offer the same backends — IsabyteAISpeechToText runs local Whisper models only, while IsabyteAIRealtimeTranscriber also reaches the browser's own speech recognition and the cloud APIs. A TranscriptorType applies with Mode="IsabyteAIRealtimeTranscriberMode.Server", the default.

Step 2: Local Whisper on the server

For most apps this is the right starting point — no API keys, no per-call cost, and you can iterate offline. Download a GGML model from the Whisper.cpp project and point ModelPath at it.

@page "/transcribe"
@using Isabyte.PowerTools.Components.AI

<IsabyteAISpeechToText Mode="IsabyteAISpeechToTextMode.Server"
                   ModelPath="C:\models\ggml-medium.bin"
                   Language="auto"
                   OnResponseCompleted="OnTranscribed" />

@code {
    private string _transcript = "";

    private async Task OnTranscribed(string text)
    {
        _transcript = text;
        await InvokeAsync(StateHasChanged);
    }
}

With ShowBuiltInUI="true" (the default), the component renders a Load-model button, a file picker, and a transcript card. Users get a working transcription page with no further code on your side.

Step 3: WebAssembly mode (in-browser)

When Mode is WebAssembly, the component resolves a model from Hugging Face and runs ONNX inference directly in the browser. No model file ships with your app; the first run downloads weights and caches them in the browser's Cache Storage.

<IsabyteAISpeechToText Mode="IsabyteAISpeechToTextMode.WebAssembly"
                   HuggingFaceModelName="Xenova/whisper-tiny.en"
                   ShowModelSelector="true"
                   ShowLanguageSelector="true"
                   ShowApiKeyInput="false"
                   OnResponseCompleted="OnTranscribed" />

WASM trade-offs

Browser-side inference is slower than the server (especially on phones), and the model download is a 40–250 MB blocking event on first run. Show a clear progress UI — the OnStatusUpdate callback fires repeatedly during download and load.

Step 4: OpenAI Whisper with the real-time transcriber

IsabyteAISpeechToText runs local models only — it has no ApiKey parameter. To send the audio to the OpenAI API instead, use IsabyteAIRealtimeTranscriber with TranscriptorType="openai whisper" and an ApiKey. The transcript still arrives as a string through OnResponseCompleted, as in the steps above.

Register the key once in Program.cs, read from configuration (an environment variable or appsettings). OpenAiSettings is your own settings class:

builder.Services.AddSingleton(new OpenAiSettings
{
    ApiKey = builder.Configuration["OpenAi:ApiKey"]
        ?? throw new InvalidOperationException("OpenAi:ApiKey missing")
});

Then inject it into the component that hosts the transcriber:

@inject OpenAiSettings Settings

<IsabyteAIRealtimeTranscriber Mode="IsabyteAIRealtimeTranscriberMode.Server"
                          TranscriptorType="openai whisper"
                          ApiKey="@Settings.ApiKey"
                          Language="en-US"
                          OnResponseCompleted="OnSpoken" />

Never hard-code an API key in markup — keep it in appsettings / environment variables and inject it as shown. The key stays on the server only when the component renders under Interactive Server with Mode set to Server, as above. A component rendered in WebAssembly runs in the browser together with its parameters, so any key you pass there is visible to the user (in WebAssembly mode the component hands ApiKey to the in-browser loader as a Hugging Face token).

Step 5: Azure Speech Services

Azure needs both an ApiKey and an AzureEndpoint (the region URL from your Azure portal). Each recognized segment is sent to Azure's fast transcription API; the component does not request speaker diarization, so OnResponseCompleted receives plain text.

<IsabyteAIRealtimeTranscriber TranscriptorType="azure fast api"
                          ApiKey="@_azureKey"
                          AzureEndpoint="https://westeurope.api.cognitive.microsoft.com/"
                          Language="en-US"
                          OnResponseCompleted="OnSpoken" />

Step 6: Voice activity detection (VAD)

Real-time transcription needs to know when a user has stopped talking, otherwise the transcript only arrives when the mic shuts off. In Server mode the browser streams the microphone audio to the server, and the server splits it into utterances with one of two VAD backends, set with VadType. (WebAssembly and Browser modes segment the audio in the browser and ignore it.)

Both whisper onnx (the default transcriptor) and silero load model files from the server's disk, so pass both paths:

<IsabyteAIRealtimeTranscriber TranscriptorType="whisper onnx"
                          TranscriberModelPath="C:\models\whisper_medium_int8_cpu_ort_1.18.0.onnx"
                          VadType="silero"
                          VadModelPath="C:\models\silero_vad.onnx"
                          Language="en-US"
                          OnResponseCompleted="OnSpoken" />

Step 7: Voice-activated form filling

The classic "fill this textbox by speaking" pattern: hide the built-in UI, render your own mic button, and let OnResponseTokenPartComplete show the current utterance in a bound field as the user talks. The component is fully controllable from outside via its public methods (LoadModel(), StartRecording(), StopRecording()). LoadModel() must complete before StartRecording() does anything — without the built-in UI, nothing else calls it. Both callbacks carry the text of the current utterance, not a single token: OnResponseTokenPartComplete repeats the growing text while the user speaks, and OnResponseCompleted delivers the finished utterance, so keep the finished ones and show the partial after them.

<IsabyteAIRealtimeTranscriber @ref="_transcriber"
                          ShowBuiltInUI="false"
                          TranscriberModelPath="C:\models\whisper_medium_int8_cpu_ort_1.18.0.onnx"
                          VadModelPath="C:\models\silero_vad.onnx"
                          OnResponseTokenPartComplete="OnPartial"
                          OnResponseCompleted="OnFinal" />

<IsabyteTextInput @bind-Value="_note" Placeholder="Speak or type…" />

<button class="btn btn-primary"
        @onclick="ToggleAsync">
    @(_recording ? "Stop" : "🎤 Dictate")
</button>

@code {
    private IsabyteAIRealtimeTranscriber? _transcriber;
    private string _note = "";
    private string _finished = "";
    private bool _recording;

    private async Task ToggleAsync()
    {
        if (_recording)
        {
            await _transcriber!.StopRecording();
        }
        else
        {
            if (!_transcriber!.IsModelLoaded) await _transcriber.LoadModel();
            await _transcriber.StartRecording();
        }
        _recording = !_recording;
    }

    // The utterance so far, repeated as it grows.
    private void OnPartial(string text) => _note = _finished + text;

    // One finished utterance.
    private void OnFinal(string text)
    {
        _finished += text + " ";
        _note = _finished;
    }
}

Common gotchas

• SignalR message size. In Server mode the recorded audio reaches the server as base64 segments over the Blazor circuit, and SignalR's MaximumReceiveMessageSize defaults to 32 KB. The package does not change it — raise it yourself (builder.Services.AddSignalR(o => o.MaximumReceiveMessageSize = …)) to fit your longest utterance, and test with long recordings before going to production.
• Microphone permission. Browsers require user-initiated permission. The first call to StartRecording() triggers the prompt; if the user denies it, OnStatusUpdate fires with an error payload. Treat this as a first-class UI state.
• Language autodetect. Whisper's autodetect needs ~5 seconds of audio. For short utterances, pin Language="en-US" explicitly — the transcription quality is markedly better.
• HuggingFace model caching. In WASM mode the first run downloads model weights. If your CSP blocks huggingface.co, the load fails silently — whitelist that origin in your Content-Security-Policy headers.

Next

• Next: Image Processing in the Browser — Use IsabyteImageAnalysis for edge detection and IsabyteAnimationEffect for GPU-accelerated visuals — all client-side.
• Live Demo — Try every backend interactively, including the headless mode used in Step 7.

See also

• IsabyteAISpeechToText demo
• IsabyteAIRealtimeTranscriber demo