Speech synthesis and transcription

Turn text into a spoken cue, check a transcript, then try a local browser conversation. The bot combines faster-whisper base.en, a small chat model in LM Studio and Kokoro ONNX. Recognized words fill an editable chat box; you decide when to send them.

No microphone starts from this static page. The local Streamlit app asks for browser permission only after you click Start mic. First use downloads libraries and model weights. Local inference needs no classroom API key. Model inference and actual microphone hardware still need a classroom-device rehearsal.

Prepare the portable environment

Follow the PFAD local conversation guide. Run commands from week06/speech in the PFAD checkout. Its separate Python 3.12 environment avoids the root Week 4 Python project. CPU speech is the default on Windows, Macs and AMD machines. NVIDIA CUDA is optional; LM Studio can accelerate its language model independently, including Metal on supported Macs.

Create the environment once; skip uv venv if it is already prepared. Installation and model preparation need internet; prepared models run locally. Existing model weights are checked and reused where possible.

Windows PowerShell

uv venv --no-project --python 3.12 .venv
uv pip install --python .venv/Scripts/python.exe -r conversation_requirements.txt
uv run --no-project --python .venv/Scripts/python.exe prepare_models.py
uv run --no-project --python .venv/Scripts/python.exe synthesise_onnx.py --check
uv run --no-project --python .venv/Scripts/python.exe transcribe.py --check

macOS / Linux

uv venv --no-project --python 3.12 .venv
uv pip install --python .venv/bin/python -r conversation_requirements.txt
uv run --no-project --python .venv/bin/python prepare_models.py
uv run --no-project --python .venv/bin/python synthesise_onnx.py --check
uv run --no-project --python .venv/bin/python transcribe.py --check

Kokoro ONNX needs no PyTorch or separate system espeak-ng install: espeakng-loader supplies the phonemizer. Browser WebRTC needs no PortAudio. The older console microphone and PyTorch synthesis exercises have separate setup requirements, including sounddevice/PortAudio for console capture. --check checks prerequisites without loading model weights or recording; it does not prove real inference will work.

Prepare models ahead of class. Use a short consented PCM WAV and a typed-message route if the microphone is unavailable. Windows CPU/CUDA was exercised on the instructor laptop; macOS and Linux installation and physical classroom microphone/speakers still need rehearsal.

1. Text to a spoken cue

From the prepared speech folder, use the Windows interpreter below. On macOS/Linux, substitute .venv/bin/python.

uv run --no-project --python .venv/Scripts/python.exe synthesise_onnx.py --text "Your file is ready." --speed 1.0 --out ../out/cue.wav

Kokoro ONNX uses a preset synthetic voice and retains all generated chunks, saving a mono PCM WAV at 24,000 samples/s and a settings record. Open the file yourself at low volume. Try --speed 0.85 with --out ../out/cue-slower.wav. Does the cue feel clearer, calmer or more interruptive? Check pronunciation as well as tone. New filenames preserve the comparison.

2. A file to a proposed transcript

uv run --no-project --python .venv/Scripts/python.exe transcribe.py --audio ../out/cue.wav --offline --out ../out/cue.txt

Again, substitute .venv/bin/python on macOS/Linux. faster-whisper runs English base.en with CPU int8 by default. Its decoder resamples audio to mono 16,000 samples/s. Resampling changes the sample sequence; changing only the rate label would change speed and pitch.

Compare the printed and saved transcript with your original sentence and the audio. A correct transcript does not prove natural pronunciation.

Recognition can hallucinate words around silence or noise. Treat the text as a prediction until you check it; get consent before capturing another person's voice.

Optional: a bounded console transcript

After the separate console setup, this command records three 5-second chunks and then closes the microphone. Use .venv/bin/python on macOS/Linux.

uv run --no-project --python .venv/Scripts/python.exe transcribe.py --microphone --chunks 3 --seconds 5 --out ../out/mic.txt

Ctrl+C stops early. This is chunked near-live transcription; boundaries can cut words and inference can lag. The browser app below listens until you press Stop mic or Send. The file exercise needs no microphone. Do not publish classmates' recordings or transcripts.

3. Open the local browser bot

Load a small instruction/chat model in LM Studio and start its local server on port 1234. The instructor uses the already-cached MiniCPM5-2B Q4_K_M; choose a model that fits your computer. See the LM Studio setup and current app source.

Windows PowerShell

$env:HF_HUB_OFFLINE = "1"
$env:TRANSFORMERS_OFFLINE = "1"
uv run --no-project --python .venv/Scripts/python.exe python -m streamlit run speech_webrtc.py --server.address 127.0.0.1 --browser.gatherUsageStats false

macOS / Linux

HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 uv run --no-project --python .venv/bin/python python -m streamlit run speech_webrtc.py --server.address 127.0.0.1 --browser.gatherUsageStats false

Open http://localhost:8501. In the sidebar choose Find loaded models, then select your LM Studio model. Type and press Enter, or click Start mic beside the chat box and allow the microphone. Recognition fills the editable input in 3-second chunks. Capture continues until Stop mic or Send; it does not end after ten seconds. Stop mic for careful correction, then press Enter. Sending also stops capture before the reply.

No microphone starts from this static course page. The Python app runs on your own laptop; the static Pages preview does not host it. This localhost exercise is not a public multi-user deployment. A remote browser needs HTTPS and suitable WebRTC networking.

4. Check, send and hear the response

Correct the draft before sending. Text goes to the selected loopback LM Studio server only when you press Enter. Kokoro ONNX speaks the reply and each assistant message retains a player; browser autoplay settings may require pressing Play. Starting the mic or sending pauses previous reply audio. The editable personality prompt introduces Pip as a playful studio companion, usually replying in three to five sentences.

Microphone audio is processed in memory; no microphone WAV is saved. Recent transcripts, conversation and reply audio live in the session. Setup and transcription logs are collapsed in the sidebar. Keep CPU speech unless you have the compatible NVIDIA libraries; optional CUDA instructions describe the extra runtime and memory requirements. TensorRT and native streaming alternatives are optional research, not dependencies.

If a model or device fails, use a file transcript and a written response. This exercise adds no assignment requirement to use speech or AI.

A small design experiment

Keep one sentence, its WAV, settings and transcript as a reference.
Change one synthesis parameter. Predict what a listener will notice.
Compare the second audio and transcript. Which result is useful evidence for your prediction?
Sketch an interface that lets a person correct a transcript or stop a spoken cue.

The visual route is to inspect the sentence, sample rate, duration and transcript. Listening or microphone use is optional.

Other speech synthesis options

Kokoro is the main exercise so everyone can discuss the same inputs and settings. Optional alternatives have different installation, hardware and licensing requirements.

Piper: a local CPU voice for interface prompts. The current engine is GPL-3.0-or-later; inspect the chosen voice’s own model card and license.
Chatterbox: expressive dialogue and delivery cues. Current upstream Nano/V3 options may not exist in the published 0.1.7 wheel. Hardware and checkpoint choice need their own tested setup.
Qwen3-TTS: multilingual voice design and delivery instructions, with different controls for different checkpoints. Its larger models suit a prepared demonstration rather than an assumed fast CPU exercise.

Compare the same text for pronunciation, controllability, delay and resource use. Check the actual model license as well as the Python package. These are options to investigate, not a quality ranking or a claim that they were run here. Use preset or designed voices; reference-speaker experiments require that person’s permission for the intended use and sharing.

KittenTTS needs a version label

The older KittenTTS 0.8.1 API can use mini-0.8, an approximately 80 MB model. That is a legacy English example. It is distinct from KittenTTS 2, a much larger multilingual model with a different API and weights license. Do not apply an old “25 MB” claim to the current model. Neither is installed or benchmarked for this lesson.

Sources and checks

faster-whisper 1.2.1 · Whisper · Kokoro 0.9.4 · Kokoro model card · Microphone setup

The PFAD environment report records instructor Windows inference and browser checks, model reuse and optional CUDA measurements. The teaching review does not rerun those model checks. Classroom audibility and physical microphone capture still need rehearsal. Other synthesis models below were not run in this review; package pins alone do not guarantee identical results across machines.