Voice Chatbot
In this coding challenge, I build a conversational voice chatbot entirely in the browser with p5.js. I combine three pieces: speech-to-text with OpenAI's Whisper model, text-to-speech with Kokoro TTS, and a "brain" for the bot. I also explore the transformers.js pipeline API and ...
Reference page for the beta version of p5.js 2.0
Paper introducing the Whisper model.
In this paper, the authors propose a framework for model cards to encourage transparency on model reporting.
ONNX facilitates greater interoperability in the AI tools community.
Model card for Whisper-tiny.en.
Github repo for Joshua Lochner, the creator of transformers.js.
Installation page for Transformers.js.
Article discussing the latest version of p5.sound.js.
Documentation about getting the sound from the mic.
Documentation about the MediaRecorder object.
Github repo for the Kokoro speech-to-text model.
Model card for the Kokoro-82M on Hugging Face.
Wikipedia article about the early natural language processing program ELIZA.
RiveScript is a simple scripting language for chatbots.
Model card for SmolLM3
Documentation about running models on the webGPU.
Documentation about available quantization options.
This video discusses loading data with async and await.
In this video, I cover what Transformers.js is, how to load it into a p5.js sketch, explain the pipeline API, and demonstrate sentiment analysis and language detection examples.
Complete unedited live stream showing the development process with all mistakes and problem-solving.
- Editing
by Mathieu Blanchette
- Animations
by Jason Heglund