A free, browser-based synthesizer controlled entirely by hand gestures. No downloads, no plugins — just your webcam and your hands. FAQ ↓
Gesture Synth Weld is a hand gesture music synthesizer that runs entirely in your web browser. No downloads, no plugins, no MIDI controllers — just your webcam and your hands. Raise your left hand to play chords, move your right hand up and down to control volume, and choose between four instrument timbres: Piano, Strings, Organ, and classic Synth.
Powered by MediaPipe Hands for real-time gesture recognition and the Web Audio API for low-latency audio synthesis with real piano samples (Salamander Grand Piano), Gesture Synth Weld turns any computer into an expressive musical instrument. Tilt your left wrist to switch between major and minor modes. Record your performance and export it as a WAV file. Use keyboard shortcuts if you don't have a camera available.
What you need: a laptop or desktop computer with a working webcam, a modern browser (Chrome, Edge, or Firefox recommended), and speakers or headphones. That's it. Open the page, click "Start", allow camera access, and start playing.
Click "Start" and grant camera permission when prompted. Your video stays on your device — nothing is uploaded to any server.
Hold up 1–5 fingers to select a chord from the diatonic scale (I, ii, iii, IV, V, vi, vii°). Tilt your wrist to toggle major/minor.
Raise your right hand higher for more volume. Lower it to fade out. Use the floating control panel to switch timbres. No camera? Use keyboard shortcuts →
Make sure your hands are well-lit and clearly visible to the camera. Avoid backlighting (a bright window behind you) as it makes hand detection harder. A plain, non-distracting background helps the MediaPipe model track your fingers more accurately. If tracking feels sluggish, move your hands closer to the camera — larger hand landmarks are easier to detect.
When switching between chords, try to move your fingers deliberately rather than quickly flicking them. The gesture recognition runs at about 30fps, and smooth finger transitions produce cleaner audio without clicking artifacts. If you hear clicks between chord changes, try raising the sustain time or switching to the Strings timbre, which has a slower attack that masks transitions.
The major/minor toggle is mapped to your left wrist angle. A subtle tilt goes a long way — you don't need to rotate your hand dramatically. Practice holding a chord with your left hand while gently rocking your wrist to hear the quality shift between major (brighter) and minor (darker). This is especially expressive with the Piano and Strings timbres.
Volume is mapped to the vertical position of your right hand's wrist in the camera frame. Top of the frame = maximum volume, bottom = near silence. Think of it like a theremin — the higher your hand, the louder the sound. For fine volume control, make small vertical movements near the middle of the frame where the mapping is most sensitive.
Hit the Record button before you start playing. When you're done, click Stop and a WAV file will download automatically. The recording captures only the synthesized audio — your camera feed is never recorded. You can open the WAV file in any audio editor (Audacity, GarageBand, etc.) to add effects, mix with other tracks, or share online.
No camera handy? Click "No camera? Try keyboard mode" on the start screen. You can play all seven chords with number keys 1–7, adjust volume with arrow keys, and switch between all four timbres with Q, W, E, and R. This is also great for learning the chord progressions before using hand gestures. Check the FAQ below for more answers.
If you see "No camera found", make sure your webcam is connected and not being used by another application (Zoom, Teams, etc.). On Mac, check System Preferences → Security & Privacy → Camera to ensure your browser has permission. On Windows, check Settings → Privacy → Camera.
You may have previously blocked camera access for this site. Click the lock icon (🔒) in your browser's address bar, find Camera permissions, and change it to "Allow". Then reload the page and click Start again.
Hand tracking uses your GPU via WebGL. Close other GPU-intensive apps (games, video editors). If you're on a laptop, make sure it's plugged in — some browsers throttle GPU performance on battery. Chrome and Edge generally offer the best MediaPipe performance.
The audio engine requires a user gesture to start (browser autoplay policy). Make sure you clicked "Start" before expecting sound. Also check that your browser isn't muted and your system volume is up. If piano samples are still loading, you'll see "Loading samples…" in the control panel — the synth will use a fallback sound until they finish.
Piano sounds use the Salamander Grand Piano sample set loaded from a CDN. If your network blocks the CDN or the connection is slow, the synth will automatically fall back to a synthesized piano sound. You can switch to Synth or Organ timbres which don't require external downloads.
| Key | Action |
|---|---|
| 1 – 7 | Select chord (I through vii°) |
| ↑ / ↓ | Increase / decrease volume |
| T | Force major mode |
| Y | Force minor mode |
| Q | Piano timbre |
| W | Strings timbre |
| E | Organ timbre |
| R | Synth timbre |
| Space | Stop all notes |
| Esc | Reset state |
No. All you need is a computer or laptop with a built-in or USB webcam, and a modern browser like Chrome, Edge, or Firefox. No MIDI controllers, no external sensors, no downloads. Your webcam handles all the input — MediaPipe's hand tracking model runs entirely on your device.
Gesture Synth Weld works in any modern browser that supports the Web Audio API and getUserMedia — including Chrome 90+, Firefox 90+, Edge 90+, and Safari 15+. For the best hand tracking experience, Chrome or Edge is recommended since they support MediaPipe's WebGL2 backend most efficiently.
Yes! Click the Record button to start capturing your performance. When you stop the recording, a WAV audio file is automatically downloaded to your computer. You can then open it in any audio editor, share it with friends, or upload it to social media. The recording captures only the synthesized audio — your camera feed is never recorded or uploaded.
The synth plays the seven diatonic chords of the C major scale: I (C), ii (Dm), iii (Em), IV (F), V (G), vi (Am), and vii° (Bdim). The chord is selected by the number of fingers on your left hand (1–5 maps to seven chords via a repeating cycle). Tilt your left wrist to the left for major tonality, or to the right for minor — this modifies the chord quality in real time. Choose from Piano, Strings, Organ, or Synth timbres using the floating control panel.
Yes! Gesture Synth Weld is free to use. We support the development and hosting of this tool through non-intrusive advertising. No registration, no sign-up, no paywall — just open and play. The source code is also available on GitHub under the MIT license for anyone who wants to study, modify, or contribute.