A free, browser-based synthesizer controlled entirely by hand gestures. No downloads, no plugins — just your webcam and your hands. FAQ ↓
Gesture Synth Weld is a hand gesture music synthesizer that runs entirely in your web browser. Using your webcam and MediaPipe hand tracking, it creates a two-hand musical instrument — your left hand controls harmony (key, scale degree, major/minor toggle) and your right hand controls expression (chord type, volume, tone, and octave). No downloads, no MIDI controllers, no plugins required.
Choose from 5 sound presets: Piano (sampled Salamander Grand Piano), Strings, Organ, Synth, and Vibraphone. Switch seamlessly between them without cutting off playing notes. Enable the Arpeggiator to sweep through chord notes like a harp, or toggle Auto Bass for an automatic low-end foundation that follows your harmony. Record up to 15 seconds of your performance as a WebM video with audio — ready to share.
Powered by the Web Audio API via Tone.js for low-latency synthesis and MediaPipe Hands for real-time finger tracking (30fps). All processing happens locally on your device — your camera feed is never uploaded to any server.
What you need: a computer with a webcam, a modern browser (Chrome, Edge, or Firefox recommended), and speakers or headphones. For the best experience, use Chrome or Edge with WebGL2 hardware acceleration enabled.
Click "Start" and grant camera permission. Your video stays on your device — nothing is uploaded.
Choose a key (C–B) and mode. Your fingers select the scale degree, and wrist tilt flips between major and minor. Or lock to a fixed mode.
Finger count sets chord complexity (1=note, 2=triad, 3=7th, 4=9th). Hand height = volume, tilt = tone. Or lock to one of 8 chord types. No camera? Use keyboard shortcuts →
If you're new to Gesture Synth Weld, start with these settings for the most intuitive experience: Left Hand Mode = "Scale + Tilt Major/Minor", Right Hand Mode = "Finger Layout = Chord Style", Sound = Piano. This gives you the classic two-hand instrument feel — left hand picks the harmony, right hand plays the chord.
Enable the Arpeggiator toggle to transform block chords into flowing arpeggios. Each note in the chord plays sequentially from low to high (like a harp sweep). Use Slow speed for ambient pads, Normal for melodic accompaniment, and Fast for energetic runs. The arpeggiator works with all chord types and sound presets.
Toggle Auto Bass to add a low-end foundation that automatically follows your left-hand harmony. The bass plays the root note of your current chord in a lower octave, giving your performance a full band feel. Adjust the bass volume independently — keep it subtle for background warmth or turn it up for a driving low-end. Auto Bass works especially well with Piano and Organ presets. See FAQ for more →
Switch your right hand to Fixed Chord Style to lock into a specific chord type. This is perfect for practicing a particular chord quality (try Diminished or Sus4 for unique textures) or when you want to focus purely on expression without worrying about finger count. You can still control volume, tone, and octave with your right hand — only the chord type is locked.
All 5 sound presets (Piano, Strings, Organ, Synth, Vibraphone) can be switched seamlessly — currently playing notes continue without interruption. Try starting a phrase on Piano, switching to Strings for a swelling pad, and finishing on Vibraphone for an ethereal ending. The tone control (right-hand tilt) also affects each preset differently — experiment to find your favorite combinations.
Click the red Record button to capture up to 15 seconds of your performance as a WebM video with audio. The recording includes your canvas visuals and synthesized sound — perfect for sharing on social media or messaging apps. Your camera feed is never recorded or uploaded. More questions? Check the FAQ →
If you see "No camera found", make sure your webcam is connected and not being used by another application (Zoom, Teams, etc.). On Mac, check System Preferences → Security & Privacy → Camera to ensure your browser has permission. On Windows, check Settings → Privacy → Camera.
You may have previously blocked camera access for this site. Click the lock icon (🔒) in your browser's address bar, find Camera permissions, and change it to "Allow". Then reload the page and click Start again.
Hand tracking uses your GPU via WebGL. Close other GPU-intensive apps (games, video editors). If you're on a laptop, make sure it's plugged in — some browsers throttle GPU performance on battery. Chrome and Edge generally offer the best MediaPipe performance.
The audio engine requires a user gesture to start (browser autoplay policy). Make sure you clicked "Start" before expecting sound. Also check that your browser isn't muted and your system volume is up. If piano samples are still loading, you'll see "Loading samples…" in the control panel — the synth will use a fallback sound until they finish.
Piano sounds use the Salamander Grand Piano sample set loaded from a CDN. If your network blocks the CDN or the connection is slow, the synth will automatically fall back to a synthesized piano sound. You can switch to Synth or Organ timbres which don't require external downloads. Back to How It Works ↑
These shortcuts work when camera mode is unavailable. Prefer hand gestures? See How It Works →
| Key | Action |
|---|---|
| 1 – 7 | Select chord (I through vii°) |
| ↑ / ↓ | Increase / decrease volume |
| T | Force major mode |
| Y | Force minor mode |
| Q | Piano timbre |
| W | Strings timbre |
| E | Organ timbre |
| R | Synth timbre |
| V | Vibraphone timbre |
| A | Toggle Arpeggiator |
| B | Toggle Auto Bass |
| Space | Stop all notes |
| Esc | Reset state |
No. All you need is a computer or laptop with a built-in or USB webcam, and a modern browser like Chrome, Edge, or Firefox. No MIDI controllers, no external sensors, no downloads. Your webcam handles all the input — MediaPipe's hand tracking model runs entirely on your device.
Gesture Synth Weld works in any modern browser that supports the Web Audio API and getUserMedia — including Chrome 90+, Firefox 90+, Edge 90+, and Safari 15+. For the best hand tracking experience, Chrome or Edge is recommended since they support MediaPipe's WebGL2 backend most efficiently.
Yes! Click the Record button to start capturing your performance (up to 15 seconds). When you stop recording, a WebM video file with audio is automatically downloaded. The recording captures both the canvas visuals and synthesized audio — your camera feed is never uploaded. You can share the video directly to social media or edit it further.
Gesture Synth Weld supports all 12 keys (C through B) with two right-hand modes. In Finger Layout mode: 1 finger = single note, 2 fingers = triad, 3 fingers = 7th chord, 4 fingers = 9th chord. In Fixed Chord mode, choose from 8 chord types: Major Triad, Major 1st Inversion, Minor Triad, Diminished, Sus2, Sus4, Major 7th, and Dominant 7th. Left-hand wrist tilt dynamically switches between major and minor tonality, or lock to a fixed mode.
Gesture Synth Weld features a unique two-hand division system: your left hand controls harmony (key selection, scale degree, major/minor toggle) while your right hand controls expression (chord complexity, volume, tone, octave). Unlike basic gesture synths, it includes an arpeggiator for harp-like sweeps, an auto bass generator for low-end foundation, 5 distinct sound presets including sampled piano, support for 9th chords (4-finger), 8 fixed chord styles, and built-in WebM recording. It's also open source under the MIT license.
Yes! Gesture Synth Weld is free to use. We support the development and hosting of this tool through non-intrusive advertising. No registration, no sign-up, no paywall — just open and play. The source code is also available on GitHub under the MIT license for anyone who wants to study, modify, or contribute.