Control Maps with Hand Gestures: Browser-Native Gesture Control for OpenLayers
Apr 2026
Ever wanted to pan, zoom, rotate, and reset a map without touching anything? I built a library that lets you control OpenLayers maps using just your hands. No mouse, no touchscreen, no backend. Everything runs in the browser.
Why I built this
I've been into webcam-based interactions for a while now (some of you might know Eyebrow Tetris. The idea of controlling things with your body instead of a mouse or keyboard just clicks with me. So when I started thinking about maps in kiosk setups, museum exhibits, and accessibility scenarios, it felt like a natural next step.
What if you could just hold up your fist and drag a map around? Or spread two open hands apart to zoom in? That's exactly what map-gesture-controls does.
How it works
The library uses MediaPipe hand tracking, running as WASM entirely in the browser. Your webcam feed never leaves the device. Privacy is built in, not bolted on.
Here's the basic flow:
1. Webcam capture opens your camera and feeds each frame to the MediaPipe Hand Landmarker, which returns 21 3D landmarks per detected hand.
2. Gesture classification A fist or pinch with your left hand means pan. Same gesture with your right hand means zoom. Both hands tilting means rotate. Hands together in a prayer pose means reset. A state machine with dwell timers and grace periods makes sure brief tracking hiccups don't trigger accidental actions.
3. OpenLayers integration translates the hand movement into actual map interactions, with dead-zone filtering and smooth transitions so it doesn't feel jittery.
There's a configurable dwell timer to prevent accidental triggers and a grace period that smooths out those moments when tracking briefly drops. It feels surprisingly natural once you try it.
The gestures
Pan: Make a fist or pinch with your left hand, then move it around. The map follows.
Zoom: Make a fist or pinch with your right hand, then move it up to zoom in or down to zoom out.
Rotate: Hold up both hands (fist or pinch) and tilt your wrists. The map rotates with you.
Reset: Bring both hands together like a prayer and hold for one second. The map snaps back to where it started.
Fist or pinch both work for every gesture, so just do whatever feels natural to you.
Getting started
If you want to try it yourself:
Then wire it up to your OpenLayers map:
That's it. A few lines and you've got gesture-controlled maps. The start() call needs to happen from a user interaction (like a button click) because of browser webcam permission rules.
Use cases
This isn't just a fun demo (though it is fun). There are real situations where touchless map interaction makes a difference:
- Kiosks and exhibits: museums, visitor centers, trade shows. Nobody wants to touch a shared screen.
- Accessibility: for users with limited mobility who can move their hands but can't grip a mouse or use a touchscreen precisely.
- Touchless environments: think medical settings, clean rooms, or food service areas where you don't want to touch shared devices.
- Creative installations: interactive art, data visualizations, or any project where the UI itself is part of the experience.
What's under the hood
The project is split into two packages:
- @map-gesture-controls/core: the gesture detection engine. It's map-agnostic, so you could hook it up to anything.
- @map-gesture-controls/ol: the OpenLayers integration. Most people just need this one since it re-exports everything from core.
It's written in TypeScript, fully typed, and the bundle is small (check the badge on the repo). The MediaPipe WASM model (~10 MB) loads on first start, and after that it's all local processing.
What's next
The controls have come a long way since the first version. Rotate and reset are in. Up next: a Google Maps adapter (@map-gesture-controls/gmaps) and framework wrappers for React and Vue. If you want to contribute or have ideas, PRs and issues are welcome.
Build in public
This project didn't stay the way I first shipped it. The original zoom gesture used two spread hands moving apart, like a pinch in the air. It worked, but people told me it felt tiring and unnatural. So I reworked the controls to use simpler fist and pinch gestures with single-hand movements. The reset gesture was also a feature asked by the community. People wanted a quick way to snap back to the starting view without reloading. All of this happened in the open on GitHub. If something feels off, open an issue or send a PR. That's how this thing gets better.
