Mobile input and virtual controls

iOS and Android provide multi-touch and accelerometer input, and web pages on phones and tablets provide multi-touch. These commands are also available on desktops, including web pages there, where they return neutral data; a web page's touches on a desktop's touch screen stand in for the mouse. Virtual gamepad and keyboard overlays are available on iOS, Android, macOS, Windows and the web.

Start with a sample

Touch Playground tracks several fingers and short taps. Tilt Collector is a playable tilt game with calibration, optional smoothing, sensor on/off, and touch/keyboard fallback arrows. Both have on-screen Exit buttons. Compile in Extended mode and select an iOS target or the Android device/emulator. See Android APK export for Android setup.

One snapshot per update

Call UpdateMobileInput before reading mobile input. It pumps native events and publishes a coherent snapshot. Other input reads can process native events without changing that snapshot. Getters do not pump events or consume held state/deltas.

UpdateMobileInput
For index=1 To TouchCount()
    id=TouchID(index)
    Oval TouchX(id)-10,TouchY(id)-10,20,20,True
Next

Indexes run from 1 through TouchCount and may change next update. IDs remain stable for a contact's lifetime and are not reused in that program run. TouchDown tests an ID; inactive IDs return neutral values. Pressure is 0..1 when supported, or -1 otherwise. TouchCaptured and TouchEventCaptured return 1 for a contact owned by virtual controls. Skip captured contacts in ordinary game touch handlers. Their complete raw contact stream remains available for diagnostics.

Short taps and cancellation

Drain GetTouchEvent after each update: 1 begin, 2 move, 3 end, 4 cancel, 0 empty. Read the TouchEvent getters before requesting the next event. End/cancel records contain final positions even after the contact leaves TouchCount. Another update discards unread events. Event time uses the MilliSecs clock domain.

phase=GetTouchEvent()
While phase
    If phase=1 Then taps=taps+1
    phase=GetTouchEvent()
Wend

There are at most 32 live contacts and 256 pending events. Adjacent moves may be coalesced. Overflow cancels exposed contacts and releases virtual controls and synthetic mouse input; TouchEventOverflow is set for the next snapshot. Focus loss, suspension, debugger pause, resize, rotation and surface closure also cancel contacts and immediately invalidate held snapshots. Existing fingers must lift before starting again. FlushTouches clears events, overflow and deltas but preserves held contacts.

Coordinates and mouse compatibility

Coordinates are floating-point graphics pixels, X right and Y down, relative to the top-left of the displayed game frame. They account for display scale and letterboxing, ignore drawing Origin, and are not clamped: touches in letterbox bars may lie outside GraphicsWidth/GraphicsHeight.

TouchMouse 1 is the default. The first eligible touch drives the mouse pointer and left button; other fingers cannot move or release that owner. A contact beginning on an overlay is captured and cannot click the game beneath it; contacts beginning outside controls stay with the game even when dragged over a control. There is no promotion of an already-held finger. The mouse coordinates are clamped to the game frame. Physical and synthetic mouse holds are independent. Use TouchMouse 0 in a touch-native game to avoid processing the same contact through both APIs.

Run an existing keyboard or joystick game with controls

Create a group after Graphics or Graphics3D. Controls draw and process input automatically, including during WaitKey, WaitJoy and Input$. No UpdateMobileInput or overlay draw call is needed for ordinary keyboard and joystick commands. The primary graphics window is supported; arbitrary native GUI windows and additional canvases are not.

Graphics 800,480,0,2
keys=CreateVirtualKeyboard("wasd")
If keys=0 Then RuntimeError MobileInputError$()
; The existing loop can keep reading KeyDown(17), KeyHit(57), etc.

Keyboard presets are wasd, arrows, full-keys and empty. The first two include movement, Space, Enter and Escape. Full-keys is a docked US gameplay keyboard supporting Input$ editing and WaitKey; it does not provide native Unicode composition or autocorrection. Physical and virtual keys combine: releasing one source cannot release the other. KeyHit and GetKey receive one press per aggregate down transition.

pad=CreateVirtualGamepad("standard")
If pad=0 Then RuntimeError MobileInputError$()
port=VirtualGamepadPort(pad)
; Use this stable port in the existing commands:
steering#=JoyX(port)
fire=GamepadHit(1,port)

The default port is the lowest free slot. Request port 0 explicitly only when adapting a game that always reads that port, and handle creation failure if it is occupied. Physical controllers keep their ports. Standard offers two sticks, d-pad, four face buttons, shoulders, separate triggers, Back/Start and stick clicks. Classic offers the d-pad, face buttons and Back/Start; its d-pad also drives the left axes. JoyVirtual(port) identifies the virtual controller. It reports eight raw axes, 21 buttons, one hat, no rumble and unknown battery.

Adjust the overlay

The draggable Controls tab opens Hide/Show, Smaller, Larger and Mirror. Opening the menu releases virtual holds. While any group is installed, the desktop pointer is visible and MoveMouse warping is suspended; freeing the last group restores the program's pointer behavior. Relative mouse aiming is not emulated. Desktop clicks operate one control at a time; a physical keyboard can be used alongside them. iOS and Android support multiple fingers.

VirtualControlsVisible, VirtualControlsOpacity, VirtualControlsScale and VirtualControlsLeftHanded adjust a group. Use VirtualControl to find a preset control by name, such as space, escape, left-stick or south. AddVirtualKey and AddVirtualKeyDPad extend keyboard groups; AddVirtualPadButton extends a pad. VirtualKeyBinding changes a key or modifier chord. Invalid bindings preserve the old control and report MobileInputError$.

jump=AddVirtualKey(keys,57,"Jump")
VirtualControlRect jump,4,20,20,112,56
VirtualControlsOpacity keys,0.8

Rectangles use logical display units within the safe game area, independent of Graphics resolution and drawing Origin. Anchors are 1 top-left, 2 top-right, 3 bottom-left, 4 bottom-right, 5 center. Corner offsets are positive inward; center offsets move right/down. Presets fit the available area; explicit rectangles retain their placement when mirrored. Group scale applies to both. Keep custom controls apart and large enough to touch.

Buttons release when dragged outside and press again on re-entry. Sticks and triggers keep their first finger; a second held finger is never promoted. Hiding, zero opacity, rebinding, resizing, focus loss and pause release virtual input immediately. Fingers must lift before pressing again. FreeVirtualControl removes one control; FreeVirtualControls removes its group and disconnects a virtual pad. EndGraphics invalidates all handles.

Overlays appear above both 2D and 3D frames, including front-buffer output, without modifying game pixels or drawing state. SaveBuffer excludes them. Try Keyboard Play, Gamepad Play and Keyboard Text.

Run an existing program with touch controls

1. Open a program and choose its Run On target.
2. Choose Program > Run with Touch Controls (Option-F5 on Mac, Alt-F5 on Windows), or the Touch Controls toolbar button.
3. On first use, choose WASD, Arrow keys, Full keyboard, Custom (empty), or a Standard/Classic gamepad. Set the unused group to None. A Classic pad drives JoyX/JoyY; Standard includes two analog sticks and triggers.
4. Click a control in the preview or choose it from the Control list. Drag to move it, edit its label and size, or use Record Key to press the keyboard key it should send. A key button can include one Shift, Control or Alt (Option on Mac) modifier. For a movement pad, choose a direction and record its unmodified key. Escape is a recordable key; use Cancel Recording to stop. Add Key and Add D-pad extend the keyboard layout.
5. Choose Save and Run. The next Run with Touch Controls reuses this source file’s layout for that target. Use Program > Edit Touch Controls to change it. Normal Run continues to run without a launch overlay.

Advanced group settings control opacity (0–100%), scale (50–200%), visibility, handedness and the virtual joystick port (0–15, or -1 for the first free port). Port 0 is the default and is reserved before physical controller enumeration, including programs that read joysticks before Graphics. The landscape and portrait previews use the same placement rules as the runtime. Dragged controls anchor to the nearest corner; mirroring applies to untouched preset positions. When combining keyboard and gamepad groups, move overlapping controls apart.

Import Layout and Export Layout share versioned .b3controls.json files across platforms. A file contains at most one keyboard and one gamepad, 256 controls and 256 KiB of data. Unknown versions, fields, duplicate names, unsupported bindings and invalid values are rejected. A failed import leaves the current layout intact; a failed save leaves the previous file intact. Runtime handles are never stored.

Create Executable has a separate Include touch controls option, initially off. Enabling it opens a preview; confirm the layout to embed. Export edits do not change the saved Run layout. The generated executable contains the configuration and does not need the layout file or IDE preferences on the destination device. Original editions omit the controls editor.

Command-line equivalents (place options before the source filename):

blitzcc --mode extended --touch-controls classic game.bb
blitzcc --mode extended --touch-controls-file game.b3controls.json --target ios-simulator -o Game.app game.bb

Preset names are standard, classic, wasd, arrows, full-keys and empty. Use one layout option per build. Web and module targets do not accept launch layouts. The first matching CreateVirtualGamepad or CreateVirtualKeyboard call adopts the launch group and replaces its controls with the requested preset. The default gamepad port -1 preserves its configured port; changing to an occupied port fails without removing the old group. Later Create calls create independent groups. EndGraphics removes launch groups permanently for that run; opening graphics again does not restore them.

Try Desktop Input with WASD or a Classic gamepad. It reads ordinary keyboard and joystick commands and contains no overlay commands.

Acceleration including gravity

Call EnableAccelerometer() to request 60 Hz sampling; its optional second parameter requests 1..120 Hz. The return value confirms acceptance, not receipt of a sample. Sampling is off until requested. Status is 0 disabled, 1 starting, 2 receiving, 3 suspended, 4 unavailable, or 5 failed. MobileInputError$ reports failures. Always check sample age: -1 means no usable sample; repeated updates without fresh data increase the age.

accepted=EnableAccelerometer()
; In the game loop:
UpdateMobileInput
If AccelerometerStatus()=2 And AccelerometerAge()>=0 And AccelerometerAge()<250
    steering#=AccelerometerX()
EndIf
; At shutdown:
accepted=EnableAccelerometer(0)

X/Y/Z form a single snapshot in g, including gravity, without smoothing or a -1..1 clamp. X points right, Y down and Z out of the display in its current interface orientation. Face-up at rest is approximately (0,0,-1); upright is approximately (0,1,0). Shake readings may exceed 1 g. The Tilt Collector sample applies its own rest calibration and smoothing, without modifying raw acceleration.

Focus loss and pause stop sensor delivery and invalidate old data; resume waits for a fresh sample. Rotation also invalidates the previous sample. AccelerometerUpdated is a non-consuming flag for a new sample in this update. The simulator may have no sensor: use a physical device to check tilt, signs and orientation. This API does not provide reliable yaw, position or gravity-free movement.

Exported iOS apps include a motion usage description and keep accelerometer hardware optional. Custom packaging must preserve the appropriate motion usage declaration. The runtime checks availability and exposes native failures instead of synthesizing device readings.

Android exports use the native sensor service without a high-rate sensor permission, keep accelerometer hardware optional, and use the same orientation and lifecycle rules. The macOS IDE can run saved touch layouts on connected Android devices and export signed APKs. See Android APK export.

Command reference · Guides