mano-cua

Desktop GUI automation driven by natural language. Captures screenshots, sends them to a cloud-based hybrid vision model, and executes the returned actions on the local machine — click, type, scroll, drag, and more.

Requirements

• - A system with a graphical desktop (macOS / Windows / Linux)
• INLINECODE0 binary installed

Installation

macOS / Linux (Homebrew):

CODEBLOCK0

Windows:

Download the latest mano-cua-windows.zip from GitHub Releases, extract it, and add the folder to your PATH.

Usage

CODEBLOCK1

CODEBLOCK2

Note: Only one task can run at a time per device. If you need to start a new task, first stop the current one with mano-cua stop.

Examples

CODEBLOCK3

How It Works

The current screenshot is captured and sent to the cloud at each step. A hybrid vision solution decides the next action:

• - Mano model — handles straightforward, lightweight tasks with rapid output.
• Claude CUA model — handles complex tasks requiring deeper reasoning.

The system automatically selects the appropriate model based on task complexity.

Supported Interactions

click · type · hotkey · scroll · drag · mouse move · screenshot · wait · app launch · url direction

Status Panel

A small UI panel is displayed on the top-right corner of the screen to track and manage the current session status.

Data, Privacy & Safety

• - What is sent: Screenshots of the primary display and the task description are sent to mano.mininglamp.com — these are the minimal inputs required for the vision model to determine the next action.
• What is NOT sent: No local files, clipboard content, or system credentials are read or transmitted. All network calls are in a single module (task_model.py) for easy review.
• Authentication: No API key or credentials are required. The client identifies itself with a locally generated device ID (~/.myapp_device_id) — no secrets are embedded in the binary.
• Supply chain: The full client is open source. The Homebrew formula builds directly from this public source, ensuring the installed binary is fully auditable.
• User control: Users can stop any session at any time via the UI panel or mano-cua stop.

Important Notes

• - Do not use the mouse or keyboard during the task. Manual input while mano-cua is running may cause unexpected behavior.
• Multiple displays: only the primary display is used. All mouse movements, clicks, and screenshots are restricted to that display.

Platform Support

macOS is the preferred and most tested platform. Adaptations for Windows and Linux are not yet fully completed — minor issues are expected.