Attention heatmaps for UI screenshots — with numeric scores and A/B compare, not just pretty pictures.

Predict where users' eyes land on a design, get a machine-readable attention share for any region, and diff two variants to see which one wins and by how much. Local, offline, MIT-licensed.

pip install hotgaze && hotgaze run screenshot.png

Status: v0.1 alpha — on PyPI. The fast heuristic backend, region scoring, A/B compare, and deep UNISAL backend are shipping. Faces layer (YuNet) is available via --layers faces. API and CLI may change before v1.

Every existing attention-prediction tool — paid (HeatScope, Attention Insight, EyeQuant) and free (Foveacast) — stops at a colored overlay for a human to eyeball. That's fine for a designer squinting at a mockup. It's useless when you want to:

• Compare two design variants and quantify which one draws more attention to the CTA.
• Wire attention checks into CI ("your button just lost 23% of its attention").
• Script or automate any part of the design-review loop.

HotGaze outputs the picture too, but the picture isn't the point. The numbers are.

pip install hotgaze          # core (fast backend)
pip install "hotgaze[deep]"  # + deep UNISAL backend

Python 3.10+. Runs on macOS and Linux. No cloud, no API keys, no telemetry.

git clone https://github.com/suryakosaraju/hotgaze
cd hotgaze
pip install -e .

# Generate an attention overlay
hotgaze run screenshot.png -o overlay.png

# Score a specific region — how much attention does the CTA get?
hotgaze score screenshot.png --region cta:250,200,200,35 --json

# Compare two variants — which one wins?
hotgaze compare landing_a.png landing_b.png --region cta:250,200,200,35

Score any region. Attention share, peak value, rank — canonical JSON, deterministic on the same machine:

$ hotgaze score design.png --region cta:250,200,200,35 --json
{
  "schema": 1,
  "regions": [
    {"name": "cta", "share": 0.035, "peak_value": 0.769, "rank": 1}
  ],
  "focal_points": [...]
}

Compare two variants. Per-region deltas, plus a 3×3 spatial grid showing where attention moved:

$ hotgaze compare landing_a.png landing_b.png --region cta:250,200,200,35 --json
{
  "compare": {
    "per_region_deltas": [
      {"name": "cta", "share_a": 0.035, "share_b": 0.022, "delta": -0.013}
    ]
  }
}

Variant B lost 37% of the CTA's attention share. That's an actionable number, not "the heatmap looks about the same."

Compare a CI-generated screenshot with a baseline using the offline fast backend. The Action uploads canonical schema-v1 score/compare JSON and both overlays, then adds a region table to the job summary.

permissions:
  contents: read

steps:
  - uses: actions/checkout@v4
  - uses: actions/setup-python@v5
    with:
      python-version: "3.12"
  - uses: suryakosaraju/hotgaze@main
    with:
      baseline: tests/screenshots/baseline.png
      candidate: tests/screenshots/candidate.png
      regions: |
        headline:0.10,0.08,0.60,0.18f
        cta:0.35,0.55,0.30,0.12f
      failure-threshold: "0.02"

failure-threshold is an absolute attention-share loss: 0.02 allows a region to lose up to two percentage points. The step fails only when a configured region loses more than that threshold or execution is invalid. Without a threshold, the Action reports differences without failing.

No PR comments, write permissions, or GitHub token input are required. The default fast backend downloads no model weights. Selecting backend: deep explicitly installs PyTorch and permits HotGaze's one-time checksummed UNISAL weight download. See the complete consumer workflow in .github/examples/hotgaze-consumer.yml.

• CLI: hotgaze run, hotgaze score, hotgaze compare, hotgaze info.
• Fast heuristic backend (default): spectral-residual saliency + contrast + center bias + F-pattern reading prior. No downloads, works offline, sub-second on CPU.
• Deep saliency backend (--backend deep): UNISAL (Apache-2.0), CPU-only, deterministic per-machine. Install: pip install hotgaze[deep]. Weights download on first use (one-time, ~30 MB).
• Faces layer (--layers faces): YuNet (MIT) face detection adds attention blobs over detected faces.
• Versioned JSON output: schema v1 covers both score and compare modes so downstream tools don't break on new features.
• Deterministic: same image + same config + same machine → byte-identical JSON.

--backend deep requires pip install hotgaze[deep] and a one-time weight download on first use. Both backends are fully offline after the initial fetch.

• Not real eye-tracking. It's a prediction from computer-vision priors. Useful for early design review; not a substitute for a user study.
• Not a conversion oracle. Attention share correlates with visibility, not with conversion — high attention on a bad CTA still doesn't sell.
• Not a designer GUI. It's a scriptable tool for developers. A GUI/Figma plugin is roadmap, not v1.

• v0.2 — GitHub Action for attention regression testing on PR screenshots.
• v0.3+ — UI-tuned text/saliency models, Figma plugin, macOS wrapper, benchmarking against public saliency datasets.

MIT. See LICENSE.

Third-party models and dependencies are recorded with their licenses and redistribution status in LICENSES-THIRD-PARTY.md.

Predictive saliency stands on decades of vision research. The deep backend builds on UNISAL (Droste et al.). The fast backend implements Hou & Zhang's spectral-residual approach (2007). Face detection uses YuNet.

Built with Claude Code & Codex as orchestrators and DeepSeek as the implementing agent.