Screenpipe

Screenpipe is a local-first tool for agents that need to observe and interact with desktop application state — it continuously captures screen content, runs OCR, transcribes audio, and exposes a local REST API for querying recent screen history. Primary use cases: AI assistants that understand what the user is doing on screen, automation agents that trigger on desktop app state changes, and productivity tools that index screen activity. Fully local — no cloud dependency for data capture. Confidence is docs-derived.

Local REST API at http://localhost:3030. Resources: search, frames, audio, health. GET /search?query={text} performs semantic search over screen content history (OCR text + audio transcription). GET /frames?start_time={ts}&end_time={ts} retrieves recent screen frames. GET /audio/transcriptions retrieves recent audio transcriptions. POST /tags creates tags for content labeling. Screenpipe runs as a local daemon; all data stored locally (SQLite + frame store).

No authentication by default on the local API (localhost-only; no inbound external access). Optional auth token for secured local deployments. Desktop app permission model: macOS Screen Recording and Microphone permissions required. Open-source — access model is entirely local with no cloud credential requirement.

Daemon reliability: Screenpipe runs as a background service; auto-restart configured via system service manager. OCR accuracy depends on screen content type and display scaling. Audio transcription: Whisper-based (local inference). macOS permission changes require app restart. Resource usage: continuous capture uses CPU/memory — tunable via capture interval settings.

docs.screenpi.pe covers installation, API reference, plugin development, and AI integration guides. Getting started: install via brew/script, grant permissions, first API call in under 5 minutes. Open-source MIT license (GitHub: mediar-ai/screenpipe). Community via Screenpipe Discord. Rapidly evolving — monitor changelog for API changes.