Capture + Selector + project
Source acquisition, timeline, main markup, schema validation and project-file storage.
Product status
What works now and what is still evolving
Screph is in active Alpha development. This status covers the complete product and distinguishes implemented functions, experimental processes, internal prototypes and development plans.
Program Modes
Mode readiness
| Mode | Status | Selectable | Workflow ready | Boundary |
|---|---|---|---|---|
| General | Implemented | Yes | Yes | Free-form markup, general computer-vision methods and project storage. |
| GUI Automation | Implemented | Yes | Yes | Interface markup, exports and subsequent execution through Automation Runtime. |
| Industrial | Experimental | Yes | No | Proposed-result review and confirmed markup changes are available; complete review, export and control tools are absent. |
| UAV | Experimental | Yes | No | Source selection, preflight, execution and applying changes to the main markup are available. This is not a mission-control, GIS or export system. |
| Ground Robotics | Internal | No | No | Internal deferred scaffold; user-facing operations that modify the project are disabled. |
Components
Key capability maturity
CV methods and pipelines
Classical and model-backed methods, linear and graph execution, preview, trace and verification data. Actual availability depends on the runtime and model files.
Automation Runtime
Project loading, image and text matching, condition waits, explicit input actions and Automation Manager.
Screph Code agentic IDE + Pro Agent
Screph Code uses the Monaco editor, the OpenHands runtime and a separate Pro Agent process. Builder can modify files in the selected code folder; Changes and Undo do not cover every edit, so external file comparison and a separate copy or branch are required.
Built-in AI Assistant
Implemented but off by default. Actions pass allowlist and policy checks; changes are previewed and require confirmation. Access is blocked when attachment or context permission is not defined.
Boundaries and activation →Action CV Consumer
LLM planning, local computer-vision execution, review, retry, cancel and an export foundation are available. Results require human review.
Voice and LLM connections
Speech-recognition connections and local or bring-your-own-key LLM profiles are implemented. Service access, network and model capabilities are checked separately.
Voice2Text
System-tray dictation, bindings, screenshot processing through an LLM and verifiable text insertion are implemented. The current legacy ZIP failed publication review because it contains development-environment binding data; a complete new build is required.
Process, archive and data →Account and device login
Profile, scoped device token, website balance, support chat and manual diagnostics submission are implemented. Minute usage is hidden in the current desktop Account tab.
Local and online boundary →Screph server speech
A versioned WAV API, duplicate-charge protection, 15-second balance accounting and failure compensation are implemented. Availability depends on the server-side service and balance.
Package purchase
A payment backend exists, but the user purchase page currently redirects to the homepage. No plan or checkout process is published.
Runtime Dependencies
A shared probe, install and repair interface; local, global and custom sources; signed machine-learning packs and installer integration.
Screph Cloud LLM
Planned component. The typed catalog, request, result, usage and quota contract and the desktop connection are incomplete.
Legacy Arduino serial
Retained only for compatibility and recovery of older scenarios. Arduino USB HID and FakerInput are available for new scenarios.
Development plans
Plans are separate from implemented functions
This section lists directions supported by current architecture decisions. Delivery dates are not defined.
Screph Cloud LLM
A typed Screph API is planned for catalog, capabilities, requests, results, errors, quotas and usage. A separate wire-protocol specification is required before implementation; direct service transport alone is not a complete Screph component.
Industrial / UAV completion
Industrial and UAV workspaces still need complete review, export and control stages. Ground Robotics remains a deferred internal component.
Component boundaries
The plan is to use typed adapters, versioned inter-process protocols, module-owned settings and smaller coordination components without creating a second runtime.
External extension platform
Internal extension interfaces and a registry exist, but a marketplace, general SDK, package discovery and compatibility rules are not confirmed as part of the public plan.
Current extension points →Dependencies
What may be required in addition
Models and runtime
YOLO, SAM, OmniParser and other model-backed methods require matching weight files and runtime components. Tesseract OCR requires a local Tesseract installation.
Models, weights and SHA →External services
Cloud speech recognition and LLM profiles require network access, credentials or account access. Vosk, Ollama and LM Studio provide local options when a model is installed.
Input execution
GUI Automation uses Arduino Leonardo HID, legacy serial or FakerInput Virtual HID. FakerInput requires a one-time signed MSI installation with UAC confirmation.
Current limitations
Current version boundaries
- Windows is the primary supported platform. Linux and macOS builds are not currently released.
- Alpha status does not imply unattended production automation without testing, result review and rollback.
- AI Assistant is not an autonomous executor: high-risk actions require confirmation and many execution permissions are off by default.
- Projects are stored locally. A selected speech-recognition, LLM or cloud-service process may send audio, images or context to an external service only after explicit selection.
- The published Voice2Text ZIP remains a legacy Alpha build: it lacks build information, an application license and signing, and a new build is required to remove development-environment binding data.
- A package balance in the profile does not mean purchasing is available: the current purchase page redirects to the homepage.
- Not every runtime JSON contract has a separate formal schema; the main project and export formats are versioned.