Sources and capture
A source can be a screen region, monitor, window, camera, URL, video, image or folder. Frames retain their source and timeline position.
AI tasks on visual data · Windows · Alpha
Screph: creating AI tasks on visual data.
Screph links source images and video to markup, relations, descriptions and optional computer-vision data. Screph Code and supported external coding tools can use the saved task data.
Data flow in Screph
Screph stores source materials, markup, descriptions and computer-vision results in one project. Changes to the main markup are applied separately.
Markup and relations
Elements, regions, features, groups, annotations and relations are edited on a shared canvas. Tree branches can be exported separately.
Main project and code
The main project stores source materials, markup, relations, descriptions and method results. Screph Code, Automation Runtime and external tools receive project data through separate paths.
Interface demo
Main Screph application interface
The demo reproduces the main Screph application interface: its workspace, panels and controls. Image processing, project saving and work with code are not performed in the demo.
Use cases
Visual-data workflows
Object types and available tools vary by mode. Source data, markup and descriptions remain a shared part of the project.
Legacy Systems Without API
Screens of legacy terminals, SCADA and banking systems are linked to elements, states and transition rules. The project provides data for preparing control or monitoring code.
Digitizing Expert Knowledge
A recording of an expert at work links frames, actions, selected elements and text or voice descriptions. The project can be used to prepare training material or an automation workflow.
Visual Compliance
Form fields, states and validation rules are described through elements, features and annotations. Project data can be used to prepare validation code and a result log.
Integration Without API
The interfaces of two systems, such as CAD and ERP or LIMS and Excel, are described as linked elements and operations. The task context records the data source, destination and visual interaction conditions.
ETL from Video
Screencast frames are linked to timeline positions, selected regions and action descriptions. The context can be used to prepare data-extraction code and process documentation.
Video Surveillance Operator
VMS screens and a facility map are stored as visual sources with selected regions, states and relations. The context can be used to prepare analysis and reporting code.
Laboratory Protocols
Instrument interface elements, measured values and the operation sequence are stored in the project. The context can be used to prepare serial-measurement and LIMS-export code.
Creative Pipelines
Panels and controls in Adobe, DaVinci or Blender are linked to operation and parameter descriptions. The context can be used to prepare batch processing and rendering code.
Accessibility Agents
Interface elements are marked with states, constraints and accessibility requirements. The context can be used when preparing an adaptation for a specific application.
Sports Analytics
Broadcast regions are linked to timestamps and event annotations. The context can be used to prepare event-extraction and statistics code.
UX Annotation
The purpose, states and relations of interface elements are stored in the project. This data can be used for checklists, metrics and A/B hypotheses.
Trading Operations
Terminal elements and states are marked in screenshots or a work recording. The context can be used to prepare reviewable code and reports; operation execution remains an external stage.
Digital Process Twins
Business-process screens are linked by sequence, states and temporal data. The project can be used as a basis for a process model and an executable workflow.
EdTech Assessments
Learning-environment elements, task states and validation rules are stored in the project. The context can be used for assessment code and learning materials.
Legacy Healthcare Systems
A closed healthcare-system interface is described through elements, states and relations. The context can be used for external integration without modifying the original application.
Visual Regression Testing
Reference interface states, regions and comparison rules are stored in the project. The context can be used to prepare test code and reports.
The project stores schema-versioned JSON, source images, markup, relations, groups and result manifests.
Method data remains a separate result. A proposed change becomes part of the main markup only after an explicit user decision.
Automation execution, external LLM use and code generation have their own dependencies and limitations.
FAQ
Product boundaries at a glance
How does Screph generate and run code?
Screph Code receives context through a separate Pro Agent process. The Builder agent can modify the selected workspace, so inspect the result with an external diff; there is no universal per-file Apply. Running a file or GUI automation remains a separate explicit stage.
Is internet access required?
Markup, project persistence and classical CV methods run locally. Cloud speech/LLM providers, model downloads and some integrations require network access.
Are projects uploaded to the cloud?
Projects and images are stored locally. Transfer happens only in an explicitly selected cloud/LLM/speech flow; the Assistant disallows images and artifacts by default.
Are Industrial and UAV ready for production use?
No. They are experimental workspaces for preview, review of proposed results and separate application of markup changes. A complete production process and autonomous execution are not available.
Main Screph project
Project data and code
The project contains a visual source, markup, relations, text and voice descriptions, and optional computer-vision data. Saved data is used in Screph Code or passed to a supported external tool through a separate action.