Screph provides tools for frame capture, annotation, computer vision and software development. Capture acquires images and video, Selector stores annotations, and the visual builder connects computer vision operations. Project data is available in Screph Code and when preparing automation scripts.
Capture can use a screen region, window, monitor, camera, URL stream, images, video or a folder as its source. Each frame retains its source and position on the timeline.
Selector links a visual source to elements, features, regions, annotations and relations. Annotations are edited on the canvas and saved in the project with a history of changes.
The visual builder connects operations into a sequence or graph. The interface shows each step’s parameters and output: images, masks, contours, detections, text, metrics and execution logs. Applying a result to annotations is a separate action.
Edge detection, thresholding, segmentation, contours, matching, comparison, alignment and motion analysis.
YOLO, SAM and OmniParser require a prepared runtime environment and suitable model files. OCR with Tesseract requires local installation of Tesseract.
After reviewing the result, the user accepts or rejects the proposed annotation change.
The main project stores the format version, source images, annotations, relations, groups, descriptions and links to attached operation results. The project structure and linked files are checked when loading and saving.
Selected project data can be passed to Screph Code and external tools. GUI Automation also requires an execution script.
Loading a project, searching for visual and textual states, waiting for conditions and acting through the selected input method. Running the script is always done separately.
Before running, prepare a script and a suitable input method, verify coordinates and monitor the target application.
Screph Code works with a code folder and selected project data through the installed Pro Agent component. Builder can modify files in that folder. Create a copy or branch before starting, then review the changes with Git or another comparison tool.
The agent explains results and creates and edits annotations and computer vision workflows. It is disabled by default. In step-by-step mode, the user confirms changes. In delegated mode, the agent continues the authorized task until completion, cancellation or a need for user input.
Projects and images are stored locally. Requests to a cloud model send the selected request data. Sending images and project files to the agent is disabled by default and requires permission.
Agent setup and limitations →Action CV associates an action with frames before and after it, runs selected methods locally, and stores suggested goals along with the user's decision. The result requires human verification.