Use cases

Scenarios using Screph modules

Each scenario lists its visual input, operations in Screph, stored output and current limitation. Computer-vision methods are optional; work with code and execution remain separate stages.

ImplementedGeneral / GUI Automation

GUI scenario data for automation

Task

Coordinates alone are insufficient for a scenario: it needs object identity, visual features, states and verification data.

Input

A Windows screen or window, a still image or a recorded sequence.

In Screph

The project stores elements, relations and annotations. OCR, matching and other computer-vision methods are optional; proposed results are reviewed separately.

Output

A main project for Automation Runtime, the selected main.py, and a separate for_ai_agent context for Screph Code or another agent tool.

Boundary: The markup graph is not executed automatically. A separate scenario, target coordinate context and a configured Arduino or FakerInput input method are required.

ImplementedGeneral

Computer-vision analysis with stored parameters

Task

An OpenCV notebook alone does not link the region of interest, parameters, visual output and project changes.

Input

An image, screenshot or validated set of videos and folders.

In Screph

A region of interest is linked to a method, linear sequence or method graph. Source data is compared with the result, mask, contours, OCR and metrics.

Output

Method parameters, an execution trace and separate result files. Changes to main-project elements and features are stored only after separate application.

Boundary: Model-backed methods require matching weight files and a runtime; Tesseract OCR requires a local installation.

ExperimentalAction CV

Targets in a recorded action sequence

Task

A keyboard and pointer event log does not contain the reviewed visual object the user interacted with.

Input

An action recording with frames before, during and after the action and with coordinate transformations.

In Screph

An LLM proposes a method plan, computer-vision methods run locally, and proposed results are compared with the source frames and accepted, edited or rejected separately.

Output

Review decisions, proposed targets, provenance data, a proposed project-graph change and enriched automation context.

Boundary: The process is Experimental and requires human review. The LLM is not the source of truth; the original action trace remains unchanged.

ExperimentalIndustrial / UAV

Industrial and UAV tasks on local data

Task

General markup does not include domain-specific types for defects, measurement zones, areas of interest, changes and tracks.

Input

An industrial image or region of interest, or a UAV frame or series with available temporal and spatial data.

In Screph

A task for the selected mode is created, a configured method plan is executed, results are compared with the source data, and a supported element or region change is applied separately.

Output

Main markup and separate mode files for further investigation.

Boundary: The processes are incomplete. Screph is not MES/QMS, a CNC/PLC controller, GIS, mission control or an autopilot.