Workflow

From visual data to a task, code and execution

A project links an image, video or screen recording to markup, relations, voice descriptions, and results from selecting and configuring computer-vision methods. Saved task data is used in Screph Code or other agent tools. Data preparation, code changes and execution remain separate stages; a preview result does not change the main markup until it is applied separately.

01
Program Mode

Program mode

The mode defines available object types and tools. General and GUI Automation support the main process; Industrial and UAV are experimental workspaces with incomplete stages. Switching changes the computer-vision and Selector profiles together.

Stored: режим программы и совместимые профили компьютерного зрения и Selector.
02
Capture

Visual source

The source can be a screen area, camera, network stream or a set of images, video and folders. Capture links frames to their source and timeline position; a file path or frame number alone is not sufficient to identify provenance.

Created: кадры, ссылки на источник, положение на шкале времени и данные проверки захвата.
03
Selector

Markup and relations

Selector creates elements, features, regions of interest, annotations and typed relations. Markup is edited on a shared canvas even when several visual-data views are open.

Changed: основная разметка через команды Selector с отменой и повтором действий.
04
Computer vision

Computer-vision results

Processing runs as one method, a linear sequence or a method graph. Input data and the region of interest are validated; output can include masks, contours, detections, OCR, metrics, an execution trace and diagnostics. Model-backed methods may require a separate runtime and weight files.

Boundary: предварительный просмотр не входит в основной проект; применение сохраняет связанный результат, а команда Save создаёт отдельный технический архив.
05
Review

Result comparison and application

Preview, overlay, mask, metrics and the execution trace allow comparison with the source data. In Industrial, UAV, Assistant and Action CV, a preview result is not an applied change, and stale context blocks project changes.

Decision: результат можно принять, отклонить, изменить или пересчитать; исходные материалы проверки сохраняются отдельно.
06
Main project

Main project

The project package stores versioned JSON, source images, elements, features, matches, groups, relations, metadata and references to result files. The schema and package integrity are checked before loading.

Output: редактируемый пакет проекта; представление в модуле дерева создаётся из его данных.
07
Code and execution

Code and execution

The for_ai_agent context is passed to Screph Code or another agent tool. Automation Runtime separately uses the main project for GUI automation. Screph Code exchanges data with Pro Agent through JSON-RPC. Project validation in Automation Runtime is started separately; the selected-monitor guard applies only during actual pointer input.

Boundary: экспорт, создание кода и исполнение ввода запускаются отдельными действиями пользователя.
Three different graphs

The same word does not mean the same entity

Project relation graph

Typed relations between elements and features in the main project. This is markup data, not an executable program.

Method graph

A graph of computer-vision methods with typed inputs and outputs and an execution trace. It transforms visual data.

Action trace

Recorded user actions and linked frames. Action CV derives proposed results without changing the original trace.

Documentation

First project: source, markup and saving

Getting started