Workflow

From visual data to a task, code and execution

The project associates an image, video, or screen recording with the markup, relationships, descriptions, and results of computer vision techniques. The saved data is used in Screph Code, supported external tools or Automation Runtime. Working with the code and executing the script are launched separately.

01
Program Mode

Program mode

The mode determines the available object types and tools. General and GUI Automation support basic workflows; Industrial and UAV remain experimental.

In the project: mode and compatible settings of Selector and computer vision.
02
Capture

Visual source

The source can be a screen area, window, monitor, camera, network stream, image, video, or folder. Capture preserves the connection of the frame with the source and position on the time scale.

Output: frames, source information and timestamps.
03
Selector

Markup and relations

In Selector you create elements, features, areas of interest, annotations, and typed relationships. Layout changes can be undone and redone.

Output: editable main project markup.
04
Computer vision

Computer-vision results

A single method, sequence, or graph of methods can create masks, outlines, detections, recognized text, metrics, and a log. Methods with models require a prepared environment and model files.

Output: derived data that can be compared with the original material.
05
Review

Checking the result

Preview, overlay, mask and metrics help you evaluate the result. In supported processes, a proposed change can be accepted, modified, rejected, or recalculated.

Important: the result of the method does not change the underlying markup without the user's decision.
06
Main project

Main project

The project stores the format version, source images, markup, links, groups, descriptions and links to attached results. When loading, the structure and associated files are checked.

Output: editable task package and its materials.
07
Code and execution

Code and execution

Screph Code and supported external tools receive selected task data. Automation Runtime uses the main project along with a separately prepared script.

Launch: work with code and input actions begin only on a separate command.
Data structure

Markup, methods and recording of actions

Project relation graph

Stores typed relationships between elements and features of the main markup.

Processing graph

A processing graph with typed inputs and outputs and an execution log. It transforms visual data.

Action trace

Contains user actions and associated frames. Action CV saves suggested results next to the original entry.

Documentation

First project: source, markup and saving

Getting started