Computer vision models

YOLO, SAM, and OmniParser operations require prepared dependencies, a compatible model file, and a model selected in the operation settings. Training custom models is in the Datasets and training tab and uses a separate catalog.

General procedure

Check the machine learning component on the dependencies page.
Open the settings of the models of the selected family.
Download a supported model or provide a trusted local file.
Select the active model and check that it loads.
Test the operation on a single image to check model loading and the result format. Evaluate quality on your own data.

Model families

Family	Purpose	Features
YOLO	Detection, segmentation, and pose estimation, depending on the selected model.	The model task must match the operation.
SAM	Segmentation using a point, box, or another prompt.	Different options require different files and resources.
OmniParser	Search and description of interface elements.	A complete process may require multiple consistent models.

Downloads and privacy

Model downloads start only after a user action. The model server receives the usual technical data of an HTTPS request, without the project, annotations or source images. Once the model is downloaded, processing runs locally.

For a closed network, you can use pre-prepared local files. Load models only from trusted sources: the model file may contain data that is processed by software libraries when opened.

File verification

If a checksum is published for a model, Screph can compare it with a local file. A match confirms the integrity of the file, but does not guarantee the quality of the model or suitability for a particular task.

Typical reasons for unavailability

Machine learning dependencies are not prepared.
The file is missing, damaged, or does not match the selected family.
The active model is not selected in the settings.
There is not enough memory or free space.

Preparing components is described in the "Dependencies and Updates" section.

Trainable models and the training environment

Learning supports single-image classification, box detection, and instance segmentation with binary masks. The catalog contains supported TorchVision and Ultralytics architectures: YOLO11 Detect, YOLO11 Seg, and RT-DETR-L. The dataset task must match the model task; arbitrary .pt/.pth files or YAML are not imported into the training catalog.

Practice example. Changes apply only here and reset when you leave the example. In the learning dataset, you can select a sample, assign a class, and view validation status. Changes remain in the example's memory.

Trainable model combines groups, recommendations, and help. Extended catalog… searches models for the current task and mode; Add model… adds a supported architecture to the main selection. Pretrained weights are downloaded by a separate action with integrity checks; missing or damaged weights block the corresponding run.

When the environment is not ready, Start training opens the nonmodal Model training environment window. Select a supported local ZIP pack or folder, or use the offered download and installation. Only training components are checked; other component diagnostics open separately. System Python is not substituted for the selected ML environment.

Input size and processing depend on the architecture: SSDLite320 uses 320×320, and RT-DETR-L requires at least 128×128. For YOLO11 Seg, exact dataset masks are converted into an internal training format; the original masks remain available for independent evaluation. Selecting CUDA requires an available device and does not silently switch to CPU.

Practice example. Changes apply only here and reset when you leave the example. The example provides training settings and pages. Training and model downloads are not started.

Candidate, active model, and final program

Successful training creates a candidate with metrics and verified files. Promotion saves a separate model and evaluation evidence. If the model was published but did not become active, repeat only the activation action; retraining is unnecessary.

Checking a candidate or promoted model in Automation Runtime uses an explicitly selected source and ML environment. This is a prediction preview; it does not create a final evaluation or include the model in the final program.

The application profile stores the model description and role separately from its weights. Inclusion in the final program must be enabled explicitly and requires an input source and a use for the result. The Agent receives verified files only for an authorized primary model. Manual model export is separate from handoff to the Agent.

Evaluate accuracy on data from your task, including new scenes, separately from the training example and preview. Distributed Ray training is experimental and available only for classification; empirical automatic tuning across a series of training runs is not yet available in the interface.