This guide covers Screph's main user workflows: element and feature markup, image inspection, annotations, the relation graph, automation preparation, saving/loading, data export and IDE integration.
The available panels, task types, and markup depend on the selected program mode. Before starting a domain workflow, choose the mode in the Selector header or under General Settings → Development Platform and read the Program Modes guide.
1. Selecting and Editing Elements
Screph provides flexible tools for precise selection and editing of interface elements.
Working Modes
Select (S): select an existing element or feature to inspect its properties or perform another action.
Create + shape: first choose the intent—an image element, container area, pixel/region feature, pattern, grid or target point—then an allowed shape: rectangle, ellipse, polygon or lasso. Available intents depend on the program mode.
Move (M): move the selected object without changing its shape.
Resize (R): adjust the size, vertices or outline of an existing rectangle, ellipse, polygon or lasso.
Hierarchy (H): create a parent → child relation only between two elements. Association, flow and logical relations are created separately through the Relation Graph or the Create relation action.
Shape creation and navigation
Unified intent/shape selection:
C, E, P and L select rectangle, ellipse, polygon and lasso for the current intent. Shift+A directly selects a rectangular selection area. Feature shortcuts are X pixel, A region, T pattern, Y target, G region grid, Shift+P polygon and Shift+L lasso; the mode profile may hide a disallowed kind.
Gestures: create rectangles and ellipses by dragging, a lasso by drawing while holding the button, and a polygon by placing vertices; finish a polygon with a double-click or by clicking its starting point. An unfinished shape does not create a canonical object.
Pan/zoom: the middle button or Space plus a left-button drag pans the viewport even over an active tool. The wheel zooms; Ctrl+wheel scrolls. Ctrl+=, Ctrl+- and Ctrl+0 are also available.
Interaction
Selection: Click on element in Select Mode.
Canvas area set:
Ctrl+click adds or removes an element/feature from the set that InfoPanel can wrap in a selection area.
Tree multi-select:
Ctrl selects individual rows and Shift selects a range.
Context Menu: Right-click an element-tree row to select it, hide/show its subtree, delete the element, remove relations, open the Module submenu, convert it to a selection area, or expand/collapse the subtree.
Tree visibility: The eye icon on the right side of an element name temporarily hides or shows that element, its children, and their canvas relations. Hidden elements are not deleted from the project, cannot be clicked, cannot become parents of new areas, and cannot receive new relations.
Subtree module: The Module menu creates a compact module.json for the selected tree branch under <project>_artifacts/modules/ plus a generated helper class next to it. The main project remains the source of truth, while the module is used as a small agent or codegen context. A small M badge in the tree row shows that the branch has a module and indicates its state. The same menu can copy an agent-ready context or refresh stale modules. During creation, you can keep refs to shared assets or build a self-contained module folder. The agent task is created next to the module, and Send to Codex writes a request through the existing Codex VS Code bridge. Opening the selected IDE/code tool and Codex sending use the project root so the agent can see dependencies and edit other project areas.
Delete: select an element or feature and press Delete. Screph asks for confirmation; an object lock, including a lock on related objects removed with an element, blocks the operation.
Move, vertices and geometry commit
Move/Resize changes an existing canonical record rather than creating a copy. An object lock or the corresponding canvas-layer lock blocks the gesture. For a polygon/path element in R mode, vertices can be dragged; Tab/Shift+Tab changes the active vertex, arrow keys move it by 1 px, Shift+arrow by 5 px and Ctrl+arrow by 10 px. Enter inserts a vertex on the nearest edge under the cursor. Double-click a vertex to delete it or an edge to insert a new one.
Element geometry has a commit boundary. After Move/Resize, the change remains a draft until a selection, tool or source switch or another finalize trigger. If CV contexts refer to the old geometry, the prompt policy shows a dialog: commit or roll back geometry and independently refresh normal CV, Video CV and mode workspaces. Without a selected refresh, old results remain visible as stale; heavy methods are not started automatically. The mark_stale and refresh_current policies make the corresponding choice without a dialog. Feature geometry records its own project-history action when the gesture finishes and does not use this element dialog.
Feature lifecycle
A feature is a canonical project object, not only a canvas rectangle. Depending on the active Selector profile, the project can store pixel (a point and sampled color), region (geometry and summary), pattern, region_grid and static_diff. These records belong to one project history and are stored in image_features. The Y tool does not create a separate kind; it assigns a target point to the selected geometry feature and stores its offset from the geometry.
Duplicate: the context menu creates a new ID/display ID and a copy name, retaining geometry/summary/settings but clearing crop, mask, preview and their asset metadata. It is a new object, not another reference to the source asset.
Make pattern: creates a separate pattern from the source feature's geometry, samples the summary from the current image, places the target at the center and starts with similarity 0.8, exact=false and resize 1.0. A real source image is required.
Grid and derived region:
Create grid from area creates a new region_grid with a 3×3 grid by default; rows and columns can be changed from 1 to 64 in the inspector. The Derived region submenu offers grow, inset, left/right/above/below; the amount is entered in pixels (16 by default for grow/inset, 32 for an adjacent region), and the result is clamped to the source-image bounds.
Static diff: the item appears when another feature was selected before the context menu opened. Both features need geometry and an available source image; the result stores the base/compare IDs, a combined bbox, changed pixels and diff percent. The default threshold is 16.
The feature inspector exposes the name, canonical description, authoring mode, owner, source, geometry/color/target, pattern settings, grid settings, relations, issues, a read-only JSON projection and separate matches. An owner can be manually assigned to the selected element or detached; automatic selection-area membership is a separate mechanism. Changing the target, geometry, source or grid marks existing bitmap assets stale when they exist. The active profile capability and an object lock can reject any such mutation.
Feature artifacts are a separate layer. The properties panel and Capture → Features tab show crop, mask and preview, their refs, source hash, bbox-at-capture and status (saved, live preview, stale or needing repair). A live preview does not mean that a file has been written. Resaving requires a saved project with an artifact store; one feature or a batch of problematic features can be repaired. Restore missing source snapshots first, then resave assets; a failed refresh rolls back the changed files.
Selection area: membership and boundaries
A selection area is a geometry-only container: it may use rectangle, ellipse, polygon or lasso geometry and stores element/feature membership, but no pixels, crop or image series. Importantly, current membership checks use the area's bounding rectangle rather than the exact filled ellipse/polygon/path. By default, objects whose full bounding box is contained by that rectangle are collected automatically. With nested areas, an object is routed to the nearest, smallest matching area.
Create: draw with the Container area intent, use Make area on an ordinary image-free element, or collect elements/features with Ctrl+click and press Create area in InfoPanel. Area around wraps the selected element, its descendants and features with 8 px padding.
Manage membership: Refresh membership reruns automatic collection; Attach... selects one contained object, Attach all handles every candidate, and Detach... removes an object from the current area.
Membership lock: freezes one element or feature in its current membership state and prevents automatic reassignment. Manual detach enables this lock; manual attach clears it. This is a canonical object property, distinct from a persisted object lock and a session-only layer lock.
Move and convert:
with default settings, moving an area moves its child elements and features. To ordinary removes container semantics. Image payload blocks conversion in the other direction; existing frame bindings are first released through the capture controller, and a release failure cancels the operation.
A selection area is not a markup group. An area is a visible geometry-driven container with membership controls. markup_groups are separate named canonical aggregates of existing elements/features with roles and sequence indexes. They load, save and protect members from deletion. A typed TRACK_GROUP can be created or updated through confirmed UAV candidate-to-markup apply, but Selector has no separate free-form visual CRUD editor; other producers use the canonical API.
Element Properties
Name: Unique name (auto or user-defined).
Type: a type from the active markup catalog for the selected program mode.
Coordinates and Dimensions: Position (X, Y) and size.
Hierarchy: Information about relationships with other elements.
Annotation: Text description of the element.
2. Image Inspection and Measurements
Pixel-inspection tools are available on InfoPanel's Image tab, while the histogram, line profile and evidence comparison are on the Capture/Images dock's Image Analysis tab. They operate on the active canvas presentation without changing its source pixels.
A/B/C/D presentations
The layout control in the menu-bar toolbar creates up to four panes of one Canvas: A; A/B side by side or stacked; A on the left with B/C on the right; or a 2×2 A/B/C/D grid. These are not project copies: canonical elements, features, relations, selection and project history are shared by every pane. The selected visual material and its local display settings may differ.
Selection and activation:
use a pane's settings button to rename it and select Source, a working CV result or a diagnostic presentation, its display mode and crop source. Activation routes the canvas tools to that pane; a stale, unmapped or read-only presentation cannot own editing. Such an activation request is rejected; if the current active pane becomes incompatible, ownership returns to A.
Linked navigation:
linked panes synchronize scale, viewport center and the canvas-space cursor. Pane A always belongs to the linked group; B/C/D can be unlinked for independent zoom/pan. Navigation from a compatible read-only pane can still move the linked group without giving it permission to edit markup.
Close and Reset: A cannot be closed; B/C/D are closed explicitly, and closing a pane does not delete its selected presentation. Choosing a smaller layout never removes existing panes implicitly. The global Reset is stronger: it assigns Source to A, activates it, closes B/C/D and returns to the single-pane layout.
What persists. Layout, splitter sizes, pane order and labels, assignments, linked/display/crop-source settings and safe presentation descriptors are stored in local ScrephData/local_state/selector_layout.json, not in the canonical project package. The snapshot contains no pixels and does not retain the current zoom/center. A working processed_raster restores its recipe/descriptor; after startup with the same Source, its result is marked for explicit recomputation.
Pixel, magnifier and navigation
Pixel probe: shows coordinates, RGB, HEX, HSV and source immediately under the cursor, then records a stable value after 0.5 seconds. Individual stable fields or the full set can be copied.
What is sampled: the probe and line profile read the exact material layer of the active presentation. If that material is unavailable or not mapped to the canvas, Screph reports a diagnostic instead of silently substituting the raw image.
Magnifier: Context 31×31, Detail 15×15 and Values 9×9 modes, with independent grid, crosshair and frame toggles.
View: Fit, 100% zoom, pixel grid and a resizable navigator/minimap. The navigator can be toggled, resized and used to recenter the viewport.
Canvas layers
The layer panel controls six fixed groups: source image, CV overlays, elements and relations, features, assistant preview and measurements. Each group has visibility, opacity, edit lock and move-up/down controls.
Persistence boundary. Layer visibility, opacity and order are part of canonical project state. Locks are session-only and reset when the project state is restored.
Tone & Profile
Histogram: combined RGB, individual channels or gray/LUT plus levels are display-only. This is a non-destructive display transform, not a change to the material or project image.
Line profile: draw a line on the active presentation to obtain distance, coordinates, R/G/B and luminance along it. Multiple measurements can be selected or deleted, have local Undo/Redo, and the current profile can be copied as TSV or explicitly exported as CSV.
Measurements are temporary. Lines and their local history are not written to the project package and are cleared when the source changes. CSV is an explicit standalone export, not part of normal project saving.
Compare & Evidence
The read-only panel pairs presentation A with a selected B, provides Fit all and 400 ms flicker comparison, then exposes Summary, Diagnostics and Provenance. Reports are bound to source/context revisions, so a mismatch marks evidence stale. Registration, stabilization and industrial components use specialized inspectors; a component report also offers Accepted/Rejected filtering, a metrics table and a read-only overlay for the selected component.
3. Annotations (Text/Voice)
An annotation is a canonical human-readable object description, not a separate prompt file. Text and voice input update the same target field and participate in project history.
What can be described
Element / feature / feature match: the description_human field in the InfoPanel card. The voice panel writes its transcript to the same field; the current UI does not create a separate parallel “voice transcript.”
Relation edge: the description in the edge inspector or Relation description dialog. Its embedded voice widget appends recognized text to the draft; the canonical edge changes only after OK.
Image Series entry: the per-entry annotation field. Text or voice in its dialog commits on OK; Cancel leaves the entry unchanged. The whole CV-run note is edited separately and is currently text-only.
Voice Annotation
The selected recognizer—local Vosk or an explicitly configured Yandex, Google or OpenAI path—is used by the shared voice widget. An unavailable provider is not replaced automatically.
Select an element, feature or feature match. Screph requests an annotation automatically after creating those objects; a feature target change does so only when no description exists. Image-series entries, CV saves, edges and CV-run notes do not auto-start the voice widget.
For a manual start, press Voice annotation. The widget opens with the current description and makes several short attempts to start recording automatically.
Speak clearly into the microphone; the record button toggles recording and stopping.
Review and, if necessary, edit the recognized text, then press Done or Ctrl+Enter.
Escape or the close button closes the widget without emitting a result. Losing focus during active recording/recognition, moving to another selection operation or starting another voice session may finalize the current session; use the close button or Escape for an explicit cancel. Under General Settings → Operating mode, Copy and hide and Copy, clear and hide write only to the clipboard and do not update the object.
Annotation and semantic command
Both paths may invoke a semantic action.Voice annotation first stores the transcript on the selected target, then, when process_voice_annotations is enabled, sends it to the router with target context. Voice command (V) sends text without storing an annotation. Supported typed actions include changing an element/feature/edge type, hierarchy to the previous element, switching a tool, wrapping a set in a selection area, and confirming/rejecting pending review; this is not an arbitrary automation command.
Semantics is enabled by default, voice annotations use fast_skip_non_commands, and a high-confidence rule at or above 0.72 may apply immediately. The ✓/× buttons activate only for an intent actually placed in the review queue. require_confirmation_for_llm confirms an intent already marked requires_review; it does not itself force every LLM result through review. For guaranteed dictation-only behavior, clear Process voice annotations or select Do not run semantics for annotations.
4. Saving and Loading Projects
A current project is a directory containing canonical <name>.json, source images and related artifacts. The current load/save flow no longer uses the .sgaip format.
Saving
File → Save Project:
Overwrites existing file or creates new. Hotkey: Ctrl+S.
File → Save Settings... (Ctrl+Shift+S): choose the project environment, code tool, post-save behavior, save location and feature-projection profile. The dialog can apply the profile without writing the project, or apply it and save immediately.
Do not confuse this dialog with the legacy Element Selection → Files & Formats tab in the main Settings window: its visible timestamp, PNG/JPG and JPEG-quality fields are persisted, but the current project writer does not use them.
Project Environment
Available save results depend on the program mode. GUI Automation can target Screph Automate or PyAutoGUI. General, Industrial Vision, and Autonomous Unmanned Vehicles provide the corresponding built-in Screph CV (OpenCV / NumPy) environment and the “No tools — Screph project only” option. The IDE or code tool is selected separately and does not switch the program mode.
Current project overwrites the existing current path; when no current path exists, Screph asks for a name and creates a directory under the default project root.
Subproject always asks for a name and creates a directory inside the current project directory; without a current project it uses the default root. The code-tool workspace remains the parent project root.
Always ask shows a name and, only when a current project exists, a subproject choice.
The dialog's “Will be created” list is a target summary, not a complete package manifest: it omits the automatic for_ai_agent, the _artifacts directory and conditional timeline/frame-bound sidecars.
elements, features, geometry, annotations, typed relations and markup groups;
tree-module registry entries, CV result references and the selected save profile;
save/load diagnostics and the artifact-integrity report.
After a successful save, Screph automatically creates <name>.for_ai_agent.json next to the project. It is a derived projection for Screph Code/IDE, not a file to reopen in Selector. Timeline and frame-bound captures use their own sidecars when present; see the data and export guide for the complete package.
Loading and recovery
File → Load Project (Ctrl+O) opens the Project Manager, not a plain file picker. It recursively considers up to 2,000 eligible JSON files under the selected root, skips known service files and artifact directories, and separates entries into Projects, Autosaves and Problems.
The selected project exposes history, a graph/tree minimap, counts, tags, notes and diagnostics.
The manager can open, duplicate or rename a project, save the current state to a selected entry, restore an autosave under a new name and export the map as GraphML/HTML/JSON.
Select file... lets you choose a JSON file explicitly. Manager opens a canonical project or autosave only after schema checks and when it contains at least one element, edge, feature or feature match. An agent export does not become an editable project: when its canonical reference exists, Open resolves to that canonical file; tree modules and arbitrary JSON remain non-openable.
Before opening, Manager diagnostics cover the JSON role, read/schema failures, missing paths and an empty domain; this is not a fresh full artifact-integrity scan. After an actual load, InfoPanel shows the current domain load report together with saved save reports.
At startup, Selector separately attempts to restore autosave_session.json. This startup flow accepts elements or edges, or a loaded base image alone, so a screenshot-only autosave may recover at startup even though Manager treats an object-free domain as non-openable. The Capture timeline has a separate recovery mechanism.
A Selector or Capture autosave failure during close requires an explicit user decision. Selector writes its close autosave only for a dirty project; Capture may persist recording frames separately.
Duplicate and rename apply only to regular canonical projects, while restore applies to an autosave. They operate on the whole directory and align the primary JSON, timeline and frame-bound sidecar, but do not rebuild the agent export or migrate Manager-local tags and notes. Save again before a handoff.
Remove from history and Clear missing only change the recent-path list; Project Manager does not delete project files. Tags and notes are stored separately by absolute path and are not part of the portable package. When creating a new project, Screph offers to save the current one, continue without saving or cancel; the existing timeline can be explicitly carried into the new project.
5. Working with Image Series
An element's Image Series is its collection of saved visual states and attached CV outputs. It is not the Capture timeline itself: the timeline supplies source frames, while series entries belong to the canonical element and move required images/artifacts into the project package when saved.
Selected element series in InfoPanel
Initial view: for a normal new element, Screph attempts to create entry 0 from the selected crop source: the active canvas presentation material or the explicitly selected canonical source. An unavailable source is not silently substituted.
Add current view to series: does not take a plain screenshot. It runs the current CV method through a forced Apply/Save for the selected element and assigns the next numeric annotation. It therefore requires the CV panel and may create multiple output tiles for one run.
Run card: outputs sharing a cv_run_id are grouped together. Double-click opens the file; an output's context menu can open it, make mask/grayscale/cutout primary, or remove that output. Removing the final output also removes the series entry.
Description and review:
the card menu edits its annotation by text or voice, edits the whole CV run note, opens the Assistant with entry context, or deletes the entry. Numeric and automatically generated labels carry a ! marker until manually reviewed.
A geometry-only selection area is not an image. Initial/current views or frame crops cannot be attached to it; attach images to child elements. Use the separate image-frame type for a cropped temporal series.
Frame series and Capture timeline
An image frame's context menu provides Capture current frame, Record series from frame/Stop series recording, and Show captured frames on timeline. During recording, moving to a new frame automatically stores the frame crop; the same frame index is not captured twice for one image frame. The frame must be unlocked, have valid geometry inside the image, and another frame cannot be recorded concurrently.
Changing the active frame's geometry stops or restarts recording according to its setting. Deleting a timeline range remaps surviving frame bindings, while deleting a series entry first clears its frame-bound binding. The index lives beside the project as <name>.frame-bound-image-series-index.json; entries and images themselves are saved through the canonical project and its artifacts.
Source timeline navigation and editing
The timeline supports first/previous/next/last navigation, playback, speed and loop.
To delete content, select a range and choose Delete selection from timeline or press Ctrl+Del. The context menu can also remove a clip or clear the timeline; original media files are unchanged.
The Capture tab is described in detail in the dedicated guide.
6. Markup Relation Graph
The Relation Graph visualizes typed links between canonical elements/features. An edge describes a semantic relationship and its properties; it does not automatically become a click, transition or execution step.
What the Graph tab shows
Nodes: elements, selection areas, features and tree-module roots. Status reflects selection/visibility, module freshness and feature issues.
Hierarchy: parent → child links are derived from canonical parent state and remain distinct from arbitrary relations.
Ownership:
toggles system owner → feature edges; this is a visual projection of owner_element_id, not another editable user relation.
Relations:
shows or hides user edges. Clicking a node selects its element/feature, clicking an edge opens its properties, and a node visibility control hides a feature or an entire element subtree.
Creating and editing a relation
Open an element or feature context menu, choose Create relation..., then select the other endpoint.
Choose Association, Flow or Logical relation; Hierarchy is also available for an element → element pair. Reverse direction swaps source and target before creation.
Select the edge on the canvas or in Graph. The inspector shows ID/source/target/type, an editable description, read-only JSON properties, and can repoint source or target to an existing element/feature.
The edge context menu also changes type, edits the description with voice input, opens the Assistant for that relation, or deletes it.
Screph rejects self/missing endpoints, a duplicate directed pair, feature hierarchy and hierarchy cycles. Assigning a new parent detaches the child from its previous parent. Feature relations also depend on the active mode's FEATURE_RELATE capability.
Object and layer locks are different. A specific element, feature or edge lock is stored in canonical canvas_object_state, blocks related mutations and participates in project Undo/Redo. A whole canvas-layer lock remains session-only.
Feature matches are not edges
The feature properties' Matches group creates a match from another feature or element and stores kind, score 0…1, source, bbox/target and description. These records persist separately under canonical feature_matches; Relation Graph does not currently render them as user edges.
Recorded action order belongs to the Action Trace, while executable logic belongs to an automation script/runtime. The CV Method Graph is a third, separate computation graph.
7. Data Preparation and Emulation
In GUI Automation mode, Screph prepares a structured automation context: images, selections, annotations, relation graphs, screen states, and an exportable project.json. This data can be used for action emulation, for the built-in Screph Code, and for external IDE or agent workflows.
In other modes, saving targets the canonical project and the mode's built-in CV environment; GUI automation consumers are not offered there.
Current GUI Automation outputs
Screph Automate:project.json, main.py and a runtime guide; the explicit Open Result button opens Automation Manager.
PyAutoGUI:project.json, main.py and a dedicated guide. Execution requires an available script.runtime.
After canonical save, Screph validates the coordinate contract and then prepares target files. An error blocks a safe automation handoff but does not remove canonical/agent files already written.
Opening a code tool after save is a best-effort downstream action: a disabled or unavailable backend, or a launch/send failure, does not roll back the saved project. The context is for_ai_agent when available, otherwise canonical JSON; a compatible backend may also receive a generation request.
The detailed GUI flow for the built-in Screph Code is documented below in sections 8-15: launch, autostart on save, transferred context, Builder, edit preview, and recovery.
8. IDE Integration
Screph integrates with IDEs and code tools in two modes. The first mode is the built-in Screph Code, which can be opened manually or started automatically when a project is saved. The second mode is exporting the project, JSON, and related artifacts to external IDEs and agent pipelines.
Integration Methods
Export Data: Save in format for import to other tools.
Copy Code: Clipboard → IDE.
File System: Save as files.
If you are working with the built-in Screph Code, continue to sections 8-15 below: they describe the Screph Code window, the Project and Builder tabs, shortcuts, diff preview, Undo, Checkpoints, and settings.
9. Screph Code GUI: General Context and Preconditions
The detailed guide to the built-in Screph Code starts below. These sections expand the short overview above and describe the currently implemented GUI flow: backend selection, launch from Screph, context transfer, Builder, edit preview, validation, and recovery.
Important: this guide applies only to the built-in Screph Code editor. In code-integration settings, select Code editor: Screph Code [Built-in]. If another backend is selected (Trae, PearAI, and so on), the project-panel buttons will open a different tool.
Screph Code works with a saved Screph project, its project.json, the workspace folder, and the guide file for the selected save strategy.
Inside Screph Code there is a dedicated window with a code editor, an Artifacts tab, a Project panel, and a Builder panel.
Generation and editing use the same LLM settings that are configured for Screph Code in the main application.
The window runs through a separate Pro Agent process and requires an available codegen.runtime and an LLM profile assigned to the codegen role. Missing dependencies are reported explicitly.
A separate process is neither a sandbox nor a universal approval gate
Pro Agent and the Screph Code window run as the current user and inherit the process environment. Builder runs in agent mode: its screph_file tools can create and modify files inside the selected workspace during the request. The current primary GUI has no mandatory per-file diff/Apply step before every such write; the Changes tab and Undo/Checkpoints controls are not a universal safety gate. Use a separate working branch or backup. Embedded Monaco loads its static assets from a temporary server on 127.0.0.1; this is a local HTTP origin, not an upload to a cloud editor.
10. How to Open Screph Code and How Autostart Works
Manual Launch
Open code-integration settings and select Code editor: Screph Code [Built-in].
If needed, configure the base Screph Code integration options: Project folder, Output folder, and Prompt template.
On the project panel, select the editor in the dropdown and use the Screph Code button.
Autostart on Project Save
The Autostart: Screph Code checkbox stores the post-save handoff flag. When enabled, Screph attempts to launch the selected code tool and send project context after target preparation succeeds. An unavailable backend or launch/send failure does not roll back the saved project.
In the current implementation, the primary save flow first creates the canonical/agent package, then validates and prepares the target, and only then performs the best-effort code-tool handoff. It uses the agent export as context when available, otherwise canonical JSON.
This rule applies to the primary save flow. Project Manager save and some legacy save entry points pass the canonical project path directly even when a for_ai_agent sidecar exists nearby. The Open editor button only launches the selected tool; without the post-save handoff it does not send project context by itself.
11. What Screph Sends to Screph Code
Launching from Screph is more than just opening the project folder. On save, Screph creates a dedicated context payload for Screph Code; the window uses it to fill the fields and optionally starts the build flow immediately.
project_json:
the path to the saved project JSON.
output_dir:
the project workspace directory for generated files.
guide_path:
the guide file for the selected save strategy, if one is found.
auto_generate is a historical field name: the current consumer places the initial message in Builder and programmatically invokes the regular Send action. The current window has no separate Ctrl+B command or isolated generate-only pipeline.
12. Screph Code Window Map and the Role of Builder
The left column contains the JSON/Output paths, manual Open/Save/Save As actions, run/stop/restart for the current file, runtime log/diagnostics/export controls and the workspace explorer.
The center contains Monaco and the Code, Plan, Changes, Tasks, Execution and Materials tabs. Plan/Tasks surface agent events, Execution reports the child-process run, and Materials stores responses and supporting results.
The right Builder column shows LLM status/model, the transcript, request input, drag-and-drop attachments, voice input and the Send button. Enter sends the request; Shift/Ctrl+Enter are left to the text widget's standard editing behavior.
The request-field context menu can insert a current-file/selection reference or attach the current/selected project file. The active document, unsaved buffer, selection and attachments are sent as structured context; large parts may be budget-truncated and reported in context chips.
13. Builder Requests, Workspace Writes and File Execution
Sending a Builder Task
Save the project or open Screph Code manually.
Check the JSON and Output paths, selected model and workspace. A post-save handoff fills them from the context payload.
Enter a concrete task, optionally attach files or a current-file/selection reference, then choose Send or press Enter.
Follow Plan/Tasks and the transcript. Stop asks the current OpenHands conversation to stop, but it does not automatically roll back workspace writes that already occurred.
After the response, refresh and inspect affected files in the explorer and use Git or another external diff tool. The agent's final text is not proof of which files changed.
What a Builder Response Means
The current Builder always creates a request with mode=agent. It may answer with text only, but it can also call write-capable workspace tools. Therefore Send is not a read-only chat action. Write tools constrain paths to the selected workspace, but there is no separate confirmation for every operation.
Manual Editing and Execution
The Monaco buffer is persisted only with Save/Save As. Run is a separate explicit action: it executes the selected script in a child process after the available preflight, while Stop/Restart control that run. This is not Automation Runtime and it does not execute code automatically after a Builder response.
14. Settings, Tabs and History
The current Builder settings dialog contains the LLM/model assignment, default project/output paths and dependency actions for Python/PyAutoGUI. The old prompt-template, validation/style and Gather/Chat/Agent controls are not part of the current surface.
Plan and Tasks project OpenHands events; they are not an approval queue. Changes shows only records explicitly added by a matching producer; it does not automatically intercept every workspace write.
Undo/Checkpoints depend on the in-memory EditHistory. The current primary Builder Send path does not register direct screph_file edits there, so these controls do not replace Git, a backup or disk-level file inspection.
Materials stores large Builder responses and supporting artifacts in local workspace state. Opening a material in Code loads text into the editor buffer; Save/Save As is still required to persist it.
15. Errors, Warnings, and Recovery
Typical pre-run problems include: missing project JSON, no output folder, an empty Builder input, or an unavailable LLM.
If the context budget trims history or retrieval, Builder shows context/prompt chips with dropped/truncated state. This means the context is incomplete; it is not a model error.
An OpenHands text response does not guarantee that the required file was created. Inspect the workspace and expected target manually.
Stop moves the request to a cancelled state but does not roll back completed tool calls. Recover an unwanted write through Git/a backup, or through screph_file undo_edit only while that tool session still retains its in-memory undo stack.
Run preflight and child-process execution have their own diagnostics/log/export. A script failure does not roll back code files and does not imply a crash of the main Screph process.
16. Practical Screph Code Tips
Create a Git commit/branch or workspace backup before sending a request, especially with post-save autostart.
State the scope explicitly: target file, allowed adjacent files, required verification and a no-run constraint when execution is unnecessary.
Use an attachment or current-file/selection reference for precise context; do not assume the agent will select the right file in a large workspace.
After Builder, inspect git diff, new/deleted files and the expected target contents, then execute the script separately with Run or in Automation Runtime.
Save the canonical project before handoff so the primary save flow can pass an up-to-date for_ai_agent projection and CV recipe references.