What BorgVR MCP provides
All three native renderers contain the same optional MCP server. It is independent of the dataset and WebGPU servers and uses the same central command layer as the macOS scripting language. An agent can therefore act reproducibly instead of operating buttons through mouse or gaze control.
- Read the dataset catalog with complete metadata and known origins, then open or close datasets
- Read and change rendering mode, isovalue, transform, clipping, and lighting
- Manage objects, sphere and stroke markers, mesh instances, and length, area, and volume measurements
- Hide objects or the complete measurement visualization without deleting data
- Wait for a completed rendered frame, retrieve the current view as a PNG image, and export isosurfaces as OBJ on macOS
Enabling MCP
- Open BorgVR Settings and select the MCP section.
- Enable Enable MCP server and choose another port if required. The default is
8765. - Add one of the displayed endpoints to the configuration of your MCP client.
http://127.0.0.1:8765/mcp. Local connections require neither TLS nor an access token.Access from the network
When Allow connections from the network is enabled, BorgVR binds the MCP server to all available network interfaces and displays an HTTPS endpoint for every reachable address. Depending on router, firewall, and port forwarding, the service may also be reachable from outside the local network.
- Generate an access token or enter your own. The client sends it as
Authorization: Bearer …. - The generation button copies the new token directly to the clipboard.
- Network connections use HTTPS. Without an imported PKCS#12 identity, BorgVR creates a temporary self-signed certificate whenever the server starts.
HTTP requests are limited to 4 MiB including at most 16 KiB of headers. Headers must arrive within 10 seconds and the complete request within 60 seconds; receive pauses longer than 10 seconds close the connection. Response transmission has a separate 30-second deadline. Command execution does not count as receive time. Network requests are authenticated as soon as their headers arrive, before the complete body is loaded.
Typical agent workflow
A complete catalog query is limited to 60 seconds, 2000 datasets, and 16 MiB of serialized catalog data. Dataset-server text replies are limited to 4 MiB and lists to 10000 entries; MCP metadata replies to 32 MiB per dataset. Timeouts and task cancellation close active catalog connections. Exceeded limits are reported as errors, not as a successful, silently truncated catalog.
- The agent uses
list_datasetsto inspect metadata and origins, then opens the selected dataset withopen_dataset. - It inspects
get_application_state, changes the view and rendering parameters, and hides objects or measurements when required. - After each change it uses
wait_until_renderedto wait for a new completed frame, then callscapture_screenshot. - On macOS it can use
export_isosurfaceto save the current dataset as an OBJ file at the current or an explicitly supplied normalized isovalue. - By comparing multiple views, rendering modes, isovalues, or clipping ranges, the agent can inspect visible structures systematically.
The JSON schemas returned by tools/list are the authoritative reference for argument names and value ranges.
Transfer functions and picture configurations
get_transfer_function and set_transfer_function read and set a zero-based channel's TF. Each entry is an array of four RGBA bytes from 0 through 255. Compact color ramps are linearly interpolated to the current table size. Other channels remain unchanged; RGB/RGBA volumes have no TFs. load_transfer_function and save_transfer_function use native .tf1d files.
get_background and set_background offer system, solid, and gradient modes on macOS and iOS. primary is the solid/top color and secondary the bottom color; both are RGBA arrays from 0 through 1. visionOS retains passthrough. For capture_screenshot, include_alpha:true (default) preserves the transparent render image. false composites the background and produces a PNG without alpha; visionOS uses black instead of passthrough. output_path optionally also saves the PNG to a file.
save_visualization_state and load_visualization_state store all channel TFs, transform, render mode/isovalue, clipping, lighting, background, and screen projection in a freely named JSON file. The same dataset must be open. Like the UI, visionOS stores positioning relative to the head rather than stale room coordinates. Objects and measurements are not part of this picture configuration.
{"name":"set_transfer_function","arguments":{"channel":0,"samples":[[0,0,0,0],[255,80,0,100],[255,255,255,255]]}}
{"name":"set_background","arguments":{"mode":"solid","primary":[0,0,0,1]}}
{"name":"save_visualization_state","arguments":{"output_path":"/path/on/app-device/picture.json"}}
{"name":"capture_screenshot","arguments":{"include_alpha":false,"output_path":"/path/on/app-device/picture.png"}}
Back up and restore settings
save_settings, reset_settings, and load_settings let agents back up registered settings, work with defaults, then restore the previous configuration. The .borgvrsettings JSON format is compatible with the UI; missing entries remain unchanged and unknown entries are ignored. Credentials are exported only with include_credentials:true. Reset requires confirm:true. Import and reset always preserve the running MCP connection's settings (enable, port, token, and TLS), so the agent stays connected.
All file paths are absolute paths on the app device and must be accessible to its sandbox. Existing output files are replaced only with overwrite:true. After an app update, the MCP client must refresh its tool list.
{"name":"save_settings","arguments":{"output_path":"/path/on/app-device/backup.borgvrsettings","include_credentials":true}}
{"name":"reset_settings","arguments":{"confirm":true}}
{"name":"load_settings","arguments":{"input_path":"/path/on/app-device/backup.borgvrsettings"}}
Brick diagnostics and optimizations
set_bounding_box uses a boolean visible to show or hide the complete dataset frame. This local setting persists and is recorded as GSC; renderingOptions.datasetBoundingBoxVisible reports its current value.
set_brick_visualization accepts optional boolean fields enabled, resident, empty, and missing. Type filters are local and also apply to subblocks: red = resident, green = skipped as empty, blue = missing or coarser substitute. Hiding a type changes neither traversal nor brick requests.
set_render_optimizations accepts boolean subbrick_skipping and opacity_adaptive_sampling, plus alpha_epsilon from 0 through 1. These settings persist locally. Alpha epsilon affects only TF empty-space classification, not the transfer function itself or iso rendering.
For both tools, omitted fields remain unchanged; at least one field is required. get_application_state reports all current values in renderingOptions. While GSC recording is active, UI and MCP changes are recorded, including the initial values of these options.
{"name":"set_brick_visualization","arguments":{"enabled":true,"resident":false,"empty":true,"missing":true}}
{"name":"set_render_optimizations","arguments":{"subbrick_skipping":true,"opacity_adaptive_sampling":true,"alpha_epsilon":0}}
macOS window control
get_window_state returns content/frame dimensions in logical points and backing pixels, display scale, and fullscreen/transition status. resize_window accepts width and height (1 through 16384), optional units: points|pixels and area: content|frame. Defaults are points/content; frame includes window decoration. Window size limits still apply, and neither backing dimension may exceed 16384 pixels. Only the main window changes, not the active panel or script log.
set_fullscreen explicitly sets native fullscreen with enabled:true or false. An already matching state is unchanged. Transitions wait for AppKit's completion notification; resize exits fullscreen first when necessary. Conflicting simultaneous transitions or open sheets blocking fullscreen report an error. On iOS and visionOS these three tools report that window control is unsupported. Camera projection stays unchanged.
{"name":"get_window_state","arguments":{}}
{"name":"resize_window","arguments":{"width":2880,"height":1800,"units":"pixels","area":"frame"}}
{"name":"set_fullscreen","arguments":{"enabled":true}}
Screen projection
set_screen_projection sets the local camera on macOS and iOS. Optional arguments are field_of_view_degrees (10 through 120 degrees) and parallel (boolean); at least one is required. Omitted values stay unchanged. The field of view also controls equivalent framing in parallel projection. Settings persist per device and are not synchronized through SharePlay. Changes are recorded while GSC recording is active.
get_application_state.renderingOptions includes screenFieldOfViewDegrees and parallelProjection on macOS and iOS. On visionOS these fields are absent, and the setter reports that the operation is unsupported.
{"name":"set_screen_projection","arguments":{"field_of_view_degrees":45,"parallel":false}}
Volume editing and export
configure_volume_brush also accepts channels: "all" or an array of one-based channel numbers, e.g. [1,3]. At least one valid channel is required. get_volume_edit_state reports componentCount and channels as a list of selected, one-based channel numbers. Opening selects all channels. Selection is local; SharePlay carries it with each operation. discard_volume_edits always resets every channel.
Dataset and isosurface exports are disabled during SharePlay. Edits and edit-state changes are rejected while an export is running. Starting SharePlay cancels a running export.
The following tools are available on macOS, iOS, and visionOS. Edits affect the active edit state and are shared during SharePlay. The original file is never modified.
get_volume_edit_statereturns availability, dataset ID, brush settings, edit status, pending brick count, and the latest export job with UUID, status, progress, and any error. The same state is included inget_application_state.volumeEditing.get_volume_edit_stateincludessavedStates,activeStateID, and SharePlay status.select_edit_stateselects a UUID or original;create_edit_statecreates an empty state or a copy with duplicate=true.rename_edit_statechanges its name.delete_edit_stateremoves an inactive state with confirm=true. State management is locked during SharePlay.configure_volume_brushaccepts optionalmode(erase/restore),radius_fraction(0.002–0.15 of the largest extent),project_onto_volume, andactivate. The latter activates hand/mouse/touch editing and defaults to false; Muse/PSVR modes stay independent.edit_volumerequiresmodeandsamples, an array of objects withcenter_meters: [x,y,z]and positiveradius_meters. Coordinates are physical meters from the volume minimum corner, not screen, hand, or world coordinates. Interactive radius and projection do not affect these explicit samples; the channel selection from configure_volume_brush applies here too. At most 65536 samples per call within the HTTP request limit; no interpolation. Erase sets selected channels to 0; restore recovers only their original values.edit_volumereturns after journaling. Usewait_volume_editswith optionaltimeout_seconds(default 30, maximum 86400) to wait for background jobs. Unreachable remote bricks may cause a timeout.prepare_volume_editsrebuilds pending edited bricks with foreground priority and waits for persistence. Uses Metal when available; cancellation preserves recorded edits.discard_volume_editsrequires explicitconfirm: trueand clears only the active edit state. During SharePlay, it clears the shared state; other states are preserved.export_edited_datasetstarts an export job with absoluteoutput_path(.data), plus optionalnameanddescription. Paths refer to the app device and must be accessible to its sandbox. Existing destinations are refused. The consistent export includes all previously recorded edits and preserves source compression; waiting for brick jobs first is unnecessary.- Poll
get_volume_edit_stateuntilexport.statusis completed, cancelled, or failed; progress ranges from 0 to 1.cancel_volume_exportwith the job UUID inidcancels only that job. Incomplete export files are removed; recorded edits remain.
{"name":"edit_volume","arguments":{"mode":"erase","samples":[{"center_meters":[0.5,0.5,0.5],"radius_meters":0.05}]}}
{"name":"export_edited_dataset","arguments":{"output_path":"/path/on/app-device/edited.data","name":"Edited volume"}}
Troubleshooting
- Empty response or no connection
- Verify that scheme and endpoint match: local access uses
http, network access useshttps. - Unauthorized
- For network access, the Authorization header must contain exactly the currently displayed bearer token. Local access on the same device does not check a token.
- Certificate error
- Explicitly trust the self-signed certificate or import your own PKCS#12 identity. Some externally hosted agents cannot reach private or local IP addresses at all.
