
LLM says something, my tool track if it followed its promise on tools
RayTrace records the prompts, context and tool calls of a coding agent’s run and, in a sandbox, the processes that actually started. Then change one thing it saw and watch what it does differently.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
RayTrace records the prompts, context and tool calls of a coding agent’s run and, in a sandbox, the processes that actually started. Then change one thing it saw and watch what it does differently.
A coding agent in a RayTrace sandbox read “use tabs” from a notes file, set "indent": "tabs" in its config, and its check passed. Forked at that step with the read edited to “use spaces”, it set "indent": "spaces" and the same check failed. An unchanged rerun matched the original.
A real run in a RayTrace cloud sandbox and its Playground fork, replayed. Timing is compressed.Watch full screen →
Eight sections connect your prompt to the project, the actions, the evidence, and the next experiment. Explore what each one does.
Find a recorded request, read the answer, and follow the run into its tool calls.
Find your project’s cloud environment, its state, connection command, and recorded history.
Understand important files, choose what to watch or restrict, and review alerts and network activity.
Follow each read, edit, command, and reported result in order, with sandbox verification alongside it.
Inspect processes, child commands, outcomes, and timing recorded independently by the sandbox.
Explore prior requests, earlier answers, file contents, and instructions present in the recorded context.
Explore possible influences on a tool call, linked to recorded requests, earlier results, and project instructions.
Change an input, continue in a separate sandbox, and compare the outcome with the original and an unchanged baseline.
What did it see? / What did it try? / What actually ran? / What would change?
A final answer only tells part of the story. RayTrace connects prompts, context, tool calls, and runtime evidence so you can inspect the steps that led there.
In one recorded Codex prompt in a RayTrace sandbox, the agent proposed 22 commands. The sandbox observed 16 of them as 49 processes, 3 were rejected before any process started, and 3 were not seen. One npm test call started seven processes.
One prompt from a real Codex run in a RayTrace sandbox, replayed. Commands, process IDs, exit codes and counts are as recorded.Watch full screen →
Scan your project for files that are risky to change. Restrict critical files, get alerts on watched changes, and inspect the sandbox’s network activity.
Initial scan ranks files by imports and role. Restrict refuses Claude Code edits immediately; stopping and starting the sandbox makes the file read-only for every tool. Report watches changes, with dashboard alerts and optional batched emails. Activity lists network destinations and connection outcomes.
A feature walkthrough based on scan-demo-shop. The blocked edit is illustrated; scan scores and network examples come from the dashboard.Watch full screen →
Was a piece of context useful? Select a step, edit or remove what the agent saw, and compare the continuation with the original run.
The replay at the top of this page is one of these experiments: a recorded cloud sandbox run, forked in the hosted Playground with one tool result edited. Watch it full screen →
The selected-step lab compares original and edited context across repeated model responses. Repeat rates describe observed behavior; they do not recover private reasoning.
For eligible sandboxed Claude Code sessions, Playground restores the project before a step and runs a fork. Compare file changes, steps, and an optional check command.
Models can vary even when nothing changes. Playground can interleave edited runs with unchanged runs, so you can examine that variation alongside your experiment.
KNOW THE BOUNDARY Forks need a recorded sandbox session with snapshots. Decision replays and Codex continuations have different setup and provider requirements.
The CLI uploads your project to an isolated sandbox on RayTrace’s servers. Your agent runs there, and its sessions and the commands that execute are recorded for your organization.
You open the sandbox’s terminal from your machine with raytrace connect and sign in to your agent inside it.
Cloud sandbox sessions are recorded on RayTrace’s servers and shared with your organization in the hosted dashboard. Other organizations cannot see them.
Run your agent in an isolated cloud sandbox and record the commands that execute alongside its sessions. Create a cloud sandbox → · How RayTrace isolates sandboxes →
A process starting or exiting successfully does not prove the task was done correctly. Missing evidence means unknown. RayTrace does not reconstruct private chain of thought.
04 / GET STARTED · CLOUD SANDBOXES · PREVIEW
Run your coding agent on RayTrace’s private hosted cloud platform with nothing to set up on your laptop. Upload your Git project, open a terminal, and run Claude Code or Codex on RayTrace’s servers.
Review your organization’s prompts, sandboxes, and tool calls at app.raytracer.si ↗. Follow the agent’s requests and reported results alongside captured command execution evidence. See which sandbox and host the work belongs to.
Cloud runs upload project files and record sessions on RayTrace’s servers. Read the upload and lifecycle guide →
npm install -g @raytrace-cli/cli@latest raytrace auth login raytrace sandbox create raytrace connect
Run the sandbox commands inside a Git repository.
create prints the sandbox ID to pass to connect.
Free in beta with $10 of model usage, Individual at $29/month, and custom Team & Enterprise plans. Hosted traces, sandbox execution evidence, and Playground forks for eligible recorded sessions.
Includes $20 of model usage each month and up to 5 sandboxes at a time. Beta is free: $10 of model usage, up to 3 sandboxes.
Cloud preview · Team & Enterprise plans by query
Join the Design Partner beta. Run up to three sandboxes at once with Claude Code already installed, and help shape RayTrace with your feedback.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。