perf(sandbox): push objects a store can't sign onto a VM in parallel, and over stdin on Modal VMs

[earakely-scale]

Summary

With a store that can't sign URLs (e.g. the local store), objects reached a VM sandbox one base64 chunk per exec, in sequence: about 0.1 MiB/s, so a 161 MB snapshot restore took 19 minutes.

• push_object_over_exec is now the default for every VM sandbox. It streams the object a chunk per exec, each written at its offset with dd, VmSandbox._PUSHES_IN_FLIGHT (8) at once, and checks sha256 at the end.
• push_object_over_stdin is for sandboxes whose exec takes stdin; ModalVmSandbox now uses it. Segments of at least 1 MiB are read from one open file by position, and each is sha256-checked by the exec that writes it. Other readers go as one stream.
• An object that fits in one exec goes in one exec, on both paths.
• Signing stores (S3) and the local provider are unchanged.

Measurements

Dev VMs, local store, every run sha256-verified.

	modal_vm	other remote VM provider
before	0.10 MiB/s · 0.22 s per 10 KiB file	0.12 MiB/s · 0.49 s
this PR	8.3 MiB/s (stdin) · 0.23 s	0.6 MiB/s (default path) · 0.45 s
once it implements _exec_with_stdin		12.3 MiB/s · 0.45 s

A 110 MiB image loads and runs in 15.8 s on modal_vm, and 15.1 s on the other provider over stdin.

Chaos (live VMs)

• Killing the writers mid-push: fails in 1.3–2 s.
• Corrupting the file mid-push: caught by sha256.
• Terminating the VM mid-push: fails the push. On Modal, execs already running finish and verify instead.
• For stdin implementations: race the feed against the process's end. On the other provider, drain() never returns once the VM dies.

E2B is untested (no credentials) and gets the default path.

Testing

make unit-test passes. vm_object_push_test.py runs both paths through a real shell, covering:

• sizes and single-exec pushes;
• segments;
• readers that aren't plain files (e.g. gzip);
• an object replaced or truncated mid-push;
• garbled data, a silent exec, and a failed chunk.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge, though a failed Modal push can still leave its VM writer waiting for input.

Findings

1.  Failed push leaves writer waiting ▶

Fix with agent prompt

### Issue 1
src/agent_env/providers/sandbox_providers/sandbox.py:undefined-642
When a local object is cut short during a Modal push, `pieces()` raises while `_exec_with_stdin` is feeding a running writer. The error skips `write_eof()` and `wait()`, leaving that VM process waiting for input after the push has failed. Close the input and clean up the process when feeding fails.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Unsigned-object transfers to VM sandboxes now use parallel exec chunks by default, while Modal VMs stream the data over stdin. Both paths check the bytes written, cutting the time spent transferring large objects.

• Unsigned objects reach VM hosts through parallel exec chunks.
• Modal VMs stream unsigned objects through exec stdin.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[VM needs an object] --> B{Store can sign a URL?}
  B -- Yes --> C[Download from signed URL]
  B -- No --> D{Modal VM?}
  D -- Yes --> E[Send segments over stdin]
  D -- No --> F[Send chunks through exec]
  E --> G[Check written bytes]
  F --> G

Loading

_{Reviews (4) · Last reviewed commit: "docs(sandbox): an object a store can't p..." · Reviewed by Greptile}

[greptile-apps]

Failed push leaves writer waiting

When a local object is cut short during a Modal push, pieces() raises while _exec_with_stdin is feeding a running writer. The error skips write_eof() and wait(), leaving that VM process waiting for input after the push has failed. Close the input and clean up the process when feeding fails.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agent_env/providers/sandbox_providers/sandbox.py
Line: 595

Comment:
**Failed push leaves writer waiting**

When a local object is cut short during a Modal push, `pieces()` raises while `_exec_with_stdin` is feeding a running writer. The error skips `write_eof()` and `wait()`, leaving that VM process waiting for input after the push has failed. Close the input and clean up the process when feeding fails.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.