Freestyle VM Evaluation: State Continuity and Execution Limits

Effloow independently executed Freestyle VM tests on September 16–19, 2026 (UTC; September 17–20 in Japan). This note summarizes the retained measurements and unsuccessful attempts. E2B cloud execution was not performed, so this is not a provider performance comparison.

Support disclosure: Freestyle provided test credits and Hobby access. Coverage and conclusions were not conditional on that support. Account identifiers, access credentials, private correspondence, resource IDs, and raw transport logs are excluded from this public note. This summary is not the complete raw dataset or a standalone executable harness.

Related article: E2B sandbox guide. E2B's separate evidence is SDK inspection only.

Environment and timing

Client: macOS arm64, Node.js 26.8.1, Freestyle SDK pinned to 0.2.13.
Guest: pinned Ubuntu 24.04.4 x86_64 snapshot; Linux 6.1.102, Python 3.12.3.
VM allocation: 2 vCPU, 4 GiB RAM, 16 GiB disk; no explicit region selection.
Docker: already present, version 29.1.3. The preparation path used explicit root execution.
Times include SDK processing and client network round trips. New-VM first response runs from before creation until a command response, rather than stopping at creation acknowledgement.
Median and observed min/max use validated samples. T1 p95 uses nearest rank. Other sample sizes do not support a p99 or SLA claim.
Three T1 warmups are excluded. Its 30 samples were collected in three blocks about five minutes apart in one session. Creating from a fixed snapshot does not demonstrate an empty provider cache.

Recorded protocol

The following were invocations in the private, pinned evaluation harness, not commands available in this site's repository. They document execution provenance and must not be read as a request to repeat completed tests.

npm ci --ignore-scripts
npm test
npm run preflight
npm run phase1
npm run snapshots
npm run network
npm run docker
npm run long-pause
npm run resume-due
npm run long-job
npm run report

resume-due collected only samples whose actual 48/72-hour due times had arrived. long-job collected or started samples up to a total of three. Resource bounds were two running VMs and six long-paused VMs. New resources stopped at $16 of conservative public-rate reservations within a $20 ceiling. No payment card or paid upgrade was used.

To reproduce the acceptance criteria in another harness:

Pin an image and SDK, record client/guest versions and VM allocation, and separate creation acknowledgement from the first usable command response.
Generate a deterministic 100,000-row CSV workload and compare both CSV and result SHA-256 with a local control. Include VM creation in total job time.
Start a service holding a nonce, PID, boot ID, original start time, open-file offset, memory SHA-256, disk SHA-256, and advancing counter. Record its state; pause for the full elapsed interval; start the VM and query the service. Compare the identity/state fields and require the counter to progress. Local controls exercised restart and mutation so a matching file alone could not satisfy the criterion.
For snapshots, allocate 512 MiB memory plus 10 or 256 MiB disk, obtain a completed snapshot, create two VMs from it, and verify continuity. Mutate clone A; ensure source and clone B retain their original checked state.
Exercise both an allowed and denied port/domain path before interpreting a network policy. A failed positive control makes isolation inconclusive.
Use a digest-pinned container, measure preparation separately from container-to-HTTP response, and record root/DNS prerequisites.
For a 45-minute PTY job, disconnect the WebSocket for at least 60 seconds, reconnect, then check duration, completion file, accumulated hash, and exit status separately. Missing exit status must not be silently substituted with artifact success.
Delete resources in cleanup paths, retry transient deletion conflicts, and verify the final inventory. Preserve failed and aborted attempts separately from corrected runs.

These steps describe the protocol; reproducing identical numeric results requires the original workload and network conditions.

Results

Test / measured endpoint	Validated / collected samples	Median	Observed range
T1 creation to first command response	30/30	376.2 ms	313.9–551.7 ms
T1 creation API acknowledgement	30/30	237.9 ms	158.9–410.4 ms
T2 CSV generation through result verification, including creation	10/10	973.3 ms	900.1–1116.7 ms
T3 resume after actual 60-second pause, 1 MiB memory	10/10	1243.2 ms	1217.0–1457.2 ms
T3 resume after actual 60-second pause, 512 MiB memory	10/10	1605.3 ms	1214.5–2113.9 ms
T4 completed snapshot, 512 MiB memory / 10 MiB disk	5/5	7719.2 ms	7245.7–26346.3 ms
T4 completed snapshot, 512 MiB memory / 256 MiB disk	5/5	11484.9 ms	9438.0–12116.3 ms
T5 resume after at least 48 hours paused	3/3	2272.3 ms	2159.5–2317.3 ms
T5 resume after at least 72 hours paused	3/3	2118.2 ms	2079.2–2290.0 ms
T6 same-VPC port rules and TLS	3/3 functional trials	—	—
T6 domain allow/deny, grant/revoke	3/3	—	—
T6 cross-VPC isolation	INCONCLUSIVE	—	Allowed positive control did not connect
T7 Docker daemon/image preparation; Docker preinstalled	5/5	3432.9 ms	3065.2–3527.2 ms
T7 prepared container to HTTP response	5/5	1474.7 ms	1394.4–1555.9 ms
T8 45-minute work artifacts/hash after WebSocket detachment	3/3	—	2700.11 / 2700.91 / 2700.27 seconds of work
T8 original criterion including clean PTY exit code	0/3; original FAIL retained	—	Exit status not successfully observed

T1 first-response p95 was 513.3 ms. T5 actual paused durations were 48.4739–48.4743 hours and 72.2142–72.2146 hours. T3/T5 restoration timings include state-service hash checks and round trips.

T4's SDK returned after snapshot materialization; we did not observe a separate acceptance acknowledgement or independently time a fork API. Memory used repeated 0xA5 bytes and disk repeated 0x5A bytes: real allocated state, but compressible synthetic data. The 10 MiB condition included a 26.35-second maximum despite a 7.72-second median.

T7 used the official Python 3.12 Alpine linux/amd64 image pinned as:

python@sha256:78e98729f8fc4099e53cffb3fe59fd15b18dfa4ace8c914dee0cefa5320068eb

Preparation timed daemon startup and image pull, not installation of Docker on a clean guest. HTTP readiness polling was included in container response time.

Failure ledger and observation limits

Attempt / issue	Finding and disposition
T0 initial snapshot lookup	SDK rejected a qualified public-image alias before a request. Corrected by passing the alias to VM creation and pinning its returned immutable snapshot ID. Initial FAIL retained; not a measured provider outage.
T4 first 256 MiB attempt	A fixed one-second wait did not establish service readiness. Switched to 250 ms readiness polling and reran that condition separately. Initial FAIL retained.
T4 accidental repeat	An additional full run was mistakenly started and then aborted. ABORTED record retained and excluded from validated timing samples.
T6 final cleanup	Same-VPC functional trials passed, but VPC deletion returned HTTP 409 after VM deletion while addresses remained reserved. Cleanup retried successfully; the run's original FAIL remains.
T6 cross-VPC positive control	Allowed traffic did not connect. Routing and firewall effects could not be separated, so failed deny-path connections do not prove cross-VPC isolation.
T6 domain/TLS controls	End-to-end allow/deny behavior was checked. DNS versus TLS rejection layers were not isolated; no bypass-resistance or security-audit claim follows.
T7 initial permissions	Default guest user could not start `docker.service`. Corrected preparation used explicit root; original FAIL retained.
T7 initial networking	HTTP/HTTPS-only rules omitted DNS. Adding port 53 UDP/TCP restored registry resolution and its expected HTTP 401 authentication response; corrected Docker trials included DNS. Original FAIL retained.
T8 exit-code observation	All three jobs completed the duration/hash/artifact checks, but none satisfied the original clean-exit observation assertion. The third sample's saved PTY list was empty at collection; the first two have no saved list, so their cause cannot be reconstructed. This does not establish job failure or abnormal exit. No fourth sample was run to replace the failures.

The result table summarizes validated operational samples, not a provider-wide failure rate. Preparation errors and aborted attempts remain in this ledger. These tests do not establish tenant escape resistance, regional consistency, random-data snapshot performance, an availability SLA, or p99 latency. Nested virtualization, FUSE, eBPF, WireGuard, framework/MCP integration, and continuous multi-day runtime were not tested. Documented features are not measured acceptance results.

Cost and cleanup

The conservative CPU/RAM/storage/transfer reservation total was $5.4328. Reservations used TTLs, maximum running windows, and transfer allowances; this is not actual consumption, an invoice, or a verified credit deduction. The calculation used public unit rates of $0.04032/vCPU-hour, $0.0129/GiB-hour RAM, $0.000086/GiB-hour storage, and $0.02/GB transfer. Actual billing remains unverified.

Inventory checked September 30, 2026 at 00:57:52 UTC: 0 VMs, 0 snapshots, 0 VPCs, 0 TLS rules, and 0 firewall rules. This verifies cleanup at that timestamp; it does not imply anything about later unrelated account activity.

Documentation versus measurement

Freestyle VM overview: vendor documentation for VM operations and memory-preserving pause.
Freestyle migration guide: vendor architecture and lifecycle claims. Its tail-latency and superiority claims were not validated by our single-provider test.
Freestyle pricing and limits: public pricing rechecked October 10, 2026; a plan/allowance description, not evidence of our invoice.
E2B pricing: separately checked plan information; no E2B runtime result exists in this evaluation.