Kubernetes Kata & CSI Block Integration for Official E2B Volumes
Source:
docs/enhancements/0026-k8s-kata-e2b-volumes/README.md· Status:implemented· Area:storage· Milestone:v0.5| Stage | Status | Created | Updated | Owning Area | | :--- | :--- | :--- | :--- | :--- | |alpha|implemented|2026-11-15|2026-11-15|storage(controller,gateway,runtime) |
Table of Contents
Section titled “Table of Contents”- Release Signoff Checklist
- Summary
- Motivation
- Proposal
- Design Details
- Production Readiness Review Questionnaire (PRR)
- Implementation History
- Drawbacks
- Alternatives
Release Signoff Checklist
Section titled “Release Signoff Checklist”- Enhancement design documented in
docs/enhancements/0026-k8s-kata-e2b-volumes/ - Shared RWX volume store backend integrated between
cubepod-gatewayand Kata Pods -
podd-initsingle-threadedpivot_rootE2B mount propagation withMS_NOSUID | MS_NODEVimplemented -
PodBoxSpec.E2BVolumeMountscontroller staging mount +/etc/cubepod/e2b-mounts.json(or env spec) implemented -
ForkFrominteraction withE2BVolumeMountsenforced and unit-tested - Live Kubernetes Kata + CSI E2E tests (
make test-k8s-e2e) passing - Security invariants verified (AGENTS.md Invariants 1, 2, 3, and 4)
Summary
Section titled “Summary”CEP-0026 lifts the 501 Not Implemented Kubernetes gate introduced in CEP-0025 so that official E2B 2.64.0 volumeMounts work end-to-end on production Kubernetes (Kata + CSI) clusters while preserving Local Development (Docker + Loopback ext4) parity.
The design solves three blockers that prevented Kubernetes support in CEP-0025: (1) eliminating split-brain between cubepod-gateway’s local data/ directory and Pod PVCs by backing /volumecontent and Kata Pod mounts with a single shared ReadWriteMany (RWX) PVC (cubepod-e2b-volumes using subPath: <volumeID>/data), (2) staging controller volume mounts at /run/cubepod/volumes/<volumeID> and propagating them into the merged OverlayFS root inside single-threaded podd-init prior to pivot_root(2) with MS_NOSUID | MS_NODEV, and (3) defining explicit attach/conflict semantics when ForkFrom is combined with E2BVolumeMounts.
Motivation
Section titled “Motivation”In CEP-0025, CubePod implemented the official E2B 2.64.0 /volumes, /volumecontent/{volumeID}/..., and POST /sandboxes volumeMounts APIs for the local Docker runtime. However, pkg/gateway/volumes.go (isKubeVolumeMountsUnsupported) explicitly returns 501 Not Implemented whenever livePodRouting && runtime == nil (the live Kubernetes control plane), and ensureE2BVolumePVC skips PVC creation.
Without Kubernetes Kata integration, E2B SDK workflows that pre-populate datasets via Volume.write_file and attach them to sandboxes via Sandbox.create(volume_mounts=...) cannot run in production CubePlex Kubernetes clusters.
- Single Source of Truth: Ensure
cubepod-gateway/volumecontentreads/writes and Kata PodvolumeMountsaccess the exact same underlying volume bytes without split-brain or stale copies. - Preserve
pivot_root& OverlayFS Isolation: Propagate dynamicE2BVolumeMountsinto the post-pivot_rootsandbox root insidepodd-initwithout breakingpodd-init’s single-threaded PID 1 invariant (AGENTS.md Invariant 2) or coupling/workspaceand/rootfs-upper(AGENTS.md Invariant 1). - Security & Ownership Parity: Enforce
nosuid,nodevat the VFS mount boundary inside the guest and preserve1000:1000(cubeplex) ownership (AGENTS.md Invariant 4). - Multi-Sandbox & Fork Safety: Support concurrent read/write attachment across sandboxes (and gateway HA replicas) and define deterministic
ForkFrombehavior. - Dual-Runtime Consistency: Keep local Docker (
HostConfig.Binds/bind,nosuid,nodevnamed volumes) working identically with zero API divergence.
Non-Goals
Section titled “Non-Goals”- Per-file POSIX ACL translation inside guest: Requested
uid/gidfromPUT /volumecontentremain stored in gatewayattrs.jsonmetadata only; on-disk files remain1000:1000(same as CEP-0025). - CoW block cloning of E2B volumes on
ForkFrom: E2B volumes are independent, reusable datasets—not per-sandbox RootFS/Workspace block volumes. Forking a sandbox does not CoW-clone an attached E2B volume. - Raw Block (
volumeMode: Block) CSI for E2B volumes: E2B/volumecontentis a file/directory HTTP API served bycubepod-gateway; raw block devices cannot be concurrently mounted read-write by both the gateway and multiple Kata Pods without a cluster filesystem.
Proposal
Section titled “Proposal”User Stories
Section titled “User Stories”- As an E2B SDK user on CubePlex (Kubernetes + Kata), I can create a volume with
Volume.create(), upload training data or tools viaVolume.write_file(), launch one or more Kata sandboxes withvolumeMounts: [{name, path}], and immediately read/write those files inside the sandbox atpathwhile/workspaceremains isolated. - As a CubePlex operator running multi-replica
cubepod-gateway(CEP-0022), any gateway replica can serve/volumecontent/{volumeID}/...and attach volumes toPodBoxCRs without restarting gateway Pods when new volumes are created. - As a security reviewer, untrusted sandbox workloads cannot mount an E2B volume over
/workspace,/rootfs-upper,/opt/cubepod,/proc,/etc, or plant setuid-root binaries that escalate privileges inside the Kata guest.
Notes / Constraints / Caveats
Section titled “Notes / Constraints / Caveats”- Dual-Runtime Behavior:
- Production Kubernetes (
Kata + CSI):cubepod-gatewaymounts a sharedReadWriteMany(RWX) PVC (cubepod-e2b-volumes) atCUBEPOD_E2B_VOLUME_DIR.cubepod-controllermounts the same RWX PVC into each Kata Pod usingsubPath: <volumeID>/dataat/run/cubepod/volumes/<volumeID>and passes mount specs topodd-initvia/etc/cubepod/e2b-mounts.json(orCUBEPOD_E2B_MOUNTS).podd-initvalidates and bind-mounts each staging path into<merged_root>/<targetPath>withMS_NOSUID | MS_NODEVbeforepivot_root(2). - Local Development (
Docker + Loopback ext4): Unchanged from CEP-0025 (pkg/volumewrites to<stateDir>/e2b-volumes/<id>/data, mounted directly into Docker atm.Pathwithbind,nosuid,nodev). When running KubernetesvolumeMode: Filesystemwithoutpivot_root(CUBEPOD_SKIP_PIVOT=1),podd-initskipspivot_rootand bind-mounts/run/cubepod/volumes/<volumeID>directly ontom.PathwithMS_NOSUID | MS_NODEV(orapplyOfficialVolumeMountsmounts atm.PathwhenCUBEPOD_SKIP_PIVOT=1).
- Production Kubernetes (
- Kata VirtioFS /
subPathConstraint: Kata Containers (kata-qemu) shares containerVolumeMountsinto the guest VM viavirtio-fs(virtiofsd). Mounting a single RWX PVC withsubPath: <volumeID>/dataexposes only that volume’sdata/directory into the guest at/run/cubepod/volumes/<volumeID>, preventing guest access to sibling tenants’ directories or gatewaymeta.json/attrs.json. - Single-Threaded
podd-initConstraint:podd-initMUST remain strictly single-threaded prior topivot_root(2)andexecvp(3)(AGENTS.md Invariant 2). All JSON parsing andmount(2)syscalls run synchronously on the main thread.
Risks and Mitigations
Section titled “Risks and Mitigations”| Risk | Impact | Mitigation |
|---|---|---|
Guest escapes subPath via symlinks created by sandbox code |
Cross-tenant read/write on shared RWX PVC | Gateway pkg/volume continues using openat2(RESOLVE_BENEATH|RESOLVE_NO_SYMLINKS) / O_NOFOLLOW dirfd walks; virtiofsd + Kata VM boundary confines guest to its subPath mount; meta.json and attrs.json reside outside <id>/data. |
E2BVolumeMounts target path overlaps /workspace, /rootfs-upper, or system dirs |
Violates Invariant 1 or breaks container boot | Validated at 3 layers: gateway (pkg/volume), controller (ValidateOfficialVolumeMounts), and podd-init (canonical lexical check before mount(2)). |
Setuid binary or device node placed on RWX volume via /volumecontent or another sandbox |
Privilege escalation inside Kata guest | Gateway masks disk mode to 0o777 and 1000:1000; podd-init remounts every E2B bind mount with MS_BIND | MS_REMOUNT | MS_NOSUID | MS_NODEV. |
Multi-replica cubepod-gateway concurrent WriteFile / Delete on RWX |
Metadata race (meta.json, mounts.json) |
Gateway Store uses flock/O_EXCL lockfile or atomic temp-file + rename on RWX POSIX filesystem; Delete checks active PodBox references in K8s API in addition to local mounts.json. |
| RWX StorageClass unavailable in cluster | K8sE2BVolumeMounts cannot provision shared PVC |
Feature gate K8sE2BVolumeMounts defaults to false in Alpha until operator configures e2bVolumes.rwxStorageClassName; when unset, gateway returns 501 Not Implemented. |
Design Details
Section titled “Design Details”Problem Analysis
Section titled “Problem Analysis”In CEP-0025 (#67), pkg/gateway/volumes.go gated off Kubernetes volumeMounts (501 Not Implemented) for three concrete technical reasons:
-
Gateway-Local vs. Pod PVC Split-Brain:
pkg/volume.Storewrites uploaded files toCUBEPOD_E2B_VOLUME_DIR/<id>/dataon thecubepod-gatewayPod’s local filesystem. Meanwhile,ensureE2BVolumePVCinpkg/gateway/volumes.go(lines 567–600) sketched creating an emptyReadWriteOncePVCe2bvol-<id>that was never attached to the gateway Pod. Consequently, files uploaded viaPUT /volumecontent/{id}/filewould never appear insidee2bvol-<id>when mounted by a Kata Pod. -
podd-initpivot_root(2)Drops Arbitrary Container Mounts: Incrates/podd-init/src/main.rs(lines 280–406),podd-initmounts the RootFS Upper block device, assembles an OverlayFS atcfg.merged_root_path, mounts/dev/cubepod-workspaceat<merged>/workspace, and explicitly bind-mounts only["proc", "sys", "dev", "run", "opt/cubepod", "etc/cubepod"](plusetc/resolv.conf,etc/hosts,etc/hostname, and the composed CA bundle) before callingpivot_root(&cfg.merged_root_path, &old_root)and lazily unmounting/.old_root. IfapplyOfficialVolumeMountsinpkg/controller/manifests.go(lines 294–314) mountse2bvol-<id>directly atm.Path(e.g.,/mnt/data), that mount lives on the pre-pivot container rootfs and is detached and discarded inside/.old_rootas soon aspivot_rootcompletes. -
RWO Multi-Attach and
ForkFromConflicts: Standard CSI block/filesystemReadWriteOnce(RWO) PVCs can only be attached to a single node at a time. Attaching an E2B volume to two sandboxes scheduled on different nodes—or forking aPodBoxviaForkFrom—would stall Pod scheduling indefinitely inContainerCreating(Multi-Attach error for volume).
Architecture & Storage Options
Section titled “Architecture & Storage Options”We evaluated three storage architectures to bridge cubepod-gateway (/volumecontent) and Kata Pods (volumeMounts):
| Option | Mechanism | Multi-Node / Fork | Gateway HA | Complexity | Verdict |
|---|---|---|---|---|---|
A. Shared RWX PVC (cubepod-e2b-volumes) + subPath |
Single cluster-wide ReadWriteMany PVC (Longhorn RWX / SeaweedFS / NFS) mounted in cubepod-gateway and in Kata Pods via subPath: <volumeID>/data |
Native multi-node RWX; multiple sandboxes and forks share seamlessly | Native (all gateway replicas mount same RWX PVC) | Low (no sidecars, no Pod restarts) | Recommended |
B. Per-Volume RWX PVC (e2bvol-<id>) + Content Proxy Pod |
Each POST /volumes creates e2bvol-<id> RWX PVC; gateway proxies /volumecontent to an on-demand helper Pod mounting e2bvol-<id> |
Native multi-node RWX | Native | High (cold-start latency per volume upload, helper Pod lifecycle/GC) | Deferred / Optional |
C. Per-Volume RWO PVC + CSI VolumeSnapshot Clone |
Gateway writes to RWO PVC via helper Pod; sandbox creation clones a CSI snapshot per sandbox | Forks/multi-sandbox get isolated CoW snapshots | Complex (detach before snapshot/attach) | Very High (diverges from E2B shared mutable volume semantics) | Rejected |
Selected Design (Option A):
- A cluster-wide RWX PVC (default name
cubepod-e2b-volumes, configurable via Helme2bVolumes.pvcName/e2bVolumes.rwxStorageClassName) is mounted intocubepod-gatewayat/var/lib/cubepod/e2b-volumes(CUBEPOD_E2B_VOLUME_DIR). pkg/volume.Storestores tenant volume metadata at<dir>/<volumeID>/meta.jsonand<dir>/<volumeID>/attrs.json, and file contents at<dir>/<volumeID>/data/....- When a
PodBoxis created withE2BVolumeMounts,cubepod-controllermounts thecubepod-e2b-volumesPVC into the Kata Pod withsubPath: <volumeID>/dataat the staging path/run/cubepod/volumes/<volumeID>. - Because
cubepod-gatewayand every Kata Pod mount the exact same RWX PVC,/volumecontentAPI calls and in-sandbox filesystem operations read and write the single source of truth immediately, with no sync lag or split-brain.
API / CRD / Wire Changes
Section titled “API / CRD / Wire Changes”api/v1alpha1/types.go(E2BVolumeMount): RetainVolumeID,PVCName, andPath. When using the shared RWX PVC model,PVCNameis set to the shared RWX claim name (e.g.cubepod-e2b-volumes) ore2bvol-<volumeID>when per-volume RWX PVCs are enabled, andSubPath(optional fieldsubPath,omitempty) defaults to<volumeID>/datawhenPVCNameis the shared claim.type E2BVolumeMount struct {VolumeID string `json:"volumeID"`PVCName string `json:"pvcName"`SubPath string `json:"subPath,omitempty"`Path string `json:"path"`}pkg/gateway/volumes.go:- Replace
isKubeVolumeMountsUnsupported()’s blankettruewhenlivePodRouting && runtime == nilwith a capability check: return501 Not Implementedonly ifK8sE2BVolumeMountsis disabled or no RWX PVC is configured (s.e2bSharedPVC == ""). - In
ensureE2BVolumePVC: when using the shared RWX PVC (cubepod-e2b-volumes), no per-volume PVC creation is needed;st.NotePVC(volumeID, s.e2bSharedPVC, ns)records the shared claim.
- Replace
pkg/controller/manifests.go(applyOfficialVolumeMounts):- Mount each volume at staging path
/run/cubepod/volumes/<volumeID>(withSubPath: <volumeID>/datawhen using the shared PVC) instead ofm.Path. - Pass the mount mapping to
podd-initviaCUBEPOD_E2B_MOUNTSenv var (JSON array, compact) or/etc/cubepod/e2b-mounts.json(mounted from a per-PodBox ConfigMap or Pod annotation/DownwardAPI):[{"volume_id": "0123456789abcdef0123456789abcdef","staging_path": "/run/cubepod/volumes/0123456789abcdef0123456789abcdef","target_path": "/mnt/dataset"}]
- Mount each volume at staging path
In-Guest podd-init pivot_root Mount Propagation
Section titled “In-Guest podd-init pivot_root Mount Propagation”In crates/podd-init/src/main.rs, after mounting /workspace (step 4) and bind-mounting proc, sys, dev, run, opt/cubepod, etc/cubepod (step 5), and strictly before pivot_root(2) (step 6):
- Single-Threaded Read & Validate:
On the single main thread (preserving AGENTS.md Invariant 2),
podd-initreadsCUBEPOD_E2B_MOUNTS(or/etc/cubepod/e2b-mounts.jsonif present). For each entry{ volume_id, staging_path, target_path }:- Verify
staging_pathstarts with/run/cubepod/volumes/and exists as a directory. - Normalize
target_pathlexically: must start with/, contain no..or.segments, and must not equal or be a prefix/child of any reserved path (/workspace,/rootfs-upper,/opt/cubepod,/mnt/cubepod,/proc,/sys,/dev,/run,/etc,/bin,/sbin,/usr,/lib,/lib64,/root,/boot). This guarantees Dual-Volume Independence (AGENTS.md Invariant 1).
- Verify
- Create Target & Bind-Mount with
MS_NOSUID | MS_NODEV:- Compute
dst = cfg.merged_root_path.join(target_path.strip_prefix("/")). fs::create_dir_all(&dst)andchown_path(&dst, cfg.default_uid, cfg.default_gid)(1000:1000).- Call
libc::mount(staging_path, dst, NULL, MS_BIND, NULL). - Immediately remount read-write with security flags:
libc::mount(NULL, dst, NULL, MS_BIND | MS_REMOUNT | MS_NOSUID | MS_NODEV, NULL). - Note: Because
/runis already--rbindmounted into<merged_root>/runin step 5, or ifpodd-initunmounts/hides/run/cubepod/volumesinside<merged_root>/runafter binding, sandbox processes only see the volume attarget_path(not duplicated under/run/cubepod/volumes). To prevent/run/cubepod/volumesfrom leaking into<merged_root>/run,podd-initperforms E2B bind-mounts from/run/cubepod/volumes/<id>after--rbind /run <merged_root>/runand thenumount2(<merged_root>/run/cubepod/volumes, MNT_DETACH)(if present) prior topivot_root.
- Compute
pivot_root(2)&execvp(3):podd-initcallspivot_root(&cfg.merged_root_path, &old_root), lazily unmounts/.old_root, andexecvp(3)s/opt/cubepod/bin/podd. All E2B mounts remain active attarget_pathinside the new root.- Non-Pivot (
CUBEPOD_SKIP_PIVOT=1) Mode: WhenCUBEPOD_SKIP_PIVOT=1(e.g.,volumeMode: Filesystemwithout Kata),podd-initskipspivot_rootand directly bind-mountsstaging_pathontotarget_pathwithMS_BIND+MS_BIND | MS_REMOUNT | MS_NOSUID | MS_NODEV.
Fork & Lifecycle Semantics
Section titled “Fork & Lifecycle Semantics”ForkFromInteraction:PodBoxSpec.ForkFrom(CEP-0022) CoW-clones theRootFS UpperandWorkspaceblock PVCs. It does not clone E2B volumes.- When
POST /sandboxes/{id}/fork(orForkFrominPodBoxSpec) is invoked on a sandbox that hasE2BVolumeMounts:- On RWX-backed K8s clusters (and local Docker): the forked sandbox inherits the same
E2BVolumeMounts(explicitly re-attaching the shared RWX volume) and increments the volume’s active mount reference (st.AttachMount(forkedID, ...)), unless the caller explicitly specifies a differentvolumeMountslist. - If an RWO-backed volume is ever present,
ForkFromMUST return409 Conflict("cannot fork sandbox with RWO E2B volume mounts") to prevent CSI multi-attach deadlocks.
- On RWX-backed K8s clusters (and local Docker): the forked sandbox inherits the same
- PVC & Mount Reference Lifecycle:
DELETE /volumes/{volumeID}checks bothmounts.jsonand activePodBoxCRs (viareconciler.KubeClient()) before deleting<dir>/<volumeID>; any active reference returns409 Conflict.- Deleting/killing a
PodBoxreleases its mount record (st.DetachSandbox(sandboxID)). - The shared RWX PVC
cubepod-e2b-volumeshasRetainreclaim policy by default and is never deleted when individual E2B volumes or sandboxes are removed.
State Machine & Data Flow
Section titled “State Machine & Data Flow”flowchart LR
SDK["E2B SDK Client"] -->|"PUT /volumecontent/{id}/file"| GW["cubepod-gateway"]
GW -->|"openat2/O_NOFOLLOW + 1000:1000"| RWX["RWX PVC: cubepod-e2b-volumes/<id>/data"]
SDK -->|"POST /sandboxes (volumeMounts)"| GW
GW -->|"Create PodBox (E2BVolumeMounts)"| CTRL["cubepod-controller"]
CTRL -->|"Mount subPath <id>/data at /run/cubepod/volumes/<id>"| KATA["Kata Pod (podd-init)"]
KATA -->|"MS_BIND + MS_NOSUID|MS_NODEV -> <merged>/<target>"| PIVOT["pivot_root(2) -> execvp(podd)"]
PIVOT -->|"Read/Write single source of truth"| RWXBoundary Conditions & Failure Recovery
Section titled “Boundary Conditions & Failure Recovery”- Missing staging directory in
podd-init: If/run/cubepod/volumes/<id>does not exist whenCUBEPOD_E2B_MOUNTSlists it,podd-initfails fast with a non-zero exit code beforepivot_rootso the Pod entersCrashLoopBackOff/Failedrather than booting with a silent empty directory. - Gateway Pod restart:
pkg/volume.Storereloads<dir>/<id>/meta.jsonand reconcilesmounts.jsonagainst activePodBoxCRs on startup. - Unclean Kata Pod kill: Because E2B volumes are POSIX RWX filesystems mounted via
virtiofsd, killing the Kata VM does not requiree2fsckon the E2B volume (unlike block/workspace), andPodBoxdeletion cleans up the mount reference.
Test Plan
Section titled “Test Plan”- Unit Tests (Rust & Go):
crates/podd-init: Unit tests forCUBEPOD_E2B_MOUNTSparsing, lexical path normalization, reserved-path overlap rejection (/workspace,/workspace/sub,/rootfs-upper,/etc), andCUBEPOD_SKIP_PIVOT=1bind-mount behavior.pkg/controller/manifests_test.go: VerifyapplyOfficialVolumeMountsmounts at/run/cubepod/volumes/<volumeID>withSubPath: <volumeID>/dataand populatesCUBEPOD_E2B_MOUNTS.pkg/gateway/volumes_test.go: Verify501whenK8sE2BVolumeMountsis disabled / no RWX PVC configured, and201+PodBoxcreation when enabled; verifyForkFrominheritance on RWX and409on RWO.
- Integration / E2E Tests:
make test-e2e: Local Docker loopback +podd-initCUBEPOD_E2B_MOUNTSbind-mount verification.make test-k8s-e2e: Live Kubernetes Kata + RWX CSI test:Volume.create->Volume.write_file->Sandbox.create(volume_mounts=...)-> verify file readable/writable inside Kata sandbox -> verifynosuid,nodevin/proc/mounts-> verify/workspaceunaffected ->ForkFrominherits mount ->Sandbox.kill->Volume.delete.
Graduation Criteria
Section titled “Graduation Criteria”- Alpha: Behind
K8sE2BVolumeMountsfeature gate (defaultfalse); requirese2bVolumes.rwxStorageClassNamein Helm; unit tests and local E2E passing. - Beta: Enabled by default when RWX StorageClass is configured; live
make test-k8s-e2epassing on Kata + Longhorn/SeaweedFS RWX; Prometheus metrics wired; gateway multi-replica locking verified. - GA: Production bake time in CubePlex with multi-tenant quota enforcement and automated RWX capacity alerting.
Upgrade / Downgrade Strategy
Section titled “Upgrade / Downgrade Strategy”E2BVolumeMount.SubPathis optional (omitempty); existingPodBoxCRs withoutE2BVolumeMountsare unaffected.- Disabling
K8sE2BVolumeMountsrestores the CEP-0025501 Not Implementedgate for newPOST /sandboxesrequests withvolumeMountswithout affecting running sandboxes or/workspacePVCs.
Production Readiness Review Questionnaire (PRR)
Section titled “Production Readiness Review Questionnaire (PRR)”- Feature Enablement and Rollback: Controlled via feature gate
K8sE2BVolumeMountsand Helme2bVolumes.rwxStorageClassName. Setting the gate tofalseimmediately returns501for new K8svolumeMountswithout touching/workspaceorrootfs-upperPVCs. - Rollout, Upgrade and Rollback Planning: Running or
PausedKata Pods withoutE2BVolumeMountsare unaffected. Pods withE2BVolumeMountscontinue running during controller/gateway rolling upgrades because mounts are already established inside the guest. - Monitoring Requirements:
- Metrics:
cubepod_e2b_volume_mounts_total,cubepod_e2b_volume_mount_errors_total,cubepod_podd_init_e2b_bind_duration_ms. - Structured
slog/eprintln!logs on staging mount failures andForkFromconflicts.
- Metrics:
- Dependencies: Kubernetes CSI driver supporting
ReadWriteMany(e.g., Longhorn RWX, SeaweedFS CSI, or NFS CSI), Kata Containers (kata-qemuwithvirtiofsd), and Linuxmount(2)(MS_BIND,MS_NOSUID,MS_NODEV). - Scalability: Using a single shared RWX PVC (
cubepod-e2b-volumes) withsubPathavoids creating/attaching hundreds of CSI volumes per node, keeping CSI attach/detach concurrency constant (O(1)PVC per node/gateway). - Troubleshooting: Inspect
PodBox.spec.e2bVolumeMounts,kubectl logs <kata-pod>for[podd-init]mount errors, and/proc/mountsinside the guest to confirmnosuid,nodev.
Implementation History
Section titled “Implementation History”- 2026-11-15: Drafted CEP-0026 (
provisional,alpha) as the Kubernetes Kata & CSI follow-up to CEP-0025.
Drawbacks
Section titled “Drawbacks”- Requires a
ReadWriteMany(RWX) CSI StorageClass in Kubernetes clusters that enableK8sE2BVolumeMounts; clusters with only RWO block CSI cannot use this feature until a per-volume content-access proxy is added. - Multi-replica
cubepod-gatewaydeployments must coordinatemeta.json/mounts.jsonupdates over the shared RWX POSIX filesystem.
Alternatives
Section titled “Alternatives”- Per-Volume RWO PVC (
e2bvol-<id>) + CSIVolumeSnapshotClone: Rejected because RWO PVCs cannot be simultaneously attached tocubepod-gatewayand a Kata Pod (or multiple Kata Pods), and CSI snapshot cloning breaks E2B’s shared mutable volume semantics. - Per-Volume RWX PVC + On-Demand Content-Access Helper Pod: Rejected as the primary design because spinning up an ephemeral Pod on
PUT /volumecontentadds seconds of cold-start latency and Pod GC complexity compared to mounting one shared RWX PVC incubepod-gateway. - Gateway
poddProxy for/volumecontentWithout Shared Storage: Rejected because/volumecontentmust work before any sandbox exists (Volume.create->Volume.write_file->Sandbox.create).