mirror of
https://github.com/google-gemini/gemini-cli.git
synced 2026-03-24 13:01:29 -07:00
docs: add architecture for Gemini CLI Workspaces
This commit is contained in:
mkorwel
57
docs/architecture/workspaces/client-connectivity.md
Normal file
57
docs/architecture/workspaces/client-connectivity.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Detailed Design: Workspace Client & Connectivity
|
||||
|
||||
## 1. Introduction
|
||||
The Workspace Client is the part of `gemini-cli` that communicates with the Workspace Hub and provides the user interface for remote task management.
|
||||
|
||||
## 2. CLI Commands
|
||||
|
||||
### Management
|
||||
- `gemini workspace list`: Lists all active, stopped, and provisioning workspaces.
|
||||
- `gemini workspace create <name> [--image <tag>]`: Provisions a new remote environment.
|
||||
- `gemini workspace delete <name>`: Terminates the remote VM and cleans up state.
|
||||
- `gemini workspace stop/start <name>`: Manages the VM power state for cost savings.
|
||||
|
||||
### Connection
|
||||
- `gemini workspace connect <name>`:
|
||||
- Fetches the VM's IAP connection details from the Hub.
|
||||
- Initiates the secure SSH tunnel.
|
||||
- Syncs the local `~/.gemini` settings.
|
||||
- Attaches to the remote `shpool` session.
|
||||
|
||||
## 3. Connectivity Logic
|
||||
The client handles the complexity of IAP (Identity-Aware Proxy) and SSH.
|
||||
|
||||
### IAP Tunnelling
|
||||
1. **Auth Check:** Verifies the user is logged in to `gcloud` and has an active OAuth token for the Workspace Hub.
|
||||
2. **Lookup:** Queries the Hub's API to get the VM's internal name and zone.
|
||||
3. **Tunnel:** Executes `gcloud compute ssh <vm_name> --tunnel-through-iap`.
|
||||
|
||||
### SSH Parameters
|
||||
- **Agent Forwarding (`-A`):** Critical for Git operations on the remote VM.
|
||||
- **Environment Injection:** Pass local environment variables (like `TERM` or `LANG`) to ensure terminal compatibility.
|
||||
- **Secret Sync:** Before starting the shell, a side-channel (e.g., `scp`) pushes the GitHub PAT to `/dev/shm/.gh_token`.
|
||||
|
||||
## 4. UI Implementation (Workspaces Ability)
|
||||
A new interactive component in `packages/cli/src/ui/abilities/workspaces/`.
|
||||
|
||||
- **Sidebar:** Tree view of all workspaces across registered Hubs.
|
||||
- **Main View:**
|
||||
- **Status Dashboard:** Shows CPU/Memory load, active repo/branch, and uptime.
|
||||
- **Control Center:** Large buttons for Connect, Start/Stop, and Delete.
|
||||
- **Batch Console:** Allows broadcasting a command (e.g., `npm run lint`) to multiple selected workspaces.
|
||||
|
||||
## 5. Multi-Hub Configuration
|
||||
The CLI supports talking to multiple Hubs simultaneously (e.g., one personal, one team-wide).
|
||||
|
||||
- **Settings (`settings.json`):**
|
||||
```json
|
||||
{
|
||||
"workspaces": {
|
||||
"hubs": [
|
||||
{ "name": "Personal", "url": "https://hub-xyz.a.run.app" },
|
||||
{ "name": "Team Alpha", "url": "https://hub-alpha.a.run.app" }
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
- **Context Awareness:** The CLI automatically switches the active Hub context based on the current directory or a command-line flag.
|
||||
41
docs/architecture/workspaces/container-image.md
Normal file
41
docs/architecture/workspaces/container-image.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Detailed Design: Workspace Container Image
|
||||
|
||||
## 1. Introduction
|
||||
The Workspace Container Image defines the standardized software environment for all remote execution. It is pre-built and optimized for fast startup on GCE instances.
|
||||
|
||||
## 2. Dockerfile Specification
|
||||
The image is maintained in `packages/grid-manager/docker/Dockerfile`.
|
||||
|
||||
- **Base:** `node:20-slim`
|
||||
- **Environment:**
|
||||
- `GEMINI_CLI_WORKSPACE=1`
|
||||
- `DEBIAN_FRONTEND=noninteractive`
|
||||
- **Tools:**
|
||||
- `git`, `rsync`, `curl`, `vim`, `tmux`, `shpool`.
|
||||
- `gh` (GitHub CLI).
|
||||
- `google-cloud-sdk` (via apt-get).
|
||||
- Pre-compiled `gemini-cli` binary.
|
||||
- **User:** `node` (UID 1000) for unprivileged execution.
|
||||
|
||||
## 3. Image Contents & Pre-loading
|
||||
- The `gemini-cli` nightly binary is pre-loaded into `/usr/local/bin/gemini`.
|
||||
- Standard node dependencies (`npm`, `yarn`, `pnpm`) are pre-installed.
|
||||
- `shpool` is used as the primary process manager to allow terminal detachment and re-attachment.
|
||||
|
||||
## 4. Entrypoint Strategy (`entrypoint.sh`)
|
||||
When the container starts on GCE:
|
||||
1. **Secret Injection:** Reads the GitHub PAT from a memory-only mount (`/dev/shm/github_token`) and authenticates `gh`.
|
||||
2. **Settings Restore:** Syncs the user's `~/.gemini/` configuration (aliased from `/home/node/.gemini_volume`).
|
||||
3. **Persistence Layer:** Starts `shpool` daemon in the background.
|
||||
4. **Ready Signal:** Notifies the Workspace Hub that the environment is ready for connection.
|
||||
|
||||
## 5. Storage Strategy
|
||||
- **System:** The container image itself is ephemeral.
|
||||
- **User Home:** A persistent GCE Disk (PD) is mounted at `/home/node`. This ensures:
|
||||
- `~/.gemini` settings persist.
|
||||
- Cloned git repositories persist between workspace restarts.
|
||||
- `npm install` artifacts (node_modules) persist.
|
||||
|
||||
## 6. Build & Release
|
||||
- The image is automatically built and pushed to the Hub's Artifact Registry on every `main` push or new `nightly` tag.
|
||||
- The Hub API defaults to using the `latest` or `nightly` tag unless specified otherwise.
|
||||
56
docs/architecture/workspaces/hub-service.md
Normal file
56
docs/architecture/workspaces/hub-service.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# Detailed Design: Workspace Hub Service
|
||||
|
||||
## 1. Introduction
|
||||
The Workspace Hub is a serverless application (deployed on Cloud Run) that manages the fleet of remote execution environments. It is designed as a deployable, self-service feature for developers and teams.
|
||||
|
||||
## 2. API Endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| `GET` | `/workspaces` | List all workspaces for the authenticated user. |
|
||||
| `POST` | `/workspaces` | Request creation of a new GCE-backed workspace. |
|
||||
| `DELETE` | `/workspaces/:id` | Destroy the VM and clean up Firestore state. |
|
||||
| `POST` | `/workspaces/:id/stop` | Suspend the GCE instance (cost-saving). |
|
||||
| `POST` | `/workspaces/:id/start` | Resume a suspended instance. |
|
||||
| `GET` | `/workspaces/:id/status` | Get real-time status from GCE and Firestore. |
|
||||
|
||||
## 3. Firestore State Store
|
||||
The Hub maintains a centralized state to enable multi-device synchronization.
|
||||
|
||||
- **Collection:** `workspaces`
|
||||
- `id`: Unique identifier (UUID).
|
||||
- `owner_id`: Google User ID (from OAuth).
|
||||
- `instance_name`: GCE VM name.
|
||||
- `zone`: GCE Zone (e.g., `us-west1-a`).
|
||||
- `image_tag`: Docker image tag currently in use.
|
||||
- `machine_type`: GCE Machine type (e.g., `e2-standard-4`).
|
||||
- `status`: One of `PROVISIONING`, `READY`, `SUSPENDED`, `ERROR`.
|
||||
- `last_connected_at`: Timestamp for auto-cleanup logic.
|
||||
- `metadata`: `{ repo: string, branch: string, device_id: string }`.
|
||||
|
||||
## 4. GCE Lifecycle Management
|
||||
The Hub uses the GCP Compute Engine Node.js SDK to interact with VMs.
|
||||
|
||||
### Provisioning
|
||||
1. Verify the user has quota and permissions.
|
||||
2. Call `instances.insert` with "Container-on-VM" configuration.
|
||||
3. Inject cloud-init or metadata scripts to:
|
||||
- Setup SSH (via IAP).
|
||||
- Configure the memory-only mount for secrets.
|
||||
- Notify the Hub when the container is ready.
|
||||
|
||||
### Auto-Cleanup (TTL)
|
||||
- A periodic Cloud Scheduler job triggers a `/cleanup` endpoint on the Hub.
|
||||
- Idle workspaces (based on `last_connected_at`) are automatically stopped or deleted to prevent unnecessary GCP costs.
|
||||
|
||||
## 5. Multi-Tenancy Implementation
|
||||
- **Team Mode:** The Hub's service account must have "Compute Admin" roles on the shared project.
|
||||
- **Access Control:** Every API request is checked against the `owner_id` in Firestore. Only the owner (or an admin in team mode) can modify or delete a workspace.
|
||||
- **Resource Isolation:** Each workspace is an independent VM. There is no sharing of CPU/Memory between workspaces.
|
||||
|
||||
## 6. Deployment
|
||||
The Hub is provided as a Terraform module (`/terraform/workspace-hub/`) for automated setup of:
|
||||
- Cloud Run service.
|
||||
- Firestore database.
|
||||
- Artifact Registry (for Workspace Images).
|
||||
- IAM roles and permissions.
|
||||
44
docs/architecture/workspaces/overview.md
Normal file
44
docs/architecture/workspaces/overview.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Gemini CLI Workspaces: High-Level Architecture Overview
|
||||
|
||||
## 1. Introduction
|
||||
Gemini CLI Workspaces provides a distributed, persistent, and multi-device compute layer for `gemini-cli`. It enables developers to provision, manage, and "teleport" into remote execution environments hosted on GCP.
|
||||
|
||||
## 2. Core Vision
|
||||
- **Persistence:** Sessions survive local device disconnects or terminal restarts.
|
||||
- **Portability:** Start a task on one device (e.g., Laptop) and seamlessly re-attach from another (e.g., Surface/Tablet).
|
||||
- **Scale:** Offload heavy compute (builds, tests, evals) to remote GCE instances.
|
||||
- **Consistency:** Pre-built container images ensure every workspace has the exact same tools and environment.
|
||||
|
||||
## 3. High-Level Architecture
|
||||
The architecture is centered around a **Workspace Hub**, which acts as the fleet manager, and **Remote Workspaces**, which are containerized GCE VMs.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Local Device: gemini-cli] -->|API Calls| B(Workspace Hub: Cloud Run)
|
||||
A -->|SSH/IAP| C(Remote Workspace: GCE VM)
|
||||
B -->|Provision/Monitor| C
|
||||
B -->|State Persistence| D(Firestore)
|
||||
C -->|Pull Image| E(Artifact Registry)
|
||||
```
|
||||
|
||||
## 4. Multi-Tenancy Models
|
||||
The Workspace Hub is a self-service, deployable feature that supports several grains of multi-tenancy:
|
||||
|
||||
### A. Per-User (Personal Cloud)
|
||||
- **Deployment:** Each developer deploys their own Workspace Hub in a personal GCP project.
|
||||
- **Isolation:** Absolute. All VMs and secrets belong to the individual.
|
||||
|
||||
### B. Per-Team (Shared Infrastructure)
|
||||
- **Deployment:** A single Hub managed by a team/org.
|
||||
- **Tenancy:** Identity-based partitioning. The Hub filters instances based on the authenticated user's Google ID.
|
||||
- **Isolation:** Instances are tagged with `owner_id`. Users can only manage their own environments.
|
||||
|
||||
### C. Per-Repository (Project Environments)
|
||||
- **Deployment:** Tied to a specific repo (e.g., for PR reviews or ephemeral test envs).
|
||||
- **Tenancy:** Project-context isolation. Users can connect to any workspace associated with the repository context.
|
||||
|
||||
## 5. Multi-Device Portability
|
||||
Since the Workspace Hub stores the state centrally (Firestore), any device with the authenticated `gemini-cli` can:
|
||||
1. Query the Hub for the list of active workspaces.
|
||||
2. Initiate a connection to a remote VM started by *another* device.
|
||||
3. Sync its local `~/.gemini` settings and GitHub PAT to ensure a consistent experience on the remote side.
|
||||
48
docs/architecture/workspaces/sync-and-auth.md
Normal file
48
docs/architecture/workspaces/sync-and-auth.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Detailed Design: Sync & Authentication Mechanism
|
||||
|
||||
## 1. Introduction
|
||||
A core requirement for Gemini CLI Workspaces is the secure and seamless synchronization of developer settings and repository credentials between the local machine and the remote environment.
|
||||
|
||||
## 2. Configuration Sync (`~/.gemini/`)
|
||||
To maintain a consistent experience, the local CLI synchronizes the developer's environment to the remote workspace.
|
||||
|
||||
### Sync Strategy
|
||||
- **Trigger:** Initial creation and subsequent connections via `gemini workspace connect`.
|
||||
- **Mechanism:** Secure `rsync` over SSH tunnel.
|
||||
- **Scope:**
|
||||
- `~/.gemini/settings.json` (aliases, UI preferences).
|
||||
- `~/.gemini/commands/` (custom user commands).
|
||||
- `~/.gemini/skills/` (custom user-defined skills).
|
||||
- **Exclusions:**
|
||||
- `.gemini/logs/` (local device specific).
|
||||
- `.gemini/cache/` (can be large and re-generated).
|
||||
- Sensitive files containing the word `*secret*` or `*key*` (these use the injection mechanism below).
|
||||
|
||||
## 3. Git Authentication (SSH Agent Forwarding)
|
||||
Private SSH keys never leave the developer's local machine.
|
||||
|
||||
- **Setup:** The local CLI ensures `ssh-agent` is running and the required keys are added.
|
||||
- **Connection:** The SSH connection from the CLI uses the `-A` (Agent Forwarding) flag.
|
||||
- **Remote:** Git on the remote VM automatically uses the forwarded agent socket for cloning or pushing to repositories (GitHub, GitLab, internal).
|
||||
|
||||
## 4. GitHub CLI (`gh`) Authentication
|
||||
For advanced integration (PRs, issues), the GitHub CLI requires a token.
|
||||
|
||||
- **Injection Process:**
|
||||
1. Local CLI fetches a GitHub Personal Access Token (PAT) from the OS Keychain (e.g., via `keytar`).
|
||||
2. During the SSH handshake, the CLI writes the token to `/dev/shm/.gh_token` (a memory-only, non-persistent mount).
|
||||
3. The remote shell profile (`.bashrc` or `.zshrc`) exports `GH_TOKEN=$(cat /dev/shm/.gh_token)`.
|
||||
- **Cleanup:** Terminating the workspace container or VM wipes the `/dev/shm` mount, ensuring no long-lived PATs remain on the remote disk.
|
||||
|
||||
## 5. Hub Authentication
|
||||
Access to the Workspace Hub API is secured via Google OAuth.
|
||||
|
||||
- **Local:** The CLI uses the `gcloud` or a built-in OAuth flow to obtain a token.
|
||||
- **Hub:** The Hub service (on Cloud Run) validates the bearer token and extracts the `sub` (Google User ID) to partition data in Firestore.
|
||||
|
||||
## 6. Multi-Device "Handover" Workflow
|
||||
1. **Device A (Laptop):** Creates workspace `work-1`. Hub records `owner=user-x`.
|
||||
2. **Device B (Surface):** User moves to a coffee shop. Runs `gemini workspace connect work-1`.
|
||||
3. **Hub Check:** Hub verifies the user is `user-x`.
|
||||
4. **Client Sync:** **Device B** pushes its local `~/.gemini` settings and its own GitHub PAT to `work-1`.
|
||||
5. **Result:** `work-1` now behaves exactly like the user's local **Device B** environment.
|
||||
@@ -136,6 +136,33 @@
|
||||
{ "label": "Sandboxing", "slug": "docs/cli/sandbox" },
|
||||
{ "label": "Settings", "slug": "docs/cli/settings" },
|
||||
{ "label": "Telemetry", "slug": "docs/cli/telemetry" },
|
||||
{
|
||||
"label": "Workspaces",
|
||||
"badge": "🔬",
|
||||
"collapsed": true,
|
||||
"items": [
|
||||
{
|
||||
"label": "Architecture Overview",
|
||||
"slug": "docs/architecture/workspaces/overview"
|
||||
},
|
||||
{
|
||||
"label": "Workspace Hub",
|
||||
"slug": "docs/architecture/workspaces/hub-service"
|
||||
},
|
||||
{
|
||||
"label": "Workspace Image",
|
||||
"slug": "docs/architecture/workspaces/container-image"
|
||||
},
|
||||
{
|
||||
"label": "Client & Connectivity",
|
||||
"slug": "docs/architecture/workspaces/client-connectivity"
|
||||
},
|
||||
{
|
||||
"label": "Sync & Authentication",
|
||||
"slug": "docs/architecture/workspaces/sync-and-auth"
|
||||
}
|
||||
]
|
||||
},
|
||||
{ "label": "Token caching", "slug": "docs/cli/token-caching" }
|
||||
]
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user