mirror of
https://github.com/google-gemini/gemini-cli.git
synced 2026-04-21 18:44:30 -07:00
feat(workspaces): transform offload into repository-agnostic Gemini Workspaces
This commit is contained in:
@@ -1,38 +0,0 @@
|
||||
# Offload Skill
|
||||
|
||||
This skill provides a high-performance, parallelized workflow for offloading intensive developer tasks (PR reviews, fixing CI, preparing merges) to a remote workstation. It leverages a Node.js orchestration engine to run complex validation playbooks concurrently in a dedicated terminal window.
|
||||
|
||||
## Performance Workflow
|
||||
|
||||
|
||||
The `offload` skill supports the following specialized playbooks:
|
||||
|
||||
- **`review`** (default): Clean build, CI status check, static analysis, and behavioral proofs.
|
||||
- **`fix`**: Build + Log analysis of CI failures + Iterative Gemini-led fixing and pushing.
|
||||
- **`ready`**: Final full validation (clean install, full preflight, and conflict checks).
|
||||
- **`open`**: Provision a worktree and drop the user directly into a remote shell/tmux session.
|
||||
- **`implement`**: Read an issue → Research → Implement → Verify → Create PR.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Initializing an Offload Task
|
||||
When the user asks to offload a task (e.g., "Offload PR 123 fix" or "Make PR 123 ready"), use the `run_shell_command` tool to execute the orchestrator:
|
||||
* **Command**: `npm run offload <PR_NUMBER> [action]`
|
||||
* **Action**: This will sync scripts to the remote host, provision a worktree, and pop a new terminal window for the playbook dashboard.
|
||||
* **Response**: Inform the user which playbook has been launched.
|
||||
|
||||
### 2. Monitoring and Synthesis
|
||||
The remote worker saves all results into `.gemini/logs/offload-<PR_NUMBER>/`. Once the playbook finishes, the agent should synthesize the results:
|
||||
* Read logs corresponding to the playbook tasks (e.g., `build.log`, `review.md`, `test-execution.log`, `diagnostics.log`).
|
||||
* Check the `.exit` files to confirm success of each parallel stage.
|
||||
|
||||
### 3. Final Recommendation
|
||||
Provide a structured assessment based on the physical proof and logs:
|
||||
* **Status**: PASS / FAIL / NEEDS_WORK.
|
||||
* **Findings**: Categorized by Critical, Improvements, or Nitpicks.
|
||||
* **Conclusion**: A clear next step for the maintainer.
|
||||
|
||||
## Best Practices
|
||||
* **Isolation First**: Always respect the user's isolation choices (`~/.offload/gemini-cli-config`).
|
||||
* **Be Behavioral**: Prioritize results from live execution (behavioral proofs) over static reading.
|
||||
* **Multi-tasking**: Remind the user they can continue chatting in the main window while the heavy offloaded task runs in the separate window.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Future State: Gemini Workspaces Platform
|
||||
|
||||
This document outlines the long-term architectural evolution of the Workspaces feature (formerly "Workspace").
|
||||
|
||||
## 🎯 Vision
|
||||
Transform Workspaces into a first-class platform capability that allows developers to seamlessly move intensive workloads (AI reasoning, complex builds, parallel testing) to any compute environment (Cloud or Local).
|
||||
|
||||
## 🗺️ Evolutionary Roadmap
|
||||
|
||||
### Phase 1: Generalization & Renaming (Current)
|
||||
- **Goal**: Make the feature useful for any repository, not just Gemini CLI.
|
||||
- **Action**: Rename to "Workspaces."
|
||||
- **Action**: Implement dynamic repository detection via Git.
|
||||
- **Action**: Isolate all state into `.gemini/workspaces/`.
|
||||
|
||||
### Phase 2: Pluggable Compute Extensions
|
||||
- **Goal**: Decouple the infrastructure logic from the core CLI.
|
||||
- **Action**: Move `WorkerProviders` into a dedicated **Workspaces Extension**.
|
||||
- **Action**: Support multiple providers (GCP, AWS, Local Docker).
|
||||
- **Action**: Define a standard API for Workspace Providers.
|
||||
|
||||
### Phase 3: Core Integration
|
||||
- **Goal**: Standardize the user experience.
|
||||
- **Action**: Move the high-level `gemini workspace` command into the core `gemini` binary.
|
||||
- **Action**: Implement automated "Environment Hand-off" where the local agent can automatically spin up a remote workspace when it detects a heavy task.
|
||||
|
||||
### Phase 4: Public Marketplace
|
||||
- **Goal**: Community adoption.
|
||||
- **Action**: Publish the official GCP Workspace Extension.
|
||||
- **Action**: Provide a "Zero-Config" public base image for standard Node/TS development.
|
||||
|
||||
## 🏗️ Architectural Principles
|
||||
1. **BYOC (Bring Your Own Cloud)**: Users connect their own infrastructure.
|
||||
2. **Nested Persistence**: Keep the environment in the container, but manage the lifecycle with the host.
|
||||
3. **Repo-Agnostic**: One set of tools should work for any project.
|
||||
@@ -1,4 +1,4 @@
|
||||
# Architectural Mandate: High-Performance Offload System
|
||||
# Architectural Mandate: High-Performance Workspace System
|
||||
|
||||
## Infrastructure Strategy
|
||||
- **Base OS**: Always use **Container-Optimized OS (COS)** (`cos-stable` family). It is security-hardened and has Docker pre-installed.
|
||||
@@ -12,7 +12,7 @@
|
||||
- **Mounts**: Standardize on these host-to-container mappings:
|
||||
- `~/dev` -> `/home/node/dev` (Persistence for worktrees)
|
||||
- `~/.gemini` -> `/home/node/.gemini` (Shared credentials)
|
||||
- `~/.offload` -> `/home/node/.offload` (Shared scripts/logs)
|
||||
- `~/.workspace` -> `/home/node/.workspace` (Shared scripts/logs)
|
||||
- **Runtime**: The container runs as a persistent service (`--restart always`) acting as a "Remote Workstation" rather than an ephemeral task.
|
||||
|
||||
## Orchestration Logic
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# Network Architecture & Troubleshooting Research
|
||||
|
||||
This document captures the empirical research and final configuration settled upon for the Gemini CLI Offload system, specifically addressing the challenges of connecting from corporate environments to private GCP workers.
|
||||
This document captures the empirical research and final configuration settled upon for the Gemini CLI Workspace system, specifically addressing the challenges of connecting from corporate environments to private GCP workers.
|
||||
|
||||
## 🔍 The Challenge
|
||||
The goal was to achieve **Direct internal SSH** access to GCE workers that have **no public IP addresses**, allowing for high-performance file synchronization (`rsync`) and interactive sessions without the overhead of `gcloud` wrappers.
|
||||
@@ -1,8 +1,8 @@
|
||||
# Mission: GCE Container-First Refactor 🚀
|
||||
|
||||
## Current State
|
||||
- **Architecture**: Persistent GCE VM (`gcli-offload-mattkorwel`) with Fast-Path SSH (`gcli-worker`).
|
||||
- **Logic**: Decoupled scripts in `~/.offload/scripts`, using Git Worktrees for concurrency.
|
||||
- **Architecture**: Persistent GCE VM (`gcli-workspace-mattkorwel`) with Fast-Path SSH (`gcli-worker`).
|
||||
- **Logic**: Decoupled scripts in `~/.workspace/scripts`, using Git Worktrees for concurrency.
|
||||
- **Auth**: Scoped GitHub PATs mirrored via setup.
|
||||
|
||||
## The Goal (Container-OS Transition)
|
||||
@@ -45,13 +45,13 @@ This phase is currently **ARCHIVED** in favor of the Persistent Workstation mode
|
||||
The orchestrator should launch isolated containers using this pattern:
|
||||
```bash
|
||||
docker run --rm -it \
|
||||
--name offload-job-id \
|
||||
--name workspace-job-id \
|
||||
-v ~/dev/worktrees/job-id:/home/node/dev/worktree:rw \
|
||||
-v ~/dev/main:/home/node/dev/main:ro \
|
||||
-v ~/.gemini:/home/node/.gemini:ro \
|
||||
-w /home/node/dev/worktree \
|
||||
maintainer-image:latest \
|
||||
sh -c "tsx ~/.offload/scripts/entrypoint.ts ..."
|
||||
sh -c "tsx ~/.workspace/scripts/entrypoint.ts ..."
|
||||
```
|
||||
|
||||
## How to Resume
|
||||
@@ -1,11 +1,11 @@
|
||||
# Offload maintainer skill
|
||||
# Workspace maintainer skill
|
||||
|
||||
The `offload` skill provides a high-performance, parallelized workflow for
|
||||
offloading intensive developer tasks to a remote workstation. It leverages a
|
||||
The `workspace` skill provides a high-performance, parallelized workflow for
|
||||
workspaceing intensive developer tasks to a remote workstation. It leverages a
|
||||
Node.js orchestrator to run complex validation playbooks concurrently in a
|
||||
dedicated terminal window.
|
||||
|
||||
## Why use offload?
|
||||
## Why use workspace?
|
||||
|
||||
As a maintainer, you eventually reach the limits of how much work you can manage
|
||||
at once on a single local machine. Heavy builds, concurrent test suites, and
|
||||
@@ -13,9 +13,9 @@ multiple PRs in flight can quickly overload local resources, leading to
|
||||
performance degradation and developer friction.
|
||||
|
||||
While manual remote management is a common workaround, it is often cumbersome
|
||||
and context-heavy. The `offload` skill addresses these challenges by providing:
|
||||
and context-heavy. The `workspace` skill addresses these challenges by providing:
|
||||
|
||||
- **Elastic compute**: Offload resource-intensive build and lint suites to a
|
||||
- **Elastic compute**: Workspace resource-intensive build and lint suites to a
|
||||
beefy remote workstation, keeping your local machine responsive.
|
||||
- **Context preservation**: The main Gemini session remains interactive and
|
||||
focused on high-level reasoning while automated tasks provide real-time
|
||||
@@ -25,11 +25,11 @@ and context-heavy. The `offload` skill addresses these challenges by providing:
|
||||
- **True parallelism**: Infrastructure validation, CI checks, and behavioral
|
||||
proofs run simultaneously, compressing a 15-minute process into 3 minutes.
|
||||
|
||||
## Agentic skills: Sync or Offload
|
||||
## Agentic skills: Sync or Workspace
|
||||
|
||||
The `offload` system is designed to work in synergy with specialized agentic
|
||||
The `workspace` system is designed to work in synergy with specialized agentic
|
||||
skills. These skills can be run **synchronously** in your current terminal for
|
||||
quick tasks, or **offloaded** to a remote session for complex, iterative loops.
|
||||
quick tasks, or **workspaceed** to a remote session for complex, iterative loops.
|
||||
|
||||
- **`review-pr`**: Conducts high-fidelity, behavioral code reviews. It assumes
|
||||
the infrastructure is already validated and focuses on physical proof of
|
||||
@@ -37,15 +37,15 @@ quick tasks, or **offloaded** to a remote session for complex, iterative loops.
|
||||
- **`fix-pr`**: An autonomous "Fix-to-Green" loop. It iteratively addresses
|
||||
CI failures, merge conflicts, and review comments until the PR is mergeable.
|
||||
|
||||
When you run `npm run offload <PR> fix`, the orchestrator provisions the remote
|
||||
When you run `npm run workspace <PR> fix`, the orchestrator provisions the remote
|
||||
environment and then launches a Gemini CLI session specifically powered by the
|
||||
`fix-pr` skill.
|
||||
|
||||
## Architecture: The Hybrid Powerhouse
|
||||
|
||||
The offload system uses a **Hybrid VM + Docker** architecture designed for maximum performance and reliability:
|
||||
The workspace system uses a **Hybrid VM + Docker** architecture designed for maximum performance and reliability:
|
||||
|
||||
1. **The GCE VM (Raw Power)**: By running on high-performance Google Compute Engine instances, we offload heavy CPU and RAM tasks (like full project builds and massive test suites) from your local machine, keeping your primary workstation responsive.
|
||||
1. **The GCE VM (Raw Power)**: By running on high-performance Google Compute Engine instances, we workspace heavy CPU and RAM tasks (like full project builds and massive test suites) from your local machine, keeping your primary workstation responsive.
|
||||
2. **The Docker Container (Consistency & Resilience)**:
|
||||
* **Source of Truth**: The `.gcp/Dockerfile.maintainer` defines the exact environment. If a tool is added there, every maintainer gets it instantly.
|
||||
* **Zero Drift**: Containers are immutable. Every job starts in a fresh state, preventing the "OS rot" that typically affects persistent VMs.
|
||||
@@ -66,7 +66,7 @@ For a complete guide on setting up your remote environment, see the [Maintainer
|
||||
|
||||
### Persistence and Job Recovery
|
||||
|
||||
The offload system is designed for high reliability and persistence. Jobs use a nested execution model to ensure they continue running even if your local terminal is closed or the connection is lost.
|
||||
The workspace system is designed for high reliability and persistence. Jobs use a nested execution model to ensure they continue running even if your local terminal is closed or the connection is lost.
|
||||
|
||||
### How it Works
|
||||
1. **Host-Level Persistence**: The orchestrator launches each job in a named **`tmux`** session on the remote VM.
|
||||
@@ -75,12 +75,12 @@ The offload system is designed for high reliability and persistence. Jobs use a
|
||||
### Re-attaching to a Job
|
||||
If you lose your connection, you can easily resume your session:
|
||||
|
||||
- **Automatic**: Simply run the exact same command you started with (e.g., `npm run offload 123 review`). The system will automatically detect the existing session and re-attach you.
|
||||
- **Manual**: Use `npm run offload:status` to find the session name, then use `ssh gcli-worker` to jump into the VM and `tmux attach -t <session>` to resume.
|
||||
- **Automatic**: Simply run the exact same command you started with (e.g., `npm run workspace 123 review`). The system will automatically detect the existing session and re-attach you.
|
||||
- **Manual**: Use `npm run workspace:status` to find the session name, then use `ssh gcli-worker` to jump into the VM and `tmux attach -t <session>` to resume.
|
||||
|
||||
## Technical details
|
||||
|
||||
This skill uses a **Worker Provider** abstraction (`GceCosProvider`) to manage the remote lifecycle. It uses an isolated Gemini profile on the remote host (`~/.offload/gemini-cli-config`) to ensure that verification tasks do not interfere with your primary configuration.
|
||||
This skill uses a **Worker Provider** abstraction (`GceCosProvider`) to manage the remote lifecycle. It uses an isolated Gemini profile on the remote host (`~/.workspace/gemini-cli-config`) to ensure that verification tasks do not interfere with your primary configuration.
|
||||
|
||||
### Directory structure
|
||||
- `scripts/providers/`: Modular worker implementations (GCE, etc.).
|
||||
@@ -95,13 +95,13 @@ This skill uses a **Worker Provider** abstraction (`GceCosProvider`) to manage t
|
||||
If you want to improve this skill:
|
||||
1. Modify the TypeScript scripts in `scripts/`.
|
||||
2. Update `SKILL.md` if the agent's instructions need to change.
|
||||
3. Test your changes locally using `npm run offload <PR>`.
|
||||
3. Test your changes locally using `npm run workspace <PR>`.
|
||||
|
||||
## Testing
|
||||
|
||||
The orchestration logic for this skill is fully tested. To run the tests:
|
||||
```bash
|
||||
npx vitest .gemini/skills/offload/tests/orchestration.test.ts
|
||||
npx vitest .gemini/skills/workspace/tests/orchestration.test.ts
|
||||
```
|
||||
These tests mock the external environment (SSH, GitHub CLI, and the file system) to ensure that the orchestration scripts generate the correct commands and handle environment isolation accurately.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Gemini Workspaces Skill
|
||||
|
||||
This skill enables the agent to utilize **Gemini Workspaces**—a high-performance, persistent remote development platform. It allows the agent to move intensive tasks (PR reviews, complex repairs, full builds) from the local environment to a dedicated cloud worker.
|
||||
|
||||
## 🛠️ Key Capabilities
|
||||
1. **Persistent Execution**: Jobs run in remote `tmux` sessions. Disconnecting or crashing the local terminal does not stop the remote work.
|
||||
2. **Parallel Infrastructure**: The agent can launch a heavy task (like a full build or CI run) in a workspace while continuing to assist the user locally.
|
||||
3. **Behavioral Fidelity**: Remote workers have full tool access (Git, Node, Docker, etc.) and high-performance compute, allowing the agent to provide behavioral proofs of its work.
|
||||
|
||||
## 📋 Instructions for the Agent
|
||||
|
||||
### When to use Workspaces
|
||||
- **Intensive Tasks**: Full preflight runs, large-scale refactors, or deep PR reviews.
|
||||
- **Persistent Logic**: When a task is expected to take longer than a few minutes and needs to survive local connection drops.
|
||||
- **Environment Isolation**: When you need a clean, high-performance environment to verify a fix without polluting the user's local machine.
|
||||
|
||||
### How to use Workspaces
|
||||
1. **Setup**: If the user hasn't initialized their environment, instruct them to run `npm run workspace:setup`.
|
||||
2. **Launch**: Use the `workspace` command to start a playbook:
|
||||
```bash
|
||||
npm run workspace <PR_NUMBER> [action]
|
||||
```
|
||||
- Actions: `review` (default), `fix`, `ready`.
|
||||
3. **Check Status**: Poll the progress using `npm run workspace:check <PR_NUMBER>` or see the global state with `npm run workspace:status`.
|
||||
|
||||
## ⚠️ Important Constraints
|
||||
- **Absolute Paths**: Always use absolute paths (e.g., `/home/node/...`) when orchestrating remote commands.
|
||||
- **Be Behavioral**: Prioritize results from live execution (behavioral proofs) over static reading.
|
||||
- **Multi-tasking**: Remind the user they can continue chatting in the main window while the heavy workspace task runs in the separate terminal window.
|
||||
+4
-4
@@ -1,4 +1,4 @@
|
||||
# Plan: Worker Provider Abstraction for Offload System
|
||||
# Plan: Worker Provider Abstraction for Workspace System
|
||||
|
||||
## Objective
|
||||
Abstract the remote execution infrastructure (GCE COS, GCE Linux, Cloud Workstations) behind a common `WorkerProvider` interface. This eliminates infrastructure-specific prompts (like "use container mode") and makes the system extensible to new backends.
|
||||
@@ -11,7 +11,7 @@ Create a modular provider system where each infrastructure type implements a sta
|
||||
- **Implementations**:
|
||||
- `GceCosProvider`: Handles COS with Cloud-Init and `docker exec` wrapping.
|
||||
- `GceLinuxProvider`: Handles standard Linux VMs with direct execution.
|
||||
- `LocalDockerProvider`: (Future) Runs offload tasks in a local container.
|
||||
- `LocalDockerProvider`: (Future) Runs workspace tasks in a local container.
|
||||
- `WorkstationProvider`: (Future) Integrates with Google Cloud Workstations.
|
||||
|
||||
### 2. Auto-Discovery
|
||||
@@ -41,6 +41,6 @@ Refactor `orchestrator.ts` to be provider-agnostic:
|
||||
- Ensure "Fast-Path SSH" is still the primary interactive gateway.
|
||||
|
||||
## Verification
|
||||
- Run `npm run offload:fleet provision` and ensure it creates a COS-native worker.
|
||||
- Run `npm run offload:setup` and verify it no longer asks cryptic infrastructure questions.
|
||||
- Run `npm run workspace:fleet provision` and ensure it creates a COS-native worker.
|
||||
- Run `npm run workspace:setup` and verify it no longer asks cryptic infrastructure questions.
|
||||
- Launch a review and verify it uses `docker exec internally for the COS provider.
|
||||
+8
-8
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Offload Attach Utility (Local)
|
||||
* Workspace Attach Utility (Local)
|
||||
*
|
||||
* Re-attaches to a running tmux session inside the container on the worker.
|
||||
*/
|
||||
@@ -20,27 +20,27 @@ export async function runAttach(args: string[], env: NodeJS.ProcessEnv = process
|
||||
const isLocal = args.includes('--local');
|
||||
|
||||
if (!prNumber) {
|
||||
console.error('Usage: npm run offload:attach <PR_NUMBER> [action] [--local]');
|
||||
console.error('Usage: npm run workspace:attach <PR_NUMBER> [action] [--local]');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return 1;
|
||||
}
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
if (!config) {
|
||||
console.error('❌ Deep Review configuration not found.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const { projectId, zone } = config;
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const provider = ProviderFactory.getProvider({ projectId, zone, instanceName: targetVM });
|
||||
|
||||
const sessionName = `offload-${prNumber}-${action}`;
|
||||
const sessionName = `workspace-${prNumber}-${action}`;
|
||||
const containerAttach = `sudo docker exec -it maintainer-worker sh -c ${q(`tmux attach-session -t ${sessionName}`)}`;
|
||||
const finalSSH = provider.getRunCommand(containerAttach, { interactive: true });
|
||||
|
||||
@@ -48,7 +48,7 @@ export async function runAttach(args: string[], env: NodeJS.ProcessEnv = process
|
||||
|
||||
const isWithinGemini = !!env.GEMINI_CLI || !!env.GEMINI_SESSION_ID || !!env.GCLI_SESSION_ID;
|
||||
if (isWithinGemini && !isLocal) {
|
||||
const tempCmdPath = path.join(process.env.TMPDIR || '/tmp', `offload-attach-${prNumber}.sh`);
|
||||
const tempCmdPath = path.join(process.env.TMPDIR || '/tmp', `workspace-attach-${prNumber}.sh`);
|
||||
fs.writeFileSync(tempCmdPath, `#!/bin/bash\n${finalSSH}\nrm "$0"`, { mode: 0o755 });
|
||||
|
||||
const appleScript = `
|
||||
+4
-4
@@ -14,19 +14,19 @@ export async function runChecker(args: string[], env: NodeJS.ProcessEnv = proces
|
||||
return 1;
|
||||
}
|
||||
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return 1;
|
||||
}
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
if (!config) {
|
||||
console.error('❌ Deep Review configuration not found.');
|
||||
return 1;
|
||||
}
|
||||
const { projectId, zone, remoteWorkDir } = config;
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const provider = ProviderFactory.getProvider({ projectId, zone, instanceName: targetVM });
|
||||
|
||||
console.log(`🔍 Checking remote status for PR #${prNumber} on ${targetVM}...`);
|
||||
+8
-8
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Universal Offload Cleanup (Local)
|
||||
* Universal Workspace Cleanup (Local)
|
||||
*
|
||||
* Surgical or full cleanup of sessions and worktrees on the GCE worker.
|
||||
* Refactored to use WorkerProvider for container compatibility.
|
||||
@@ -27,25 +27,25 @@ export async function runCleanup(args: string[], env: NodeJS.ProcessEnv = proces
|
||||
const prNumber = args[0];
|
||||
const action = args[1];
|
||||
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
if (!config) {
|
||||
console.error('❌ Offload configuration not found.');
|
||||
console.error('❌ Workspace configuration not found.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const { projectId, zone } = config;
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const provider = ProviderFactory.getProvider({ projectId, zone, instanceName: targetVM });
|
||||
|
||||
if (prNumber && action) {
|
||||
const sessionName = `offload-${prNumber}-${action}`;
|
||||
const sessionName = `workspace-${prNumber}-${action}`;
|
||||
const worktreePath = `/home/node/dev/worktrees/${sessionName}`;
|
||||
|
||||
console.log(`🧹 Surgically removing session and worktree for ${prNumber}-${action}...`);
|
||||
@@ -78,7 +78,7 @@ export async function runCleanup(args: string[], env: NodeJS.ProcessEnv = proces
|
||||
if (shouldWipe) {
|
||||
console.log(`🔥 Wiping /home/node/dev/main...`);
|
||||
await provider.exec(`rm -rf /home/node/dev/main && mkdir -p /home/node/dev/main`, { wrapContainer: 'maintainer-worker' });
|
||||
console.log('✅ Remote hub wiped. You will need to run npm run offload:setup again.');
|
||||
console.log('✅ Remote hub wiped. You will need to run npm run workspace:setup again.');
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
+1
-1
@@ -13,7 +13,7 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const prNumber = process.argv[2];
|
||||
const branchName = process.argv[3];
|
||||
const policyPath = process.argv[4];
|
||||
const ISOLATED_CONFIG = process.env.GEMINI_CLI_HOME || path.join(process.env.HOME || '', '.offload/gemini-cli-config');
|
||||
const ISOLATED_CONFIG = process.env.GEMINI_CLI_HOME || path.join(process.env.HOME || '', '.workspace/gemini-cli-config');
|
||||
|
||||
async function main() {
|
||||
if (!prNumber || !branchName || !policyPath) {
|
||||
+14
-14
@@ -1,7 +1,7 @@
|
||||
/**
|
||||
* Offload Fleet Manager
|
||||
* Workspace Fleet Manager
|
||||
*
|
||||
* Manages dynamic GCP workers for offloading tasks.
|
||||
* Manages dynamic GCP workers for workspaces tasks.
|
||||
*/
|
||||
import { spawnSync } from 'child_process';
|
||||
import path from 'path';
|
||||
@@ -13,15 +13,15 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = path.resolve(__dirname, '../../../..');
|
||||
|
||||
const USER = process.env.USER || 'mattkorwel';
|
||||
const INSTANCE_PREFIX = `gcli-offload-${USER}`;
|
||||
const INSTANCE_PREFIX = `gcli-workspace-${USER}`;
|
||||
const DEFAULT_ZONE = 'us-west1-a';
|
||||
|
||||
function getProjectId(): string {
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (fs.existsSync(settingsPath)) {
|
||||
try {
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
return settings.deepReview?.projectId;
|
||||
return settings.workspace?.projectId;
|
||||
} catch (e) {}
|
||||
}
|
||||
return process.env.GOOGLE_CLOUD_PROJECT || '';
|
||||
@@ -30,11 +30,11 @@ function getProjectId(): string {
|
||||
async function listWorkers() {
|
||||
const projectId = getProjectId();
|
||||
if (!projectId) {
|
||||
console.error('❌ Project ID not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Project ID not found. Run "npm run workspace:setup" first.');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`🔍 Listing Offload Workers for ${USER} in ${projectId}...`);
|
||||
console.log(`🔍 Listing Workspace Workers for ${USER} in ${projectId}...`);
|
||||
|
||||
spawnSync('gcloud', [
|
||||
'compute', 'instances', 'list',
|
||||
@@ -47,7 +47,7 @@ async function listWorkers() {
|
||||
async function provisionWorker() {
|
||||
const projectId = getProjectId();
|
||||
if (!projectId) {
|
||||
console.error('❌ Project ID not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Project ID not found. Run "npm run workspace:setup" first.');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -74,18 +74,18 @@ async function stopWorker() {
|
||||
instanceName: INSTANCE_PREFIX
|
||||
});
|
||||
|
||||
console.log(`🛑 Stopping offload worker: ${INSTANCE_PREFIX}...`);
|
||||
console.log(`🛑 Stopping workspace worker: ${INSTANCE_PREFIX}...`);
|
||||
await provider.stop();
|
||||
}
|
||||
|
||||
async function remoteStatus() {
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return;
|
||||
}
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
|
||||
const provider = ProviderFactory.getProvider({
|
||||
projectId: config?.projectId || getProjectId(),
|
||||
@@ -94,14 +94,14 @@ async function remoteStatus() {
|
||||
});
|
||||
|
||||
console.log(`📡 Fetching remote status from ${INSTANCE_PREFIX}...`);
|
||||
await provider.exec('tsx .offload/scripts/status.ts');
|
||||
await provider.exec('tsx .workspaces/scripts/status.ts');
|
||||
}
|
||||
|
||||
async function rebuildWorker() {
|
||||
const projectId = getProjectId();
|
||||
console.log(`🔥 Rebuilding worker ${INSTANCE_PREFIX}...`);
|
||||
|
||||
const knownHostsPath = path.join(REPO_ROOT, '.gemini/offload_known_hosts');
|
||||
const knownHostsPath = path.join(REPO_ROOT, '.gemini/workspaces_known_hosts');
|
||||
if (fs.existsSync(knownHostsPath)) {
|
||||
console.log(` - Clearing isolated known_hosts...`);
|
||||
fs.unlinkSync(knownHostsPath);
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Offload Log Tailer (Local)
|
||||
* Workspace Log Tailer (Local)
|
||||
*
|
||||
* Tails the latest remote logs for a specific job.
|
||||
*/
|
||||
@@ -16,17 +16,17 @@ export async function runLogs(args: string[]) {
|
||||
const action = args[1] || 'review';
|
||||
|
||||
if (!prNumber) {
|
||||
console.error('Usage: npm run offload:logs <PR_NUMBER> [action]');
|
||||
console.error('Usage: npm run workspace:logs <PR_NUMBER> [action]');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/settings.json');
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.maintainer?.deepReview;
|
||||
const config = settings.maintainer?.workspace;
|
||||
const { remoteHost, remoteHome } = config;
|
||||
const sshConfigPath = path.join(REPO_ROOT, '.gemini/offload_ssh_config');
|
||||
const sshConfigPath = path.join(REPO_ROOT, '.gemini/workspace_ssh_config');
|
||||
|
||||
const jobDir = `${remoteHome}/dev/worktrees/offload-${prNumber}-${action}`;
|
||||
const jobDir = `${remoteHome}/dev/worktrees/workspace-${prNumber}-${action}`;
|
||||
const logDir = `${jobDir}/.gemini/logs`;
|
||||
|
||||
console.log(`📋 Tailing latest logs for job ${prNumber}-${action}...`);
|
||||
+10
-10
@@ -14,25 +14,25 @@ export async function runOrchestrator(args: string[], env: NodeJS.ProcessEnv = p
|
||||
const action = args[1] || 'review';
|
||||
|
||||
if (!prNumber) {
|
||||
console.error('Usage: npm run offload <PR_NUMBER> [action]');
|
||||
console.error('Usage: npm run workspace <PR_NUMBER> [action]');
|
||||
return 1;
|
||||
}
|
||||
|
||||
// 1. Load Settings
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return 1;
|
||||
}
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
if (!config) {
|
||||
console.error('❌ Deep Review configuration not found.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const { projectId, zone } = config;
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const provider = ProviderFactory.getProvider({ projectId, zone, instanceName: targetVM });
|
||||
|
||||
// 2. Wake Worker & Verify Container
|
||||
@@ -41,9 +41,9 @@ export async function runOrchestrator(args: string[], env: NodeJS.ProcessEnv = p
|
||||
// Use Absolute Container Paths
|
||||
const containerHome = '/home/node';
|
||||
const remoteWorkDir = `${containerHome}/dev/main`;
|
||||
const remotePolicyPath = `${containerHome}/.gemini/policies/offload-policy.toml`;
|
||||
const persistentScripts = `${containerHome}/.offload/scripts`;
|
||||
const sessionName = `offload-${prNumber}-${action}`;
|
||||
const remotePolicyPath = `${containerHome}/.gemini/policies/workspace-policy.toml`;
|
||||
const persistentScripts = `${containerHome}/.workspace/scripts`;
|
||||
const sessionName = `workspace-${prNumber}-${action}`;
|
||||
const remoteWorktreeDir = `${containerHome}/dev/worktrees/${sessionName}`;
|
||||
|
||||
// 3. Remote Context Setup
|
||||
@@ -71,7 +71,7 @@ export async function runOrchestrator(args: string[], env: NodeJS.ProcessEnv = p
|
||||
|
||||
// 4. Execution Logic
|
||||
const remoteWorker = `tsx ${persistentScripts}/entrypoint.ts ${prNumber} . ${remotePolicyPath} ${action}`;
|
||||
const remoteTmuxCmd = `tmux attach-session -t ${sessionName} 2>/dev/null || tmux new-session -s ${sessionName} -n 'offload' 'cd ${remoteWorktreeDir} && ${remoteWorker}; exec $SHELL'`;
|
||||
const remoteTmuxCmd = `tmux attach-session -t ${sessionName} 2>/dev/null || tmux new-session -s ${sessionName} -n 'workspace' 'cd ${remoteWorktreeDir} && ${remoteWorker}; exec $SHELL'`;
|
||||
const containerWrap = `sudo docker exec -it maintainer-worker sh -c ${q(remoteTmuxCmd)}`;
|
||||
|
||||
const finalSSH = provider.getRunCommand(containerWrap, { interactive: true });
|
||||
@@ -80,7 +80,7 @@ export async function runOrchestrator(args: string[], env: NodeJS.ProcessEnv = p
|
||||
const forceMainTerminal = true; // For debugging
|
||||
|
||||
if (!forceMainTerminal && isWithinGemini && env.TERM_PROGRAM === 'iTerm.app') {
|
||||
const tempCmdPath = path.join(process.env.TMPDIR || '/tmp', `offload-ssh-${prNumber}.sh`);
|
||||
const tempCmdPath = path.join(process.env.TMPDIR || '/tmp', `workspace-ssh-${prNumber}.sh`);
|
||||
fs.writeFileSync(tempCmdPath, `#!/bin/bash\n${finalSSH}\nrm "$0"`, { mode: 0o755 });
|
||||
const appleScript = `on run argv\ntell application "iTerm"\ntell current window\nset newTab to (create tab with default profile)\ntell current session of newTab\nwrite text (item 1 of argv) & return\nend tell\nend tell\nactivate\nend tell\nend run`;
|
||||
spawnSync('osascript', ['-', tempCmdPath], { input: appleScript });
|
||||
+1
-1
@@ -2,7 +2,7 @@ import { spawnSync } from 'child_process';
|
||||
import path from 'path';
|
||||
|
||||
export async function runFixPlaybook(prNumber: string, targetDir: string, policyPath: string, geminiBin: string) {
|
||||
console.log(`🚀 Offload | FIX | PR #${prNumber}`);
|
||||
console.log(`🚀 Workspace | FIX | PR #${prNumber}`);
|
||||
console.log('Switching to agentic fix loop inside Gemini CLI...');
|
||||
|
||||
// Use the nightly gemini binary to activate the fix-pr skill and iterate
|
||||
+1
-1
@@ -8,7 +8,7 @@ import { spawnSync } from 'child_process';
|
||||
import fs from 'fs';
|
||||
|
||||
export async function runImplementPlaybook(issueNumber: string, workDir: string, policyPath: string, geminiBin: string) {
|
||||
console.log(`🚀 Offload | IMPLEMENT (Supervisor Loop) | Issue #${issueNumber}`);
|
||||
console.log(`🚀 Workspace | IMPLEMENT (Supervisor Loop) | Issue #${issueNumber}`);
|
||||
|
||||
const ghView = spawnSync('gh', ['issue', 'view', issueNumber, '--json', 'title,body', '-q', '{title:.title,body:.body}'], { shell: true });
|
||||
const meta = JSON.parse(ghView.stdout.toString());
|
||||
+2
-2
@@ -3,8 +3,8 @@ import path from 'path';
|
||||
|
||||
export async function runReadyPlaybook(prNumber: string, targetDir: string, policyPath: string, geminiBin: string) {
|
||||
const runner = new TaskRunner(
|
||||
path.join(targetDir, `.gemini/logs/offload-${prNumber}`),
|
||||
`🚀 Offload | READY | PR #${prNumber}`
|
||||
path.join(targetDir, `.gemini/logs/workspace-${prNumber}`),
|
||||
`🚀 Workspace | READY | PR #${prNumber}`
|
||||
);
|
||||
|
||||
runner.register([
|
||||
+3
-3
@@ -3,14 +3,14 @@ import path from 'path';
|
||||
|
||||
export async function runReviewPlaybook(prNumber: string, targetDir: string, policyPath: string, geminiBin: string) {
|
||||
const runner = new TaskRunner(
|
||||
path.join(targetDir, `.gemini/logs/offload-${prNumber}`),
|
||||
`🚀 Offload | REVIEW | PR #${prNumber}`
|
||||
path.join(targetDir, `.gemini/logs/workspace-${prNumber}`),
|
||||
`🚀 Workspace | REVIEW | PR #${prNumber}`
|
||||
);
|
||||
|
||||
runner.register([
|
||||
{ id: 'build', name: 'Fast Build', cmd: `cd ${targetDir} && npm ci && npm run build` },
|
||||
{ id: 'ci', name: 'CI Checks', cmd: `gh pr checks ${prNumber}` },
|
||||
{ id: 'review', name: 'Offloaded Review', cmd: `${geminiBin} --policy ${policyPath} --cwd ${targetDir} -p "Please activate the 'review-pr' skill and use it to conduct a behavioral review of PR #${prNumber}."` }
|
||||
{ id: 'review', name: 'Workspaceed Review', cmd: `${geminiBin} --policy ${policyPath} --cwd ${targetDir} -p "Please activate the 'review-pr' skill and use it to conduct a behavioral review of PR #${prNumber}."` }
|
||||
]);
|
||||
|
||||
return runner.run();
|
||||
+11
-11
@@ -5,22 +5,22 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* WorkerProvider interface defines the contract for different remote
|
||||
* WorkspaceProvider interface defines the contract for different remote
|
||||
* execution environments (GCE, Workstations, etc.).
|
||||
*/
|
||||
export interface WorkerProvider {
|
||||
export interface WorkspaceProvider {
|
||||
/**
|
||||
* Provisions the underlying infrastructure.
|
||||
*/
|
||||
provision(): Promise<number>;
|
||||
|
||||
/**
|
||||
* Ensures the worker is running and accessible.
|
||||
* Ensures the workspace is running and accessible.
|
||||
*/
|
||||
ensureReady(): Promise<number>;
|
||||
|
||||
/**
|
||||
* Performs the initial setup of the worker (SSH, scripts, auth).
|
||||
* Performs the initial setup of the workspace (SSH, scripts, auth).
|
||||
*/
|
||||
setup(options: SetupOptions): Promise<number>;
|
||||
|
||||
@@ -30,27 +30,27 @@ export interface WorkerProvider {
|
||||
getRunCommand(command: string, options?: ExecOptions): string;
|
||||
|
||||
/**
|
||||
* Executes a command on the worker.
|
||||
* Executes a command on the workspace.
|
||||
*/
|
||||
exec(command: string, options?: ExecOptions): Promise<number>;
|
||||
|
||||
/**
|
||||
* Executes a command on the worker and returns the output.
|
||||
* Executes a command on the workspace and returns the output.
|
||||
*/
|
||||
getExecOutput(command: string, options?: ExecOptions): Promise<{ status: number; stdout: string; stderr: string }>;
|
||||
|
||||
/**
|
||||
* Synchronizes local files to the worker.
|
||||
* Synchronizes local files to the workspace.
|
||||
*/
|
||||
sync(localPath: string, remotePath: string, options?: SyncOptions): Promise<number>;
|
||||
|
||||
/**
|
||||
* Returns the status of the worker.
|
||||
* Returns the status of the workspace.
|
||||
*/
|
||||
getStatus(): Promise<WorkerStatus>;
|
||||
getStatus(): Promise<WorkspaceStatus>;
|
||||
|
||||
/**
|
||||
* Stops the worker to save costs.
|
||||
* Stops the workspace to save costs.
|
||||
*/
|
||||
stop(): Promise<number>;
|
||||
}
|
||||
@@ -73,7 +73,7 @@ export interface SyncOptions {
|
||||
exclude?: string[];
|
||||
}
|
||||
|
||||
export interface WorkerStatus {
|
||||
export interface WorkspaceStatus {
|
||||
name: string;
|
||||
status: string;
|
||||
internalIp?: string;
|
||||
+8
-8
@@ -8,10 +8,10 @@ import { spawnSync } from 'child_process';
|
||||
import path from 'path';
|
||||
import fs from 'fs';
|
||||
import os from 'os';
|
||||
import { WorkerProvider, SetupOptions, ExecOptions, SyncOptions, WorkerStatus } from './BaseProvider.ts';
|
||||
import { WorkspaceProvider, SetupOptions, ExecOptions, SyncOptions, WorkspaceStatus } from './BaseProvider.ts';
|
||||
import { GceConnectionManager } from './GceConnectionManager.ts';
|
||||
|
||||
export class GceCosProvider implements WorkerProvider {
|
||||
export class GceCosProvider implements WorkspaceProvider {
|
||||
private projectId: string;
|
||||
private zone: string;
|
||||
private instanceName: string;
|
||||
@@ -24,10 +24,10 @@ export class GceCosProvider implements WorkerProvider {
|
||||
this.projectId = projectId;
|
||||
this.zone = zone;
|
||||
this.instanceName = instanceName;
|
||||
const offloadDir = path.join(repoRoot, '.gemini/offload');
|
||||
if (!fs.existsSync(offloadDir)) fs.mkdirSync(offloadDir, { recursive: true });
|
||||
this.sshConfigPath = path.join(offloadDir, 'ssh_config');
|
||||
this.knownHostsPath = path.join(offloadDir, 'known_hosts');
|
||||
const workspacesDir = path.join(repoRoot, '.gemini/workspaces');
|
||||
if (!fs.existsSync(workspacesDir)) fs.mkdirSync(workspacesDir, { recursive: true });
|
||||
this.sshConfigPath = path.join(workspacesDir, 'ssh_config');
|
||||
this.knownHostsPath = path.join(workspacesDir, 'known_hosts');
|
||||
this.conn = new GceConnectionManager(projectId, zone, instanceName);
|
||||
}
|
||||
|
||||
@@ -76,7 +76,7 @@ export class GceCosProvider implements WorkerProvider {
|
||||
# Run if not already exists
|
||||
if ! docker ps -a | grep -q "maintainer-worker"; then
|
||||
docker run -d --name maintainer-worker --restart always \\
|
||||
-v ~/.offload:/home/node/.offload:rw \\
|
||||
-v ~/.workspace:/home/node/.workspace:rw \\
|
||||
-v ~/dev:/home/node/dev:rw \\
|
||||
-v ~/.gemini:/home/node/.gemini:rw \\
|
||||
${imageUri} /bin/bash -c "while true; do sleep 1000; done"
|
||||
@@ -149,7 +149,7 @@ export class GceCosProvider implements WorkerProvider {
|
||||
if (needsUpdate) {
|
||||
console.log(' ⚠️ Container missing or stale. Attempting refresh...');
|
||||
const imageUri = 'us-docker.pkg.dev/gemini-code-dev/gemini-cli/maintainer:latest';
|
||||
const recoverCmd = `sudo docker pull ${imageUri} && (sudo docker rm -f maintainer-worker || true) && sudo docker run -d --name maintainer-worker --restart always -v ~/.offload:/home/node/.offload:rw -v ~/dev:/home/node/dev:rw -v ~/.gemini:/home/node/.gemini:rw ${imageUri} /bin/bash -c "while true; do sleep 1000; done"`;
|
||||
const recoverCmd = `sudo docker pull ${imageUri} && (sudo docker rm -f maintainer-worker || true) && sudo docker run -d --name maintainer-worker --restart always -v ~/.workspace:/home/node/.workspace:rw -v ~/dev:/home/node/dev:rw -v ~/.gemini:/home/node/.gemini:rw ${imageUri} /bin/bash -c "while true; do sleep 1000; done"`;
|
||||
const recoverRes = await this.exec(recoverCmd);
|
||||
if (recoverRes !== 0) {
|
||||
console.error(' ❌ Critical: Failed to refresh maintainer container.');
|
||||
+2
-2
@@ -5,7 +5,7 @@
|
||||
*/
|
||||
|
||||
import { GceCosProvider } from './GceCosProvider.ts';
|
||||
import { WorkerProvider } from './BaseProvider.ts';
|
||||
import { WorkspaceProvider } from './BaseProvider.ts';
|
||||
import path from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
@@ -13,7 +13,7 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = path.resolve(__dirname, '../../../../..');
|
||||
|
||||
export class ProviderFactory {
|
||||
static getProvider(config: { projectId: string; zone: string; instanceName: string }): WorkerProvider {
|
||||
static getProvider(config: { projectId: string; zone: string; instanceName: string }): WorkspaceProvider {
|
||||
// Currently we only have GceCosProvider, but this is where we'd branch
|
||||
return new GceCosProvider(config.projectId, config.zone, config.instanceName, REPO_ROOT);
|
||||
}
|
||||
+33
-24
@@ -41,9 +41,9 @@ async function confirm(question: string): Promise<boolean> {
|
||||
export async function runSetup(env: NodeJS.ProcessEnv = process.env) {
|
||||
console.log(`
|
||||
================================================================================
|
||||
🚀 GEMINI CLI: HIGH-PERFORMANCE OFFLOAD SYSTEM
|
||||
🚀 GEMINI WORKSPACES: HIGH-PERFORMANCE REMOTE DEVELOPMENT
|
||||
================================================================================
|
||||
The offload system allows you to delegate heavy tasks (PR reviews, agentic fixes,
|
||||
Workspaces allow you to delegate heavy tasks (PR reviews, agentic fixes,
|
||||
and full builds) to a dedicated, high-performance GCP worker.
|
||||
================================================================================
|
||||
`);
|
||||
@@ -52,25 +52,31 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
console.log('--------------------------------------------------------------------------------');
|
||||
|
||||
// 1. Project Identity
|
||||
const defaultProject = env.GOOGLE_CLOUD_PROJECT || env.OFFLOAD_PROJECT || '';
|
||||
const defaultProject = env.GOOGLE_CLOUD_PROJECT || env.WORKSPACE_PROJECT || '';
|
||||
const projectId = await prompt('GCP Project ID', defaultProject,
|
||||
'The GCP Project where your offload worker will live. Your personal project is recommended.');
|
||||
'The GCP Project where your workspace worker will live. Your personal project is recommended.');
|
||||
|
||||
if (!projectId) {
|
||||
console.error('❌ Project ID is required. Set GOOGLE_CLOUD_PROJECT or enter it manually.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const zone = await prompt('Compute Zone', env.OFFLOAD_ZONE || 'us-west1-a',
|
||||
const zone = await prompt('Compute Zone', env.WORKSPACE_ZONE || 'us-west1-a',
|
||||
'The physical location of your worker. us-west1-a is the team default.');
|
||||
|
||||
const terminalTarget = await prompt('Terminal UI Target (tab or window)', env.OFFLOAD_TERM_TARGET || 'tab',
|
||||
const terminalTarget = await prompt('Terminal UI Target (tab or window)', env.WORKSPACE_TERM_TARGET || 'tab',
|
||||
'When a job starts, should it open in a new iTerm2 tab or a completely new window?');
|
||||
|
||||
// 2. Repository Discovery
|
||||
// 2. Repository Discovery (Dynamic)
|
||||
console.log('\n🔍 Detecting repository origins...');
|
||||
|
||||
// Get remote URL
|
||||
const remoteUrlRes = spawnSync('git', ['remote', 'get-url', 'origin'], { stdio: 'pipe' });
|
||||
const remoteUrl = remoteUrlRes.stdout.toString().trim();
|
||||
|
||||
// Use gh to get full details
|
||||
const repoInfoRes = spawnSync('gh', ['repo', 'view', '--json', 'nameWithOwner,parent,isFork'], { stdio: 'pipe' });
|
||||
let upstreamRepo = 'google-gemini/gemini-cli';
|
||||
let upstreamRepo = 'google-gemini/gemini-cli'; // Fallback
|
||||
let userFork = upstreamRepo;
|
||||
|
||||
if (repoInfoRes.status === 0) {
|
||||
@@ -79,25 +85,28 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
if (repoInfo.isFork && repoInfo.parent) {
|
||||
upstreamRepo = repoInfo.parent.nameWithOwner;
|
||||
userFork = repoInfo.nameWithOwner;
|
||||
console.log(` ✅ Detected Fork: ${userFork} (Upstream: ${upstreamRepo})`);
|
||||
} else {
|
||||
console.log(` ✅ Working on Upstream: ${upstreamRepo}`);
|
||||
upstreamRepo = repoInfo.nameWithOwner;
|
||||
userFork = repoInfo.nameWithOwner;
|
||||
}
|
||||
} catch (e) {}
|
||||
}
|
||||
|
||||
console.log(` ✅ Target Repo: ${userFork}`);
|
||||
console.log(` ✅ Upstream: ${upstreamRepo}`);
|
||||
|
||||
// 3. Security & Auth
|
||||
let githubToken = env.OFFLOAD_GH_TOKEN || '';
|
||||
let githubToken = env.WORKSPACE_GH_TOKEN || '';
|
||||
if (!githubToken) {
|
||||
const shouldGenToken = await confirm('\nGenerate a scoped GitHub token for the remote agent? (Highly Recommended)');
|
||||
if (shouldGenToken) {
|
||||
const baseUrl = 'https://github.com/settings/personal-access-tokens/new';
|
||||
const name = `Offload-${env.USER}`;
|
||||
const name = `Workspace-${env.USER}`;
|
||||
const repoParams = userFork !== upstreamRepo
|
||||
? `&repositories[]=${encodeURIComponent(upstreamRepo)}&repositories[]=${encodeURIComponent(userFork)}`
|
||||
: `&repositories[]=${encodeURIComponent(upstreamRepo)}`;
|
||||
|
||||
const magicLink = `${baseUrl}?name=${encodeURIComponent(name)}&description=Gemini+CLI+Offload+Worker${repoParams}&contents=write&pull_requests=write&metadata=read`;
|
||||
const magicLink = `${baseUrl}?name=${encodeURIComponent(name)}&description=Gemini+Workspaces+Worker${repoParams}&contents=write&pull_requests=write&metadata=read`;
|
||||
const terminalLink = `\u001b]8;;${magicLink}\u0007${magicLink}\u001b]8;;\u0007`;
|
||||
|
||||
console.log(`\n🔐 ACTION REQUIRED: Create a token with "Read/Write" access to contents & PRs:`);
|
||||
@@ -106,16 +115,16 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
githubToken = await prompt('Paste Scoped Token', '');
|
||||
}
|
||||
} else {
|
||||
console.log(' ✅ Using GitHub token from environment (OFFLOAD_GH_TOKEN).');
|
||||
console.log(' ✅ Using GitHub token from environment (WORKSPACE_GH_TOKEN).');
|
||||
}
|
||||
|
||||
// 4. Save Confirmed State
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(path.dirname(settingsPath))) fs.mkdirSync(path.dirname(settingsPath), { recursive: true });
|
||||
|
||||
const settings = {
|
||||
deepReview: {
|
||||
workspace: {
|
||||
projectId, zone, terminalTarget,
|
||||
userFork, upstreamRepo,
|
||||
remoteHost: 'gcli-worker',
|
||||
@@ -135,7 +144,7 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
let status = await provider.getStatus();
|
||||
|
||||
if (status.status === 'UNKNOWN' || status.status === 'ERROR') {
|
||||
const shouldProvision = await confirm(`❌ Worker ${targetVM} not found. Provision it now?`);
|
||||
const shouldProvision = await confirm(`❓ Worker ${targetVM} not found. Provision it now?`);
|
||||
if (!shouldProvision) return 1;
|
||||
|
||||
const provisionRes = await provider.provision();
|
||||
@@ -153,19 +162,19 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
const setupRes = await provider.setup({ projectId, zone, dnsSuffix: '.internal.gcpnode.com' });
|
||||
if (setupRes !== 0) return setupRes;
|
||||
|
||||
const persistentScripts = `~/.offload/scripts`;
|
||||
const persistentScripts = `~/.workspaces/scripts`;
|
||||
|
||||
console.log(`\n📦 Synchronizing Logic & Credentials...`);
|
||||
await provider.exec(`mkdir -p ~/dev/main ~/.gemini/policies ~/.offload/scripts`);
|
||||
await provider.sync('.gemini/skills/offload/scripts/', `${persistentScripts}/`, { delete: true });
|
||||
await provider.sync('.gemini/skills/offload/policy.toml', `~/.gemini/policies/offload-policy.toml`);
|
||||
await provider.exec(`mkdir -p ~/dev/main ~/.gemini/policies ~/.workspaces/scripts`);
|
||||
await provider.sync('.gemini/skills/workspaces/scripts/', `${persistentScripts}/`, { delete: true });
|
||||
await provider.sync('.gemini/skills/workspaces/policy.toml', `~/.gemini/policies/workspace-policy.toml`);
|
||||
|
||||
if (fs.existsSync(path.join(env.HOME || '', '.gemini/google_accounts.json'))) {
|
||||
await provider.sync(path.join(env.HOME || '', '.gemini/google_accounts.json'), `~/.gemini/google_accounts.json`);
|
||||
}
|
||||
|
||||
if (githubToken) {
|
||||
await provider.exec(`mkdir -p ~/.offload && echo ${githubToken} > ~/.offload/.gh_token && chmod 600 ~/.offload/.gh_token`);
|
||||
await provider.exec(`mkdir -p ~/.workspaces && echo ${githubToken} > ~/.workspaces/.gh_token && chmod 600 ~/.workspaces/.gh_token`);
|
||||
}
|
||||
|
||||
// Final Repo Sync
|
||||
@@ -174,7 +183,7 @@ and full builds) to a dedicated, high-performance GCP worker.
|
||||
const cloneCmd = `rm -rf ~/dev/main && git clone --filter=blob:none ${repoUrl} ~/dev/main && cd ~/dev/main && git remote add upstream https://github.com/${upstreamRepo}.git && git fetch upstream`;
|
||||
await provider.exec(cloneCmd);
|
||||
|
||||
console.log('\n✨ ALL SYSTEMS GO! Your offload environment is ready.');
|
||||
console.log('\n✨ ALL SYSTEMS GO! Your Gemini Workspace is ready.');
|
||||
return 0;
|
||||
}
|
||||
|
||||
+8
-8
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Offload Status Inspector (Local)
|
||||
* Workspace Status Inspector (Local)
|
||||
*
|
||||
* Orchestrates remote status retrieval via the WorkerProvider.
|
||||
*/
|
||||
@@ -12,23 +12,23 @@ const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = path.resolve(__dirname, '../../../..');
|
||||
|
||||
async function runStatus(env: NodeJS.ProcessEnv = process.env) {
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/offload/settings.json');
|
||||
const settingsPath = path.join(REPO_ROOT, '.gemini/workspaces/settings.json');
|
||||
if (!fs.existsSync(settingsPath)) {
|
||||
console.error('❌ Settings not found. Run "npm run offload:setup" first.');
|
||||
console.error('❌ Settings not found. Run "npm run workspace:setup" first.');
|
||||
return 1;
|
||||
}
|
||||
const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf8'));
|
||||
const config = settings.deepReview;
|
||||
const config = settings.workspace;
|
||||
if (!config) {
|
||||
console.error('❌ Deep Review configuration not found.');
|
||||
return 1;
|
||||
}
|
||||
|
||||
const { projectId, zone } = config;
|
||||
const targetVM = `gcli-offload-${env.USER || 'mattkorwel'}`;
|
||||
const targetVM = `gcli-workspace-${env.USER || 'mattkorwel'}`;
|
||||
const provider = ProviderFactory.getProvider({ projectId, zone, instanceName: targetVM });
|
||||
|
||||
console.log(`\n🛰️ Offload Mission Control: ${targetVM}`);
|
||||
console.log(`\n🛰️ Workspace Mission Control: ${targetVM}`);
|
||||
console.log(`--------------------------------------------------------------------------------`);
|
||||
|
||||
const status = await provider.getStatus();
|
||||
@@ -43,10 +43,10 @@ async function runStatus(env: NodeJS.ProcessEnv = process.env) {
|
||||
if (tmuxRes.status === 0 && tmuxRes.stdout.trim()) {
|
||||
const sessions = tmuxRes.stdout.trim().split('\n');
|
||||
sessions.forEach(s => {
|
||||
if (s.startsWith('offload-')) {
|
||||
if (s.startsWith('workspace-')) {
|
||||
console.log(` ✅ ${s}`);
|
||||
} else {
|
||||
console.log(` 🔹 ${s} (Non-offload)`);
|
||||
console.log(` 🔹 ${s} (Non-workspace)`);
|
||||
}
|
||||
});
|
||||
} else {
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Universal Offload Worker (Remote)
|
||||
* Universal Workspace Worker (Remote)
|
||||
*
|
||||
* Stateful orchestrator for complex development loops.
|
||||
*/
|
||||
+2
-2
@@ -11,10 +11,10 @@ vi.mock('fs');
|
||||
vi.mock('readline');
|
||||
vi.mock('../scripts/providers/ProviderFactory.ts');
|
||||
|
||||
describe('Offload Tooling Matrix', () => {
|
||||
describe('Workspace Tooling Matrix', () => {
|
||||
const mockSettings = {
|
||||
maintainer: {
|
||||
deepReview: {
|
||||
workspace: {
|
||||
projectId: 'test-project',
|
||||
zone: 'us-west1-a',
|
||||
remoteWorkDir: '/home/node/dev/main'
|
||||
+2
-2
@@ -11,10 +11,10 @@ vi.mock('fs');
|
||||
vi.mock('readline');
|
||||
vi.mock('../scripts/providers/ProviderFactory.ts');
|
||||
|
||||
describe('Offload Orchestration (Refactored)', () => {
|
||||
describe('Workspace Orchestration (Refactored)', () => {
|
||||
const mockSettings = {
|
||||
maintainer: {
|
||||
deepReview: {
|
||||
workspace: {
|
||||
projectId: 'test-project',
|
||||
zone: 'us-west1-a',
|
||||
remoteWorkDir: '/home/node/dev/main'
|
||||
+24
-48
@@ -1,88 +1,66 @@
|
||||
# Maintainer Onboarding: High-Performance Offload System 🚀
|
||||
# Gemini Workspaces: High-Performance Remote Development 🚀
|
||||
|
||||
Welcome to the Gemini CLI maintainer team! This guide will help you set up your
|
||||
remote development environment, which offloads heavy tasks (reviews, fixes,
|
||||
preflight) to a dedicated GCP worker.
|
||||
Welcome to the Gemini Workspaces platform! This guide will help you set up your
|
||||
remote development environment, which allows you to offload heavy tasks (reviews, fixes,
|
||||
preflight) to a dedicated, high-performance GCP worker.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Google Cloud Access**: You will need a Google Cloud Project with billing enabled. You can use a shared team project or, **ideally, your own personal GCP project** for maximum isolation.
|
||||
2. **GCloud CLI**: Authenticated locally (`gcloud auth login`).
|
||||
3. **GitHub CLI**: Authenticated locally (`gh auth login`).
|
||||
4. **IAP Permissions**: Ensure you have the `IAP-secured Tunnel User` role on your chosen project.
|
||||
5. **Corporate Identity**: Run `gcert` (or your internal equivalent) recently to ensure SSH certificates are valid.
|
||||
4. **Corporate Identity**: Run `gcert` (or your internal equivalent) recently to ensure SSH certificates are valid.
|
||||
|
||||
## Architecture: Hybrid VM + Container 🏗️
|
||||
## Architecture: Persistent Cloud Workstations 🏗️
|
||||
|
||||
The offload system uses a **Worker Provider** architecture to abstract the underlying infrastructure:
|
||||
The system uses a **Workspace Provider** architecture to abstract the underlying infrastructure:
|
||||
|
||||
1. **GCE VM (Host)**: A high-performance machine running **Container-Optimized OS (COS)**.
|
||||
2. **maintainer-worker (Container)**: A persistent Docker container acting as your remote workstation.
|
||||
3. **Resilient Connectivity**: A dual-path strategy that uses **Fast-Path SSH** by default and automatically falls back to **IAP Tunneling** if direct access is blocked.
|
||||
3. **Resilient Connectivity**: A verified corporate routing path using `nic0` and `.internal.gcpnode.com` for direct, high-speed access.
|
||||
|
||||
---
|
||||
|
||||
## Setup Workflow
|
||||
|
||||
### 1. Initial Configuration (Discovery)
|
||||
### 1. The Turn-Key Setup
|
||||
|
||||
This interactive script configures your local environment to recognize the remote worker.
|
||||
The entire environment can be initialized with a single command:
|
||||
|
||||
```bash
|
||||
npm run offload:setup
|
||||
npm run workspace:setup
|
||||
```
|
||||
|
||||
* **What it does**: Generates `.gemini/offload_ssh_config`, verifies project access, and establishes the initial identity handshake.
|
||||
* **Connectivity**: If direct internal SSH fails, it will attempt to verify access via an IAP tunnel.
|
||||
|
||||
### 2. Provisioning Your Worker (Infrastructure)
|
||||
|
||||
Spin up your dedicated, high-performance VM. If it already exists, this command will verify its health.
|
||||
|
||||
```bash
|
||||
npm run offload:fleet provision
|
||||
```
|
||||
|
||||
* **Specs**: n2-standard-8, 200GB PD-Balanced disk.
|
||||
* **Self-Healing**: It uses a COS startup script to ensure the `maintainer-worker` container is always running.
|
||||
|
||||
### 3. Remote Initialization
|
||||
|
||||
Once provisioned, return to the setup script to finalize the remote environment.
|
||||
|
||||
```bash
|
||||
npm run offload:setup
|
||||
```
|
||||
|
||||
* **Auth Sync**: Pushes your `google_accounts.json` to the worker.
|
||||
* **Scoped Token**: Generates a magic link for a GitHub PAT and stores it securely on the worker.
|
||||
* **Repo Clone**: Performs a filtered (shallow) clone of the repository onto the remote disk.
|
||||
This interactive script will:
|
||||
- **Phase 1: Configuration**: Auto-detect your repository origins, ask for your GCP project, and guide you through creating a secure GitHub token.
|
||||
- **Phase 2: Infrastructure**: Automatically provision the "Magic" corporate network (VPC, Subnets, Firewalls) and the high-performance VM.
|
||||
- **Phase 3: Initialization**: Synchronize your credentials and clone your repository into a persistent remote volume.
|
||||
|
||||
---
|
||||
|
||||
## Daily Workflow
|
||||
|
||||
### Running an Offloaded Job
|
||||
### Running a Workspace Job
|
||||
|
||||
To perform a deep behavioral review or an agentic fix on your remote worker:
|
||||
|
||||
```bash
|
||||
# For a review
|
||||
npm run offload <PR_NUMBER> review
|
||||
npm run workspace <PR_NUMBER> review
|
||||
|
||||
# For an automated fix
|
||||
npm run offload <PR_NUMBER> implement
|
||||
npm run workspace <PR_NUMBER> fix
|
||||
```
|
||||
|
||||
* **Isolation**: Each job runs in a dedicated **Git Worktree** (`~/dev/worktrees/offload-<id>`).
|
||||
* **Persistence**: Jobs run inside a `tmux` session on the remote host. You can disconnect and reconnect without losing progress.
|
||||
* **Isolation**: Each job runs in a dedicated **Git Worktree**.
|
||||
* **Persistence**: Jobs run inside a `tmux` session. You can disconnect and reconnect without losing progress.
|
||||
|
||||
### Monitoring "Mission Control"
|
||||
|
||||
View the real-time state of your worker and all in-flight jobs:
|
||||
|
||||
```bash
|
||||
npm run offload:status
|
||||
npm run workspace:status
|
||||
```
|
||||
|
||||
### Stopping Your Worker
|
||||
@@ -90,7 +68,7 @@ npm run offload:status
|
||||
To save costs, shut down your worker when finished. The orchestrator will automatically wake it up when you start a new task.
|
||||
|
||||
```bash
|
||||
npm run offload:fleet stop
|
||||
npm run workspace:fleet stop
|
||||
```
|
||||
|
||||
---
|
||||
@@ -98,10 +76,8 @@ npm run offload:fleet stop
|
||||
## Resilience & Troubleshooting
|
||||
|
||||
### "SSH Connection Failed"
|
||||
If the setup or orchestrator reports a connection failure:
|
||||
1. **Check Identity**: Run `gcert` to refresh your SSH credentials.
|
||||
2. **IAP Fallback**: The system should automatically attempt IAP tunneling. If it still fails, verify your GCP project permissions.
|
||||
3. **Waking Up**: If the worker was stopped, the first command may take ~30 seconds to wake the VM.
|
||||
2. **Direct Path**: Ensure you are on the corporate network or VPN if required for `nic0` routing.
|
||||
|
||||
### "Worker Not Found"
|
||||
If `offload:setup` can't find your worker, ensure you have run `npm run offload:fleet provision` at least once in the current project.
|
||||
The `setup` script will automatically offer to provision a worker if it can't find one. Simply follow the prompts.
|
||||
|
||||
+9
-9
@@ -64,15 +64,15 @@
|
||||
"telemetry": "node scripts/telemetry.js",
|
||||
"check:lockfile": "node scripts/check-lockfile.js",
|
||||
"clean": "node scripts/clean.js",
|
||||
"offload": "tsx .gemini/skills/offload/scripts/orchestrator.ts",
|
||||
"offload:setup": "tsx .gemini/skills/offload/scripts/setup.ts",
|
||||
"offload:check": "tsx .gemini/skills/offload/scripts/check.ts",
|
||||
"offload:clean": "tsx .gemini/skills/offload/scripts/clean.ts",
|
||||
"offload:fleet": "tsx .gemini/skills/offload/scripts/fleet.ts",
|
||||
"offload:status": "npm run offload:fleet status",
|
||||
"offload:attach": "tsx .gemini/skills/offload/scripts/attach.ts",
|
||||
"offload:logs": "tsx .gemini/skills/offload/scripts/logs.ts",
|
||||
"offload:remove": "tsx .gemini/skills/offload/scripts/clean.ts",
|
||||
"workspace": "tsx .gemini/skills/workspaces/scripts/orchestrator.ts",
|
||||
"workspace:setup": "tsx .gemini/skills/workspaces/scripts/setup.ts",
|
||||
"workspace:check": "tsx .gemini/skills/workspaces/scripts/check.ts",
|
||||
"workspace:clean": "tsx .gemini/skills/workspaces/scripts/clean.ts",
|
||||
"workspace:fleet": "tsx .gemini/skills/workspaces/scripts/fleet.ts",
|
||||
"workspace:status": "npm run workspace:fleet status",
|
||||
"workspace:attach": "tsx .gemini/skills/workspaces/scripts/attach.ts",
|
||||
"workspace:logs": "tsx .gemini/skills/workspaces/scripts/logs.ts",
|
||||
"workspace:remove": "tsx .gemini/skills/workspaces/scripts/clean.ts",
|
||||
"pre-commit": "node scripts/pre-commit.js"
|
||||
},
|
||||
"overrides": {
|
||||
|
||||
Reference in New Issue
Block a user