diff --git a/docs/architecture/workspaces/container-image.md b/docs/architecture/workspaces/container-image.md index 4c2003994b..3a14eb9afa 100644 --- a/docs/architecture/workspaces/container-image.md +++ b/docs/architecture/workspaces/container-image.md @@ -1,10 +1,14 @@ # Detailed Design: Workspace Container Image ## 1. Introduction -The Workspace Container Image defines the standardized software environment for all remote execution. It is pre-built and optimized for fast startup on GCE instances. + +The Workspace Container Image defines the standardized software environment for +all remote execution. It is pre-built and optimized for fast startup on GCE +instances. ## 2. Dockerfile Specification -The image is maintained in `packages/grid-manager/docker/Dockerfile`. + +The image is maintained in `packages/workspace-manager/docker/Dockerfile`. - **Base:** `node:20-slim` - **Environment:** @@ -18,24 +22,36 @@ The image is maintained in `packages/grid-manager/docker/Dockerfile`. - **User:** `node` (UID 1000) for unprivileged execution. ## 3. Image Contents & Pre-loading + - The `gemini-cli` nightly binary is pre-loaded into `/usr/local/bin/gemini`. - Standard node dependencies (`npm`, `yarn`, `pnpm`) are pre-installed. -- `shpool` is used as the primary process manager to allow terminal detachment and re-attachment. +- `shpool` is used as the primary process manager to allow terminal detachment + and re-attachment. ## 4. Entrypoint Strategy (`entrypoint.sh`) + When the container starts on GCE: -1. **Secret Injection:** Reads the GitHub PAT from a memory-only mount (`/dev/shm/github_token`) and authenticates `gh`. -2. **Settings Restore:** Syncs the user's `~/.gemini/` configuration (aliased from `/home/node/.gemini_volume`). + +1. **Secret Injection:** Reads the GitHub PAT from a memory-only mount + (`/dev/shm/github_token`) and authenticates `gh`. +2. **Settings Restore:** Syncs the user's `~/.gemini/` configuration (aliased + from `/home/node/.gemini_volume`). 3. **Persistence Layer:** Starts `shpool` daemon in the background. -4. **Ready Signal:** Notifies the Workspace Hub that the environment is ready for connection. +4. **Ready Signal:** Notifies the Workspace Hub that the environment is ready + for connection. ## 5. Storage Strategy -- **System:** The container image itself is ephemeral. -- **User Home:** A persistent GCE Disk (PD) is mounted at `/home/node`. This ensures: - - `~/.gemini` settings persist. - - Cloned git repositories persist between workspace restarts. - - `npm install` artifacts (node_modules) persist. + +- **System:** The container image itself is ephemeral. +- **User Home:** A persistent GCE Disk (PD) is mounted at `/home/node`. This + ensures: + - `~/.gemini` settings persist. + - Cloned git repositories persist between workspace restarts. + - `npm install` artifacts (node_modules) persist. ## 6. Build & Release -- The image is automatically built and pushed to the Hub's Artifact Registry on every `main` push or new `nightly` tag. -- The Hub API defaults to using the `latest` or `nightly` tag unless specified otherwise. + +- The image is automatically built and pushed to the Hub's Artifact Registry on + every `main` push or new `nightly` tag. +- The Hub API defaults to using the `latest` or `nightly` tag unless specified + otherwise. diff --git a/package-lock.json b/package-lock.json index 914d66d3ac..31fe7156e1 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1376,6 +1376,10 @@ "resolved": "packages/test-utils", "link": true }, + "node_modules/@google/gemini-cli-workspace-manager": { + "resolved": "packages/workspace-manager", + "link": true + }, "node_modules/@google/genai": { "version": "1.30.0", "resolved": "https://registry.npmjs.org/@google/genai/-/genai-1.30.0.tgz", @@ -4312,6 +4316,13 @@ "integrity": "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q==", "license": "MIT" }, + "node_modules/@types/uuid": { + "version": "10.0.0", + "resolved": "https://registry.npmjs.org/@types/uuid/-/uuid-10.0.0.tgz", + "integrity": "sha512-7gqG38EyHgyP1S+7+xomFtL+ZNHcKv6DwNaCZmJmo1vgMugyF3TCnXVg4t1uk89mLNwnLtnY3TpOpCOyp1/xHQ==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/ws": { "version": "8.18.1", "resolved": "https://registry.npmjs.org/@types/ws/-/ws-8.18.1.tgz", @@ -5434,6 +5445,12 @@ "node": ">=8" } }, + "node_modules/array-flatten": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz", + "integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==", + "license": "MIT" + }, "node_modules/array-includes": { "version": "3.1.9", "resolved": "https://registry.npmjs.org/array-includes/-/array-includes-3.1.9.tgz", @@ -7375,6 +7392,16 @@ "node": ">=6" } }, + "node_modules/destroy": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.2.0.tgz", + "integrity": "sha512-2sJGJTaXIIaR1w4iJSNoN0hnMY7Gpc/n8D4qSCJw8QqFWXf7cuAgnEHxBpweaVcPevC2l3KpjYCx3NypQQgaJg==", + "license": "MIT", + "engines": { + "node": ">= 0.8", + "npm": "1.2.8000 || >= 1.4.16" + } + }, "node_modules/detect-file": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/detect-file/-/detect-file-1.0.0.tgz", @@ -9104,6 +9131,12 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/functional-red-black-tree": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/functional-red-black-tree/-/functional-red-black-tree-1.0.1.tgz", + "integrity": "sha512-dsKNQNdj6xA3T+QlADDA7mOSlX0qiMINjn0cgr+eGHGsbSHzTabcIogz2+p/iqP1Xs6EP/sS2SbqH+brGTbq0g==", + "license": "MIT" + }, "node_modules/functions-have-names": { "version": "1.2.3", "resolved": "https://registry.npmjs.org/functions-have-names/-/functions-have-names-1.2.3.tgz", @@ -11905,7 +11938,6 @@ "version": "1.1.2", "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz", "integrity": "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==", - "dev": true, "license": "MIT", "engines": { "node": ">= 0.6" @@ -13255,6 +13287,12 @@ "node": "20 || >=22" } }, + "node_modules/path-to-regexp": { + "version": "0.1.12", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.12.tgz", + "integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==", + "license": "MIT" + }, "node_modules/path-type": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/path-type/-/path-type-3.0.0.tgz", @@ -16576,6 +16614,15 @@ "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==", "license": "MIT" }, + "node_modules/utils-merge": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz", + "integrity": "sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==", + "license": "MIT", + "engines": { + "node": ">= 0.4.0" + } + }, "node_modules/uuid": { "version": "9.0.1", "resolved": "https://registry.npmjs.org/uuid/-/uuid-9.0.1.tgz", @@ -18047,6 +18094,442 @@ "integrity": "sha512-30sjmas1hQ0gVbX68LAWlm/YYlEqUErunPJJKLpEl+xhK0mKn+jyzlCOpsdTwfkZfPy4U6CDkmygBLC3AB8W9Q==", "dev": true, "license": "MIT" + }, + "packages/workspace-manager": { + "name": "@google/gemini-cli-workspace-manager", + "version": "0.1.0", + "dependencies": { + "@google-cloud/compute": "^4.10.0", + "@google-cloud/firestore": "^7.11.0", + "express": "^4.21.2", + "uuid": "^13.0.0", + "zod": "^3.25.0" + }, + "devDependencies": { + "@types/express": "^5.0.0", + "@types/node": "^20.0.0", + "@types/uuid": "^10.0.0", + "supertest": "^7.2.2", + "typescript": "^5.7.0", + "vitest": "^3.2.0" + } + }, + "packages/workspace-manager/node_modules/@google-cloud/compute": { + "version": "4.12.0", + "resolved": "https://registry.npmjs.org/@google-cloud/compute/-/compute-4.12.0.tgz", + "integrity": "sha512-N3isWbxIMd02qzdlFFxHxEM+2B/vNgn9N7WMpteY2sfwN1yT9PJHcilKDLyw+uIwuQAoErNxBM+JOLq3r/Tv+Q==", + "license": "Apache-2.0", + "dependencies": { + "google-gax": "^4.0.3" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "packages/workspace-manager/node_modules/@google-cloud/firestore": { + "version": "7.11.6", + "resolved": "https://registry.npmjs.org/@google-cloud/firestore/-/firestore-7.11.6.tgz", + "integrity": "sha512-EW/O8ktzwLfyWBOsNuhRoMi8lrC3clHM5LVFhGvO1HCsLozCOOXRAlHrYBoE6HL42Sc8yYMuCb2XqcnJ4OOEpw==", + "license": "Apache-2.0", + "dependencies": { + "@opentelemetry/api": "^1.3.0", + "fast-deep-equal": "^3.1.1", + "functional-red-black-tree": "^1.0.1", + "google-gax": "^4.3.3", + "protobufjs": "^7.2.6" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "packages/workspace-manager/node_modules/accepts": { + "version": "1.3.8", + "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz", + "integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==", + "license": "MIT", + "dependencies": { + "mime-types": "~2.1.34", + "negotiator": "0.6.3" + }, + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/body-parser": { + "version": "1.20.4", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.4.tgz", + "integrity": "sha512-ZTgYYLMOXY9qKU/57FAo8F+HA2dGX7bqGc71txDRC1rS4frdFI5R7NhluHxH6M0YItAP0sHB4uqAOcYKxO6uGA==", + "license": "MIT", + "dependencies": { + "bytes": "~3.1.2", + "content-type": "~1.0.5", + "debug": "2.6.9", + "depd": "2.0.0", + "destroy": "~1.2.0", + "http-errors": "~2.0.1", + "iconv-lite": "~0.4.24", + "on-finished": "~2.4.1", + "qs": "~6.14.0", + "raw-body": "~2.5.3", + "type-is": "~1.6.18", + "unpipe": "~1.0.0" + }, + "engines": { + "node": ">= 0.8", + "npm": "1.2.8000 || >= 1.4.16" + } + }, + "packages/workspace-manager/node_modules/content-disposition": { + "version": "0.5.4", + "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz", + "integrity": "sha512-FveZTNuGw04cxlAiWbzi6zTAL/lhehaWbTtgluJh4/E95DqMwTmha3KZN1aAWA8cFIhHzMZUvLevkw5Rqk+tSQ==", + "license": "MIT", + "dependencies": { + "safe-buffer": "5.2.1" + }, + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/cookie-signature": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.7.tgz", + "integrity": "sha512-NXdYc3dLr47pBkpUCHtKSwIOQXLVn8dZEuywboCOJY/osA0wFSLlSawr3KN8qXJEyX66FcONTH8EIlVuK0yyFA==", + "license": "MIT" + }, + "packages/workspace-manager/node_modules/debug": { + "version": "2.6.9", + "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz", + "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==", + "license": "MIT", + "dependencies": { + "ms": "2.0.0" + } + }, + "packages/workspace-manager/node_modules/debug/node_modules/ms": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", + "integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==", + "license": "MIT" + }, + "packages/workspace-manager/node_modules/express": { + "version": "4.22.1", + "resolved": "https://registry.npmjs.org/express/-/express-4.22.1.tgz", + "integrity": "sha512-F2X8g9P1X7uCPZMA3MVf9wcTqlyNp7IhH5qPCI0izhaOIYXaW9L535tGA3qmjRzpH+bZczqq7hVKxTR4NWnu+g==", + "license": "MIT", + "dependencies": { + "accepts": "~1.3.8", + "array-flatten": "1.1.1", + "body-parser": "~1.20.3", + "content-disposition": "~0.5.4", + "content-type": "~1.0.4", + "cookie": "~0.7.1", + "cookie-signature": "~1.0.6", + "debug": "2.6.9", + "depd": "2.0.0", + "encodeurl": "~2.0.0", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "finalhandler": "~1.3.1", + "fresh": "~0.5.2", + "http-errors": "~2.0.0", + "merge-descriptors": "1.0.3", + "methods": "~1.1.2", + "on-finished": "~2.4.1", + "parseurl": "~1.3.3", + "path-to-regexp": "~0.1.12", + "proxy-addr": "~2.0.7", + "qs": "~6.14.0", + "range-parser": "~1.2.1", + "safe-buffer": "5.2.1", + "send": "~0.19.0", + "serve-static": "~1.16.2", + "setprototypeof": "1.2.0", + "statuses": "~2.0.1", + "type-is": "~1.6.18", + "utils-merge": "1.0.1", + "vary": "~1.1.2" + }, + "engines": { + "node": ">= 0.10.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" + } + }, + "packages/workspace-manager/node_modules/finalhandler": { + "version": "1.3.2", + "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz", + "integrity": "sha512-aA4RyPcd3badbdABGDuTXCMTtOneUCAYH/gxoYRTZlIJdF0YPWuGqiAsIrhNnnqdXGswYk6dGujem4w80UJFhg==", + "license": "MIT", + "dependencies": { + "debug": "2.6.9", + "encodeurl": "~2.0.0", + "escape-html": "~1.0.3", + "on-finished": "~2.4.1", + "parseurl": "~1.3.3", + "statuses": "~2.0.2", + "unpipe": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "packages/workspace-manager/node_modules/form-data": { + "version": "4.0.5", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz", + "integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==", + "dev": true, + "license": "MIT", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "es-set-tostringtag": "^2.1.0", + "hasown": "^2.0.2", + "mime-types": "^2.1.12" + }, + "engines": { + "node": ">= 6" + } + }, + "packages/workspace-manager/node_modules/fresh": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz", + "integrity": "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/iconv-lite": { + "version": "0.4.24", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz", + "integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==", + "license": "MIT", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "packages/workspace-manager/node_modules/media-typer": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", + "integrity": "sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/merge-descriptors": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.3.tgz", + "integrity": "sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "packages/workspace-manager/node_modules/mime": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz", + "integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==", + "license": "MIT", + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=4" + } + }, + "packages/workspace-manager/node_modules/mime-db": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", + "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/mime-types": { + "version": "2.1.35", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", + "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", + "license": "MIT", + "dependencies": { + "mime-db": "1.52.0" + }, + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/negotiator": { + "version": "0.6.3", + "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz", + "integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/raw-body": { + "version": "2.5.3", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.3.tgz", + "integrity": "sha512-s4VSOf6yN0rvbRZGxs8Om5CWj6seneMwK3oDb4lWDH0UPhWcxwOWw5+qk24bxq87szX1ydrwylIOp2uG1ojUpA==", + "license": "MIT", + "dependencies": { + "bytes": "~3.1.2", + "http-errors": "~2.0.1", + "iconv-lite": "~0.4.24", + "unpipe": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, + "packages/workspace-manager/node_modules/send": { + "version": "0.19.2", + "resolved": "https://registry.npmjs.org/send/-/send-0.19.2.tgz", + "integrity": "sha512-VMbMxbDeehAxpOtWJXlcUS5E8iXh6QmN+BkRX1GARS3wRaXEEgzCcB10gTQazO42tpNIya8xIyNx8fll1OFPrg==", + "license": "MIT", + "dependencies": { + "debug": "2.6.9", + "depd": "2.0.0", + "destroy": "1.2.0", + "encodeurl": "~2.0.0", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "fresh": "~0.5.2", + "http-errors": "~2.0.1", + "mime": "1.6.0", + "ms": "2.1.3", + "on-finished": "~2.4.1", + "range-parser": "~1.2.1", + "statuses": "~2.0.2" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "packages/workspace-manager/node_modules/serve-static": { + "version": "1.16.3", + "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.16.3.tgz", + "integrity": "sha512-x0RTqQel6g5SY7Lg6ZreMmsOzncHFU7nhnRWkKgWuMTu5NN0DR5oruckMqRvacAN9d5w6ARnRBXl9xhDCgfMeA==", + "license": "MIT", + "dependencies": { + "encodeurl": "~2.0.0", + "escape-html": "~1.0.3", + "parseurl": "~1.3.3", + "send": "~0.19.1" + }, + "engines": { + "node": ">= 0.8.0" + } + }, + "packages/workspace-manager/node_modules/superagent": { + "version": "10.3.0", + "resolved": "https://registry.npmjs.org/superagent/-/superagent-10.3.0.tgz", + "integrity": "sha512-B+4Ik7ROgVKrQsXTV0Jwp2u+PXYLSlqtDAhYnkkD+zn3yg8s/zjA2MeGayPoY/KICrbitwneDHrjSotxKL+0XQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "component-emitter": "^1.3.1", + "cookiejar": "^2.1.4", + "debug": "^4.3.7", + "fast-safe-stringify": "^2.1.1", + "form-data": "^4.0.5", + "formidable": "^3.5.4", + "methods": "^1.1.2", + "mime": "2.6.0", + "qs": "^6.14.1" + }, + "engines": { + "node": ">=14.18.0" + } + }, + "packages/workspace-manager/node_modules/superagent/node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, + "packages/workspace-manager/node_modules/superagent/node_modules/mime": { + "version": "2.6.0", + "resolved": "https://registry.npmjs.org/mime/-/mime-2.6.0.tgz", + "integrity": "sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg==", + "dev": true, + "license": "MIT", + "bin": { + "mime": "cli.js" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "packages/workspace-manager/node_modules/supertest": { + "version": "7.2.2", + "resolved": "https://registry.npmjs.org/supertest/-/supertest-7.2.2.tgz", + "integrity": "sha512-oK8WG9diS3DlhdUkcFn4tkNIiIbBx9lI2ClF8K+b2/m8Eyv47LSawxUzZQSNKUrVb2KsqeTDCcjAAVPYaSLVTA==", + "dev": true, + "license": "MIT", + "dependencies": { + "cookie-signature": "^1.2.2", + "methods": "^1.1.2", + "superagent": "^10.3.0" + }, + "engines": { + "node": ">=14.18.0" + } + }, + "packages/workspace-manager/node_modules/supertest/node_modules/cookie-signature": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.2.2.tgz", + "integrity": "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.6.0" + } + }, + "packages/workspace-manager/node_modules/type-is": { + "version": "1.6.18", + "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz", + "integrity": "sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==", + "license": "MIT", + "dependencies": { + "media-typer": "0.3.0", + "mime-types": "~2.1.24" + }, + "engines": { + "node": ">= 0.6" + } + }, + "packages/workspace-manager/node_modules/uuid": { + "version": "13.0.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-13.0.0.tgz", + "integrity": "sha512-XQegIaBTVUjSHliKqcnFqYypAd4S+WCYt5NIeRs6w/UAry7z8Y9j5ZwRRL4kzq9U3sD6v+85er9FvkEaBpji2w==", + "funding": [ + "https://github.com/sponsors/broofa", + "https://github.com/sponsors/ctavan" + ], + "license": "MIT", + "bin": { + "uuid": "dist-node/bin/uuid" + } } } } diff --git a/packages/core/src/core/contentGenerator.test.ts b/packages/core/src/core/contentGenerator.test.ts index 57ce1fed23..4bacd1b488 100644 --- a/packages/core/src/core/contentGenerator.test.ts +++ b/packages/core/src/core/contentGenerator.test.ts @@ -725,6 +725,26 @@ describe('createContentGeneratorConfig', () => { expect(config.apiKey).toBeUndefined(); expect(config.vertexai).toBeUndefined(); }); + it('should configure for GATEWAY using dummy placeholder if GEMINI_API_KEY is set', async () => { + vi.stubEnv('GEMINI_API_KEY', 'env-gemini-key'); + const config = await createContentGeneratorConfig( + mockConfig, + AuthType.GATEWAY, + ); + expect(config.apiKey).toBe('gateway-placeholder-key'); + expect(config.vertexai).toBe(false); + }); + + it('should configure for GATEWAY using dummy placeholder if GEMINI_API_KEY is not set', async () => { + vi.stubEnv('GEMINI_API_KEY', ''); + vi.mocked(loadApiKey).mockResolvedValue(null); + const config = await createContentGeneratorConfig( + mockConfig, + AuthType.GATEWAY, + ); + expect(config.apiKey).toBe('gateway-placeholder-key'); + expect(config.vertexai).toBe(false); + }); }); describe('validateBaseUrl', () => { diff --git a/packages/core/src/core/contentGenerator.ts b/packages/core/src/core/contentGenerator.ts index 60641abdeb..ff1739c04b 100644 --- a/packages/core/src/core/contentGenerator.ts +++ b/packages/core/src/core/contentGenerator.ts @@ -150,6 +150,13 @@ export async function createContentGeneratorConfig( return contentGeneratorConfig; } + if (authType === AuthType.GATEWAY) { + contentGeneratorConfig.apiKey = apiKey || 'gateway-placeholder-key'; + contentGeneratorConfig.vertexai = false; + + return contentGeneratorConfig; + } + return contentGeneratorConfig; } diff --git a/packages/workspace-manager/docker/Dockerfile b/packages/workspace-manager/docker/Dockerfile index 197440da98..16910ea3a0 100644 --- a/packages/workspace-manager/docker/Dockerfile +++ b/packages/workspace-manager/docker/Dockerfile @@ -1,6 +1,13 @@ # Copyright 2026 Google LLC # SPDX-License-Identifier: Apache-2.0 +# Stage 1: Build shpool +FROM rust:1.81-slim-bookworm AS builder + +RUN apt-get update && apt-get install -y build-essential curl && rm -rf /var/lib/apt/lists/* +RUN cargo install shpool + +# Stage 2: Final Image FROM node:20-slim # Install system dependencies @@ -11,7 +18,6 @@ RUN apt-get update && apt-get install -y \ vim \ tmux \ procps \ - build-essential \ && rm -rf /var/lib/apt/lists/* # Install GitHub CLI @@ -21,13 +27,8 @@ RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | d && apt-get update \ && apt-get install gh -y -# Install Rust (to install shpool) -RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y -ENV PATH="/root/.cargo/bin:${PATH}" - -# Install shpool -RUN cargo install shpool \ - && mv /root/.cargo/bin/shpool /usr/local/bin/shpool +# Copy shpool from builder +COPY --from=builder /usr/local/cargo/bin/shpool /usr/local/bin/shpool # Install global dev tools RUN npm install -g tsx eslint vitest typescript prettier @google/gemini-cli@nightly diff --git a/packages/workspace-manager/docker/entrypoint.sh b/packages/workspace-manager/docker/entrypoint.sh index 1c9a7b2f2d..41c3e6c8f1 100644 --- a/packages/workspace-manager/docker/entrypoint.sh +++ b/packages/workspace-manager/docker/entrypoint.sh @@ -2,14 +2,25 @@ # Copyright 2026 Google LLC # SPDX-License-Identifier: Apache-2.0 +set -e + # Ensure GH_TOKEN is set from memory-only mount if available if [ -f /dev/shm/.gh_token ]; then export GH_TOKEN=$(cat /dev/shm/.gh_token) echo "GitHub token injected from memory." fi -# Start shpool daemon in the background +# Start shpool daemon in the background and verify it stays up /usr/local/bin/shpool daemon & +SHPOOL_PID=$! + +sleep 2 +if ! kill -0 $SHPOOL_PID 2>/dev/null; then + echo "Error: shpool daemon failed to start" + exit 1 +fi + +echo "shpool daemon started successfully (PID: $SHPOOL_PID)" # Restore ~/.gemini settings if they are provided in a mount or PD # (Assuming PD is mounted at /home/node/persistent_home for now) diff --git a/packages/workspace-manager/package.json b/packages/workspace-manager/package.json index 76122a730b..ec5d65526f 100644 --- a/packages/workspace-manager/package.json +++ b/packages/workspace-manager/package.json @@ -21,6 +21,7 @@ "@types/express": "^5.0.0", "@types/node": "^20.0.0", "@types/uuid": "^10.0.0", + "supertest": "^7.2.2", "typescript": "^5.7.0", "vitest": "^3.2.0" } diff --git a/packages/workspace-manager/src/index.ts b/packages/workspace-manager/src/index.ts index 7dd77f4530..d8eb0f0234 100644 --- a/packages/workspace-manager/src/index.ts +++ b/packages/workspace-manager/src/index.ts @@ -5,121 +5,24 @@ */ import express from 'express'; -import type { Request, Response, RequestHandler } from 'express'; -import { v4 as uuidv4 } from 'uuid'; -import { Firestore } from '@google-cloud/firestore'; +import { workspaceRouter } from './routes/workspaceRoutes.js'; -interface WorkspaceData { - owner_id: string; - name: string; - instance_name: string; - status: string; - machine_type: string; - created_at: string; -} - -const app = express(); +export const app = express(); app.use(express.json()); -// Initialize Firestore -const firestore = new Firestore(); - const PORT = process.env.PORT || 8080; -app.get('/health', (_req: Request, res: Response) => { +app.get('/health', (_req, res) => { res.send({ status: 'ok' }); }); -/** - * List all workspaces for the authenticated user - */ -const listWorkspaces: RequestHandler = async (_req, res) => { - try { - const ownerId = 'default-user'; // TODO: Get from OAuth/IAP headers - const snapshot = await firestore - .collection('workspaces') - .where('owner_id', '==', ownerId) - .get(); +// Register Workspace Routes +app.use('/workspaces', workspaceRouter); - const workspaces = snapshot.docs.map((doc) => { - // eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion - const data = doc.data() as WorkspaceData; - return { - id: doc.id, - ...data, - }; - }); - - res.json(workspaces); - } catch (error: unknown) { - const message = error instanceof Error ? error.message : String(error); - res.status(500).json({ error: message }); - } -}; - -app.get('/workspaces', listWorkspaces); - -/** - * Create a new workspace (GCE VM) - */ -const createWorkspace: RequestHandler = async (req, res) => { - try { - // eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion - const body = req.body as Record; - const name = typeof body['name'] === 'string' ? body['name'] : 'unnamed'; - const machineType = - typeof body['machineType'] === 'string' - ? body['machineType'] - : 'e2-standard-4'; - - const ownerId = 'default-user'; // TODO: Get from OAuth/IAP headers - const workspaceId = uuidv4(); - const instanceName = `workspace-${workspaceId.slice(0, 8)}`; - - const workspaceData: WorkspaceData = { - owner_id: ownerId, - name, - instance_name: instanceName, - status: 'PROVISIONING', - machine_type: machineType, - created_at: new Date().toISOString(), - }; - - await firestore - .collection('workspaces') - .doc(workspaceId) - .set(workspaceData); - - res.status(201).json({ id: workspaceId, ...workspaceData }); - } catch (error: unknown) { - const message = error instanceof Error ? error.message : String(error); - res.status(500).json({ error: message }); - } -}; - -app.post('/workspaces', createWorkspace); - -/** - * Delete a workspace - */ -const deleteWorkspace: RequestHandler = async (req, res) => { - try { - const { id } = req.params; - if (!id) { - res.status(400).json({ error: 'Workspace ID is required' }); - return; - } - await firestore.collection('workspaces').doc(id).delete(); - res.status(204).send(); - } catch (error: unknown) { - const message = error instanceof Error ? error.message : String(error); - res.status(500).json({ error: message }); - } -}; - -app.delete('/workspaces/:id', deleteWorkspace); - -app.listen(PORT, () => { - // eslint-disable-next-line no-console - console.log(`Workspace Hub listening on port ${PORT}`); -}); +// Only listen if not in test mode +if (process.env.NODE_ENV !== 'test') { + app.listen(PORT, () => { + // eslint-disable-next-line no-console + console.log(`Workspace Hub listening on port ${PORT}`); + }); +} diff --git a/packages/workspace-manager/src/routes/workspaceRoutes.test.ts b/packages/workspace-manager/src/routes/workspaceRoutes.test.ts new file mode 100644 index 0000000000..d638c0764d --- /dev/null +++ b/packages/workspace-manager/src/routes/workspaceRoutes.test.ts @@ -0,0 +1,64 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, vi, beforeEach } from 'vitest'; +import request from 'supertest'; +import { app } from '../index.js'; + +// Mock the services +vi.mock('../services/workspaceService.js', () => ({ + WorkspaceService: vi.fn().mockImplementation(() => ({ + listWorkspaces: vi.fn().mockResolvedValue([]), + getWorkspace: vi.fn().mockResolvedValue(null), + createWorkspace: vi.fn().mockResolvedValue(undefined), + deleteWorkspace: vi.fn().mockResolvedValue(undefined), + })), + })); + +vi.mock('../services/computeService.js', () => ({ + ComputeService: vi.fn().mockImplementation(() => ({ + createWorkspaceInstance: vi.fn().mockResolvedValue(undefined), + deleteWorkspaceInstance: vi.fn().mockResolvedValue(undefined), + })), + })); + +describe('Workspace Routes', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('GET /workspaces', () => { + it('should return an empty list of workspaces', async () => { + const response = await request(app).get('/workspaces'); + expect(response.status).toBe(200); + expect(response.body).toEqual([]); + }); + }); + + describe('POST /workspaces', () => { + it('should create a new workspace', async () => { + const payload = { name: 'test-workspace' }; + const response = await request(app).post('/workspaces').send(payload); + + expect(response.status).toBe(201); + expect(response.body.name).toBe('test-workspace'); + expect(response.body.owner_id).toBe('default-user'); + expect(response.body.status).toBe('PROVISIONING'); + }); + + it('should fail if name is missing', async () => { + const response = await request(app).post('/workspaces').send({}); + expect(response.status).toBe(400); + }); + }); + + describe('DELETE /workspaces/:id', () => { + it('should return 404 if workspace not found', async () => { + const response = await request(app).delete('/workspaces/non-existent'); + expect(response.status).toBe(404); + }); + }); +}); diff --git a/packages/workspace-manager/src/routes/workspaceRoutes.ts b/packages/workspace-manager/src/routes/workspaceRoutes.ts new file mode 100644 index 0000000000..22e4327772 --- /dev/null +++ b/packages/workspace-manager/src/routes/workspaceRoutes.ts @@ -0,0 +1,113 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Router } from 'express'; +import { v4 as uuidv4 } from 'uuid'; +import { z } from 'zod'; +import { WorkspaceService } from '../services/workspaceService.js'; +import { ComputeService } from '../services/computeService.js'; + +const router = Router(); +const workspaceService = new WorkspaceService(); +const computeService = new ComputeService(); + +const CreateWorkspaceSchema = z.object({ + name: z.string().min(1), + machineType: z.string().optional().default('e2-standard-4'), + imageTag: z.string().optional().default('latest'), + zone: z.string().optional().default('us-west1-a'), +}); + +const DEFAULT_OWNER = 'default-user'; // TODO: Replace with IAP identity middleware + +router.get('/', async (_req, res) => { + try { + const workspaces = await workspaceService.listWorkspaces(DEFAULT_OWNER); + res.json(workspaces); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + res.status(500).json({ error: message }); + } +}); + +router.post('/', async (req, res) => { + try { + const validation = CreateWorkspaceSchema.safeParse(req.body); + if (!validation.success) { + res.status(400).json({ error: validation.error.format() }); + return; + } + + const { name, machineType, imageTag, zone } = validation.data; + const workspaceId = uuidv4(); + const instanceName = `workspace-${workspaceId.slice(0, 8)}`; + + const workspaceData = { + owner_id: DEFAULT_OWNER, + name, + instance_name: instanceName, + status: 'PROVISIONING', + machine_type: machineType, + zone, + created_at: new Date().toISOString(), + }; + + // 1. Save to state store + await workspaceService.createWorkspace(workspaceId, workspaceData); + + // 2. Trigger GCE provisioning (Async) + computeService + .createWorkspaceInstance({ + instanceName, + machineType, + imageTag, + zone, + }) + .catch((err) => { + // eslint-disable-next-line no-console + console.error(`Failed to provision GCE instance ${instanceName}:`, err); + }); + + res.status(201).json({ id: workspaceId, ...workspaceData }); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + res.status(500).json({ error: message }); + } +}); + +router.delete('/:id', async (req, res) => { + try { + const { id } = req.params; + const workspace = await workspaceService.getWorkspace(id); + + if (!workspace) { + res.status(404).json({ error: 'Workspace not found' }); + return; + } + + // SECURITY: Ownership Check + if (workspace.owner_id !== DEFAULT_OWNER) { + res.status(403).json({ error: 'Unauthorized to delete this workspace' }); + return; + } + + // 1. Delete GCE instance + await computeService.deleteWorkspaceInstance( + workspace.instance_name, + workspace.zone, + ); + + // 2. Delete from state store + await workspaceService.deleteWorkspace(id); + + res.status(204).send(); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + res.status(500).json({ error: message }); + } +}); + +export const workspaceRouter = router; diff --git a/packages/workspace-manager/src/services/computeService.ts b/packages/workspace-manager/src/services/computeService.ts new file mode 100644 index 0000000000..428e5cde44 --- /dev/null +++ b/packages/workspace-manager/src/services/computeService.ts @@ -0,0 +1,48 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { InstancesClient } from '@google-cloud/compute'; + +export interface ProvisionOptions { + instanceName: string; + machineType: string; + imageTag: string; + zone: string; +} + +export class ComputeService { + private client: InstancesClient; + + constructor() { + this.client = new InstancesClient(); + } + + /** + * Provision a new GCE VM with the Workspace Container + */ + async createWorkspaceInstance(options: ProvisionOptions): Promise { + // TODO: Implement actual instancesClient.insert call + // For now, we just log and return + // eslint-disable-next-line no-console + console.log( + `[ComputeService] Mocking creation of ${options.instanceName} in ${options.zone}`, + ); + } + + /** + * Terminate a GCE VM + */ + async deleteWorkspaceInstance( + instanceName: string, + zone: string, + ): Promise { + // TODO: Implement actual instancesClient.delete call + // eslint-disable-next-line no-console + console.log( + `[ComputeService] Mocking deletion of ${instanceName} in ${zone}`, + ); + } +} diff --git a/packages/workspace-manager/src/services/workspaceService.ts b/packages/workspace-manager/src/services/workspaceService.ts new file mode 100644 index 0000000000..9ae21549d8 --- /dev/null +++ b/packages/workspace-manager/src/services/workspaceService.ts @@ -0,0 +1,60 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Firestore } from '@google-cloud/firestore'; + +export interface WorkspaceData { + owner_id: string; + name: string; + instance_name: string; + status: string; + machine_type: string; + zone: string; + created_at: string; +} + +export interface WorkspaceRecord extends WorkspaceData { + id: string; +} + +export class WorkspaceService { + private firestore: Firestore; + + constructor() { + this.firestore = new Firestore(); + } + + async listWorkspaces(ownerId: string): Promise { + const snapshot = await this.firestore + .collection('workspaces') + .where('owner_id', '==', ownerId) + .get(); + + return snapshot.docs.map((doc) => { + // eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion + const data = doc.data() as WorkspaceData; + return { + id: doc.id, + ...data, + }; + }); + } + + async getWorkspace(id: string): Promise { + const doc = await this.firestore.collection('workspaces').doc(id).get(); + if (!doc.exists) return null; + // eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion + return { id: doc.id, ...(doc.data() as WorkspaceData) }; + } + + async createWorkspace(id: string, data: WorkspaceData): Promise { + await this.firestore.collection('workspaces').doc(id).set(data); + } + + async deleteWorkspace(id: string): Promise { + await this.firestore.collection('workspaces').doc(id).delete(); + } +} diff --git a/plans/phase-1-workspace-core.md b/plans/phase-1-workspace-core.md index 059e98a401..b0737dc02b 100644 --- a/plans/phase-1-workspace-core.md +++ b/plans/phase-1-workspace-core.md @@ -1,35 +1,51 @@ # Phase 1 Sub-plan: The Workspace Core ## 1. Objective -Establish the foundational execution environment (Container Image) and the initial management service (Hub API). + +Establish the foundational execution environment (Container Image) and the +initial management service (Hub API). ## 2. Tasks ### Task 1.1: Define and Build Workspace Image -Create a Dockerfile that provides a complete, persistent development environment for `gemini-cli`. -- [ ] Create `packages/grid-manager/docker/Dockerfile`. + +Create a Dockerfile that provides a complete, persistent development environment +for `gemini-cli`. + +- [ ] Create `packages/workspace-manager/docker/Dockerfile`. - [ ] Include: `node:20-slim`, `git`, `gh`, `rsync`, `tmux`, `shpool`. - [ ] Add the pre-built `gemini-cli` binary. - [ ] Define `entrypoint.sh` with secret injection and `shpool` daemon startup. - [ ] Verify image build locally: `docker build -t gemini-workspace:v1 .`. ### Task 1.2: Workspace Hub API (v1) + Implement the core API to manage GCE-based workspaces. -- [ ] Initialize `packages/grid-manager/src/hub-service/`. -- [ ] Implement Express or Fastify server for `/workspaces` (List, Create, Delete). + +- [ ] Initialize `packages/workspace-manager/src/hub-service/`. +- [ ] Implement Express or Fastify server for `/workspaces` (List, Create, + Delete). - [ ] Integrate Firestore to track workspace state (owner, instance_id, status). - [ ] Integrate `@google-cloud/compute` for GCE instance lifecycle. -- [ ] Provision a VM with `Container-on-VM` settings pointing to the `gemini-workspace` image. +- [ ] Provision a VM with `Container-on-VM` settings pointing to the + `gemini-workspace` image. ### Task 1.3: Cloud Run Deployment (v1) + Prepare the Hub for self-service deployment. -- [ ] Create `packages/grid-manager/terraform/` for basic Hub provisioning. + +- [ ] Create `packages/workspace-manager/terraform/` for basic Hub provisioning. - [ ] Setup IAP/OAuth authentication on the Cloud Run endpoint. ## 3. Verification & Success Criteria -- **Image:** A container started from the image must have `gemini --version` and `gh --version` available. -- **API:** A `POST /workspaces` call must result in a new VM appearing in the specified GCP project with the correct container image. -- **State:** Firestore must correctly reflect the VM's `PROVISIONING` and `READY` status. + +- **Image:** A container started from the image must have `gemini --version` and + `gh --version` available. +- **API:** A `POST /workspaces` call must result in a new VM appearing in the + specified GCP project with the correct container image. +- **State:** Firestore must correctly reflect the VM's `PROVISIONING` and + `READY` status. ## 4. Next Steps + - Implement Task 1.1: Build the Dockerfile.