diff --git a/TELEPORTATION_ONE_PAGER.md b/TELEPORTATION_ONE_PAGER.md new file mode 100644 index 0000000000..4eba71d29c --- /dev/null +++ b/TELEPORTATION_ONE_PAGER.md @@ -0,0 +1,96 @@ +# One-Pager: Session Teleportation (External Tool Resumption) + +## 1. Objective + +Enable Gemini CLI users to seamlessly resume work sessions (trajectories) that +were originated and executed in external coding tools or IDE extensions. This +"teleportation" of sessions establishes Gemini CLI as a centralized hub for +AI-assisted development across various environments. + +## 2. Background & Motivation + +Developers frequently switch between multiple tools (e.g., IDE extensions, +web-based coding assistants, and the terminal). Currently, context is +fragmented; a session started in an external tool cannot be natively continued +in Gemini CLI. By allowing Gemini CLI to parse and ingest trajectories from +these external tools, we prevent context loss, avoid duplicate work, and +significantly enhance developer productivity. + +## 3. Goals & Non-Goals + +**Goals:** + +- Provide a standardized abstraction (`TrajectoryProvider`) for fetching + external session histories. +- Implement a core translation layer (`teleporter` module) to map external tool + actions (e.g., `find`, `search_web`, `read_url`, `write_to_file`) to Gemini + CLI's native tool formats. +- Update the `/resume` CLI command to display and filter these external sessions + alongside native ones. +- Support dynamic loading of new trajectory providers via the extension system. + +**Non-Goals:** + +- Two-way sync: We do not intend to export Gemini CLI sessions back to the + originating external tool in this phase. +- Perfect fidelity for unsupported external tools: Only explicitly supported and + mapped tools will be translated. Others will be gracefully ignored or logged + as plain text. + +## 4. Proposed Design / Architecture + +The design revolves around a pluggable architecture that decouples the fetching +of external trajectories from the core CLI logic. + +### 4.1. `TrajectoryProvider` Interface + +A standard contract that external extensions implement. It defines methods for: + +- Retrieving a list of available sessions. +- Fetching the full trajectory payload for a specific session ID. +- Providing metadata (display name, icon, source tool name) for the CLI UI. + +### 4.2. Core `teleporter` Module + +This module acts as the ingestion engine. It performs: + +- **Deserialization:** Parsing the raw JSON/format of the external tool. +- **Conversion:** Mapping external tool calls (e.g., an Antigravity `search_web` + call) into the equivalent Gemini CLI tool call format, ensuring the LLM + understands the historical context correctly. +- **Validation:** Ensuring the resulting session history conforms to Gemini + CLI's internal session schema. + +### 4.3. UI Integration (`/resume`) + +The `SessionBrowser` UI will be updated to handle multiple sources. + +- Introduce dynamic tabs or source filters in the TUI (Terminal UI). +- Display metadata (e.g., "Imported from Tool X", timestamps) to help users + distinguish between native and teleported sessions. + +### 4.4. Extension Manager Updates + +Enhance the existing `extensionLoader` to discover, load, and register plugins +that implement the `TrajectoryProvider` interface dynamically at runtime. + +## 5. Rollout Plan + +The implementation is broken down into incremental PRs tracked by Epic #22801: + +1. **Define Core Interfaces:** Establish `TrajectoryProvider`. +2. **Implement Translation Logic:** Build the `teleporter` module with initial + conversion mappings. +3. **UI Updates:** Hook the providers into the `/resume` interface. +4. **Extension Support:** Enable dynamic loading of providers. +5. **Documentation:** Publish `TELEPORTATION.md` outlining how to build new + providers. + +## 6. Security & Privacy + +- **Local First:** Trajectories should be read from local filesystem artifacts + or secure local IPC mechanisms where possible, avoiding unnecessary network + calls. +- **Data Sanitization:** The conversion process must not execute any historical + commands; it strictly translates historical records into past-tense context + for the LLM. diff --git a/packages/core/src/teleportation/TELEPORTATION.md b/packages/core/src/teleportation/TELEPORTATION.md new file mode 100644 index 0000000000..6bb1f87741 --- /dev/null +++ b/packages/core/src/teleportation/TELEPORTATION.md @@ -0,0 +1,251 @@ +# Trajectory Teleportation: Antigravity to Gemini CLI + +This document explains how the Gemini CLI discovers, reads, and converts +Antigravity (Jetski) trajectories to enable session resumption. + +## Overview + +The teleportation feature allows you to pick up a conversation in the Gemini CLI +that was started in the Antigravity (Jetski) IDE. + +## 1. Discovery + +The CLI identifies Antigravity sessions by scanning the local filesystem. + +- **Storage Location**: `~/.antigravity/conversations` +- **File Format**: Binary Protobuf files with the `.pb` extension. +- **Session IDs**: The filenames (e.g., `f81d4fae-7dec.pb`) serve as the unique + identifiers for resumption. + +## 2. Decryption & Parsing + +Since Antigravity stores data in a specialized binary format, the CLI uses a +dedicated teleporter bundle: + +- **Logic**: `trajectory_teleporter.min.js` (bundled in + `@google/gemini-cli-core`). +- **Process**: The binary `.pb` file is read into a Buffer and passed to the + teleporter's `trajectoryToJson` function, which outputs a standard JavaScript + object. +- **Generation**: To ensure the CLI stays in sync with Jetski's protobuf schema + (`cortex.proto`), this bundle isn't written entirely by hand. It is + auto-generated by compiling the latest schema via `esbuild`. You can rebuild + it anytime by running `npm run build:teleporter` from the CLI root. _(See + [How to Regenerate the Bundle](#how-to-regenerate-the-bundle) for exact + steps)._ + +## 3. Workspace Identification + +To filter sessions by the user's active workspace, the CLI attempts to read the +target workspace from the trajectory. + +1. **Primary**: It reads `metadata.workspaces[0].workspaceFolderAbsoluteUri` + from the top-level trajectory metadata. This is the authoritative data + populated by the Jetski Go backend. +2. **Fallback**: For older trajectories without top-level metadata, it attempts + to extract `workspaceUri` from the first user input's + `activeUserState.activeDocument` context. + +## 4. Conversion Logic + +The conversion layer +([converter.ts](file:///Users/sshon/developments/gemini-cli/packages/core/src/teleportation/converter.ts)) +translates the technical "Steps" of an Antigravity trajectory into the CLI's +`ConversationRecord` format: + +- **User Input**: Maps `CORTEX_STEP_TYPE_USER_INPUT` (type 14) to `user` + messages. +- **Model Responses**: Maps `CORTEX_STEP_TYPE_PLANNER_RESPONSE` (type 15) to + `gemini` messages. +- **Thoughts & Reasoning**: Extracts reasoning content from the Antigravity step + and populates the `thoughts` array in the CLI record, preserving the model's + logic. +- **Tool Calls**: Maps Antigravity tool execution steps to CLI `ToolCallRecord` + objects. The integration handles these in three main categories: + +| Tool Capability | Gemini CLI Tool | Jetski Protobuf Step | Teleportation Status | +| :------------------- | :------------------------------------- | :---------------------------------------------- | :----------------------- | +| **Read File** | `read_file` | `CORTEX_STEP_TYPE_VIEW_FILE` | ✅ Native Mapping | +| **List Directory** | `ls` | `CORTEX_STEP_TYPE_LIST_DIRECTORY` | ✅ Native Mapping | +| **Search Code** | `grep_search` | `CORTEX_STEP_TYPE_GREP_SEARCH` | ✅ Native Mapping | +| **Edit File** | `replace` | `CORTEX_STEP_TYPE_FILE_CHANGE` | ✅ Native Mapping | +| **Search Web** | `google_web_search` | `CORTEX_STEP_TYPE_SEARCH_WEB` | ✅ Native Mapping | +| **Execute Shell** | `run_shell_command` | `CORTEX_STEP_TYPE_RUN_COMMAND` | ✅ Native Mapping | +| **Browser Agent** | _N/A_ \* | `CORTEX_STEP_TYPE_BROWSER_SUBAGENT` | ❌ Dropped | +| **User Input** | `ask_user` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **Read URL Content** | `web_fetch` | `CORTEX_STEP_TYPE_READ_URL_CONTENT` | ❌ Dropped | +| **Find Files** | `glob` | `CORTEX_STEP_TYPE_FIND` | ❌ Dropped | +| **Write File** | `write_file` | `CORTEX_STEP_TYPE_WRITE_TO_FILE` | ❌ Dropped | +| **Read Many Files** | `read_many_files` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **Write Todos** | `write_todos` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **Save Memory** | `save_memory` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **Docs/Skills** | `get_internal_docs` & `activate_skill` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **Plan Mode** | `enter_plan_mode` & `exit_plan_mode` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP | +| **IDE Actions** | _N/A_ | `VIEW_CODE_ITEM`, `LINT_DIFF`, `ADD_ANNOTATION` | 🚫 Jetski Only (Dropped) | + +**1. Native Mappings** Antigravity has specific protobuf message types for +common tool calls, which map directly to the CLI's native tools: + +- `CORTEX_STEP_TYPE_VIEW_FILE` -> `read_file` +- `CORTEX_STEP_TYPE_LIST_DIRECTORY` -> `ls` +- `CORTEX_STEP_TYPE_GREP_SEARCH` -> `grep_search` +- `CORTEX_STEP_TYPE_RUN_COMMAND` -> `run_shell_command` +- `CORTEX_STEP_TYPE_FILE_CHANGE` -> `replace` +- `CORTEX_STEP_TYPE_BROWSER_SUBAGENT` -> (Dropped) + +**2. Generic / MCP Tools** + +- Jetski relies heavily on `CORTEX_STEP_TYPE_GENERIC` and + `CORTEX_STEP_TYPE_MCP_TOOL` to route non-native or dynamic tools. +- This is fully supported! The CLI reads the `toolName` and `argsJson` directly + from the generic step payload. For instance, the Jetski `ask_user` tool + natively maps to the CLI's `ask_user` tool, and any custom MCP commands are + preserved as-is. + +**3. Unsupported Tools** Many isolated actions, sub-agent tools, and +IDE-specific UI interactions are dropped by the teleporter to maintain strict +CLI compatibility and preserve valid context-window state. + +**4. Tool UI & Context Representation** When importing dynamic/generic tools, +the CLI UI uses internal synthesis to properly reflect tool usage and outputs +that the user saw in JetSki without blank rendering: + +- **Arguments:** Arguments extracted from `argsJson` are synthesized into the + `.description` field enabling the CLI UI to display the exact call arguments + (e.g., specific files or search strings) below the tool name. +- **Output (Result Display):** Tool outputs, like terminal output payloads or + file text, are iteratively extracted from the trajectory steps and rendered + explicitly using `resultDisplay`. + +
+Click to view exhaustive list of all 75+ dropped Jetski steps + +Any step mapped in `cortex.proto` that isn't functionally replicated by the CLI +is skipped. This includes: + +- **Browser UI & Automation**: `CORTEX_STEP_TYPE_BROWSER_CLICK_ELEMENT`, + `CORTEX_STEP_TYPE_BROWSER_MOVE_MOUSE`, `CORTEX_STEP_TYPE_BROWSER_SUBAGENT`, + `CORTEX_STEP_TYPE_EXECUTE_BROWSER_JAVASCRIPT`, + `CORTEX_STEP_TYPE_CAPTURE_BROWSER_SCREENSHOT`, + `CORTEX_STEP_TYPE_READ_BROWSER_PAGE`, `CORTEX_STEP_TYPE_BROWSER_GET_DOM`, + `CORTEX_STEP_TYPE_BROWSER_LIST_NETWORK_REQUESTS`, + `CORTEX_STEP_TYPE_BROWSER_SCROLL_UP`, `CORTEX_STEP_TYPE_OPEN_BROWSER_URL`, and + 15+ other browser controls. +- **IDE & UI Actions**: `CORTEX_STEP_TYPE_VIEW_CODE_ITEM`, + `CORTEX_STEP_TYPE_LINT_DIFF`, `CORTEX_STEP_TYPE_ADD_ANNOTATION`, + `CORTEX_STEP_TYPE_VIEW_FILE_OUTLINE`, `CORTEX_STEP_TYPE_CODE_SEARCH`, + `CORTEX_STEP_TYPE_FIND_ALL_REFERENCES`, `CORTEX_STEP_TYPE_CODE_ACTION`. +- **Agent Framework**: `CORTEX_STEP_TYPE_TASK_BOUNDARY`, + `CORTEX_STEP_TYPE_NOTIFY_USER`, `CORTEX_STEP_TYPE_INVOKE_SUBAGENT`, + `CORTEX_STEP_TYPE_CHECKPOINT`, `CORTEX_STEP_TYPE_EPHEMERAL_MESSAGE`, + `CORTEX_STEP_TYPE_COMMAND_STATUS`, `CORTEX_STEP_TYPE_SEND_COMMAND_INPUT`. +- **Knowledge & Workspace**: `CORTEX_STEP_TYPE_KNOWLEDGE_GENERATION`, + `CORTEX_STEP_TYPE_KI_INSERTION`, `CORTEX_STEP_TYPE_TRAJECTORY_SEARCH`, + `CORTEX_STEP_TYPE_MQUERY`, `CORTEX_STEP_TYPE_INTERNAL_SEARCH`, + `CORTEX_STEP_TYPE_LIST_RESOURCES`, `CORTEX_STEP_TYPE_WORKSPACE_API`. +- **Misc Commands**: `CORTEX_STEP_TYPE_GIT_COMMIT`, + `CORTEX_STEP_TYPE_GENERATE_IMAGE`, `CORTEX_STEP_TYPE_COMPILE`, + `CORTEX_STEP_TYPE_CLIPBOARD`, `CORTEX_STEP_TYPE_ERROR_MESSAGE`, etc. + +
+ +## 5. Session Resumption + +Once converted: + +1. The record is injected into the CLI's `ChatRecordingService`. +2. Users can continue the conversation seamlessly via the `/resume` command. + +## Maintenance & Updates + +You are correct that if Antigravity's Protobuf definitions change, the +`trajectory_teleporter.min.js` bundle will need to be updated to maintain +compatibility. + +### When to Update + +- If new step types are added to Antigravity that the CLI should support in + `converter.ts`. +- If the binary format of the `.pb` files changes. +- If the encryption key or algorithm is rotated. + +### How to Regenerate the Bundle + +To keep the CLI up to date, you can run the automated build script: + +```bash +npm run build:teleporter +``` + +This runs `packages/core/scripts/build-teleporter.sh`, which automatically: + +1. Navigates to the neighboring `Exafunction` directory. +2. Generates the latest Protobuf JS schemas (`pnpm --dir exa/proto_ts build`). +3. Uses `esbuild` to re-bundle the CLI's teleporter script + (`trajectory_teleporter.ts`) against the fresh schemas. +4. Outputs the new `trajectory_teleporter.min.js` directly into the `gemini-cli` + source tree. + +> [!TIP] In the long term, this logic could be moved to a shared NPM package +> published from the Antigravity repository, allowing the Gemini CLI to stay +> updated via a simple `npm update`. + +## Productionization Roadmap + +To safely and seamlessly bring the Jetski trajectory teleportation feature to a +fully production-ready state in the Gemini CLI, several key areas need to be +addressed: + +### 1. Security & Key Management + +- **Dynamic Key Exchange:** ✅ The CLI now supports loading encryption keys from + `JETSKI_TELEPORT_KEY` environment variables or a local + `~/.gemini/jetski/key.txt` file. +- **Permission Scoping:** Ensure the CLI enforces the same file-access + permission rules (`file_permission_request`) that Jetski enforces so the AI + doesn't suddenly gain destructive permissions when transitioning to the + terminal. + +### 2. Architecture & Build Process Decoupling + +- **Trajectory Provider Interface:** ✅ The CLI now uses a generic + `TrajectoryProvider` interface, allowing teleportation logic to be decoupled + into extensions. +- **Shared NPM Package:** Publish the compiled Protobufs and parsing logic as a + private internal package (e.g., `@google/cortex-teleporter`). The Gemini CLI + should simply `npm install` this, rather than generating `.min.js` blobs + manually. +- **Schema Versioning:** Add version checks. If the CLI encounters a trajectory + from a newer version of Jetski with breaking protobuf changes, it should + gracefully prompt the user to update the CLI. + +### 3. User Experience (UX) + +- **Clear UI Indicators:** ✅ Jetski sessions are now grouped in a dedicated tab + in the `/resume` menu. +- **Missing Context Warnings:** ✅ The UI now synthesizes `description` and + `resultDisplay` for generic tool calls to ensure a smooth conversation flow. +- **Seamless Handoff Prompt:** ✅ The CLI now intelligently prompts the user on + startup if a recent Jetski session is found: _"🛸 You have a recent session in + Antigravity. Type /resume agy: to bring it into the terminal."_ + +### 4. Data Fidelity & Error Handling + +- **Graceful Degradation:** If the CLI fails to parse a specific `generic` step + or misses a tool argument, it shouldn't crash the entire resume process. It + should skip the broken step and append a system message warning the model. +- **File State Synchronization:** If a user made uncommitted file edits in + Jetski (e.g., `fileChange` steps that haven't been saved to disk), the CLI + needs to ensure it doesn't accidentally overwrite or hallucinate disk state. + It might be worth requiring a file save before teleporting. + +### 5. Performance + +- **Lazy Loading / Streaming:** When populating the list of sessions for + `/resume`, the CLI shouldn’t decrypt and parse the _entire_ history of every + file. It should only read the `metadata` headers to render the UI picker, and + only parse the full multi-megabyte `ConversationRecord` when a specific + session is selected. +- **Memory Limits:** Set an upper limit on how many historical tool turns the + CLI loads into memory when teleporting to avoid OOM (Out of Memory) crashes in + Node.js. diff --git a/packages/core/src/teleportation/remote_teleportation_proposal.md b/packages/core/src/teleportation/remote_teleportation_proposal.md new file mode 100644 index 0000000000..c89e41ae55 --- /dev/null +++ b/packages/core/src/teleportation/remote_teleportation_proposal.md @@ -0,0 +1,154 @@ +# Remote Teleportation Architecture Proposal + +## Objective + +Prevent leaking proprietary JetSki Protobuf schemas and decryption keys directly +within the public Gemini CLI bundle. When a user requests to resume a JetSki +trajectory, the CLI will interact with an external, secure, or isolated +converter to fetch the standard `ConversationRecord` JSON. + +## Guarantees & Constraints + +- No Protobuf definitions are packaged or visible in the Gemini CLI + distribution. +- Telemetry/Decryption keys do not need to be hardcoded in the CLI. +- Conversion remains effortless and instantaneous for the end-user. +- **Strict Logic Confidentiality**: Proprietary conversion logic and Protobuf + schemas cannot be readable or reverse-engineered by the user via local caches + or easily accessible client-side code. + +--- + +## Option 1: Local MCP Server (JetSki Daemon Extension) + +Since Gemini CLI already natively supports the **Model Context Protocol (MCP)**, +we can utilize JetSki directly as an MCP provider. + +### How it works + +The JetSki extension (or its already running background daemon) can expose an +MCP tool, for example, `get_trajectory_history`. When the user runs `/resume`, +Gemini CLI invokes this tool via the standard MCP pipeline. + +1. **User runs `/resume`**: Gemini CLI lists available trajectories (it can + list .pb files from the local directory or query the MCP server for a + history list). +2. **Conversation Selection**: The user selects a session. +3. **MCP Tool Call**: Gemini calls the MCP tool + `get_trajectory(filePath: string)`. +4. **Local Conversion**: JetSki decrypts, parses, and returns the strictly + CLI-compliant `HistoryItem` JSON payload over stdio/SSE. +5. **No Overhead**: The CLI injects the payload directly into its state without + ever touching a Protobuf. + +### Effort & Trade-offs + +- **Effort**: Very Low. Gemini CLI already supports MCP out of the box. No new + endpoints, infrastructure, or routing is needed. We only need to write a small + MCP server module using `@modelcontextprotocol/sdk` inside JetSki. +- **Pros**: Zero remote network latency; highly secure (stays on local machine); + incredibly seamless. +- **Cons**: Requires the JetSki service to be installed and running in the + background. + +--- + +## Option 2: Stateless Cloud Function (API/Microservice) + +Host the trajectory parsing logic completely server-side. + +### How it works + +1. **User runs `/resume`**: The CLI scans local `.pb` trajectories. +2. **Cloud Request**: The CLI sends a + `POST https://api.exafunction.com/v1/teleport` request. The body contains + the binary payload (and maybe an authorization token/key). +3. **Cloud Conversion**: The cloud function decrypts, parses, and formats the + trajectory. +4. **Response**: The API responds with the `ConversationRecord` JSON. + +### Effort & Trade-offs + +- **Effort**: Medium. Requires setting up a secured API endpoint, likely using + Google Cloud Functions or AWS Lambda. +- **Pros**: Completely decouples schema and keys from the end-user's device + entirely. The conversion environment is completely under Exafunction's + control. +- **Cons**: Network roundtrip latency whenever the user converts a session. + Involves sending binary trajectory files (potentially containing source code + snippets) over the network, which could be a privacy concern for some users. + +--- + +## Option 3: Remote "Plugin" Loader (WASM / Transient Javascript) + +The CLI downloads the interpreter on-demand, or runs it inside an isolated +Sandbox (like WebAssembly). + +### How it works + +1. **On-Demand Download**: When `/resume` is first used, the CLI downloads a + private, versioned conversion payload (e.g., `teleporter_v1.2.3.wasm` or an + obfuscated JS bundle) from a secure URL and caches it locally. +2. **Local Execution**: It runs this script locally to perform decryption and + conversion. +3. It keeps the schema out of the open-source CLI bundle, though a determined + user could still reverse engineer the WASM/JS if they inspect their local + caches. + +### Effort & Trade-offs + +- **Effort**: Medium-High. Requires robust WebAssembly compilation from the + Protobuf definitions, or a dynamic code loading execution chain in Node.js + that doesn't trigger security flags. +- **Pros**: Speed of local processing, decoupled from the main CLI installation. +- **Cons**: Adds complexity to the bundle distribution system. +- **Cons (Security)**: A determined user could reverse engineer the cached + WebAssembly or obfuscated JS bundle to reconstruct the JetSki Protobuf schema, + breaking the logic confidentiality requirement. + +--- + +## Option 4: Remote MCP Server (via SSE) + +A remote MCP server would bridge the user's local trajectory files with +Exafunction's parser inside a totally separate host. + +### How it works + +- We run an SSE MCP service on the Exafunction side. +- Instead of using stdio transport from a local background process (Option 1), + the Gemini CLI establishes an SSE connection to + `https://mcp.exafunction.com/sse`. +- The local CLI still queries the filesystem for the `.pb` trajectories. +- It invokes a remote tool: `parse_trajectory(trajectoryPb: string)` and passes + the local protobuf string as the request argument. +- The remote server unmarshalls, decrypts, and maps the proprietary protobufs + into the standard JSON response, which the CLI renders natively. + +### Effort & Trade-offs + +- **Effort**: Medium. Gemini CLI already supports SSE network transports for + MCP. You would need to host an MCP SSE server remotely. +- **Pros**: Proprietary mapping logic, credentials, and Protobuf schemas are + hosted totally remote and are decoupled from JetSki OR the CLI. Since it uses + standard MCP, it requires absolutely no specialized HTTP routing or bespoke + protocol headers in the CLI because Gemini naturally maps arguments + dynamically already. +- **Cons**: High network latency (sending large Protobuf strings back and + forth), privacy concerns because user code trajectories are being transmitted + remotely. + +--- + +## Recommendation + +**Option 4 (Remote MCP)** or **Option 2 (Cloud Function Isolation)** is the +recommended production approach to ensure strict confidentiality. + +By keeping the proprietary deserialization binary completely remote behind an +authentication layer, Gemini CLI ensures that end users cannot observe execution +state, trace unmarshalled arrays, or scrape proprietary JetSki Protobuf +primitives natively. If removing heavy network latency is the highest priority, +**Option 1 (JetSki Local MCP)** remains the most effortless and robust path +forward without modifying the open-source CLI distribution.