docs: add documentation for teleportation feature

This commit is contained in:
Sehoon Shon
2026-03-19 00:32:44 -04:00
parent e806947c87
commit a04e8cce43
3 changed files with 501 additions and 0 deletions

View File

@@ -0,0 +1,96 @@
# One-Pager: Session Teleportation (External Tool Resumption)
## 1. Objective
Enable Gemini CLI users to seamlessly resume work sessions (trajectories) that
were originated and executed in external coding tools or IDE extensions. This
"teleportation" of sessions establishes Gemini CLI as a centralized hub for
AI-assisted development across various environments.
## 2. Background & Motivation
Developers frequently switch between multiple tools (e.g., IDE extensions,
web-based coding assistants, and the terminal). Currently, context is
fragmented; a session started in an external tool cannot be natively continued
in Gemini CLI. By allowing Gemini CLI to parse and ingest trajectories from
these external tools, we prevent context loss, avoid duplicate work, and
significantly enhance developer productivity.
## 3. Goals & Non-Goals
**Goals:**
- Provide a standardized abstraction (`TrajectoryProvider`) for fetching
external session histories.
- Implement a core translation layer (`teleporter` module) to map external tool
actions (e.g., `find`, `search_web`, `read_url`, `write_to_file`) to Gemini
CLI's native tool formats.
- Update the `/resume` CLI command to display and filter these external sessions
alongside native ones.
- Support dynamic loading of new trajectory providers via the extension system.
**Non-Goals:**
- Two-way sync: We do not intend to export Gemini CLI sessions back to the
originating external tool in this phase.
- Perfect fidelity for unsupported external tools: Only explicitly supported and
mapped tools will be translated. Others will be gracefully ignored or logged
as plain text.
## 4. Proposed Design / Architecture
The design revolves around a pluggable architecture that decouples the fetching
of external trajectories from the core CLI logic.
### 4.1. `TrajectoryProvider` Interface
A standard contract that external extensions implement. It defines methods for:
- Retrieving a list of available sessions.
- Fetching the full trajectory payload for a specific session ID.
- Providing metadata (display name, icon, source tool name) for the CLI UI.
### 4.2. Core `teleporter` Module
This module acts as the ingestion engine. It performs:
- **Deserialization:** Parsing the raw JSON/format of the external tool.
- **Conversion:** Mapping external tool calls (e.g., an Antigravity `search_web`
call) into the equivalent Gemini CLI tool call format, ensuring the LLM
understands the historical context correctly.
- **Validation:** Ensuring the resulting session history conforms to Gemini
CLI's internal session schema.
### 4.3. UI Integration (`/resume`)
The `SessionBrowser` UI will be updated to handle multiple sources.
- Introduce dynamic tabs or source filters in the TUI (Terminal UI).
- Display metadata (e.g., "Imported from Tool X", timestamps) to help users
distinguish between native and teleported sessions.
### 4.4. Extension Manager Updates
Enhance the existing `extensionLoader` to discover, load, and register plugins
that implement the `TrajectoryProvider` interface dynamically at runtime.
## 5. Rollout Plan
The implementation is broken down into incremental PRs tracked by Epic #22801:
1. **Define Core Interfaces:** Establish `TrajectoryProvider`.
2. **Implement Translation Logic:** Build the `teleporter` module with initial
conversion mappings.
3. **UI Updates:** Hook the providers into the `/resume` interface.
4. **Extension Support:** Enable dynamic loading of providers.
5. **Documentation:** Publish `TELEPORTATION.md` outlining how to build new
providers.
## 6. Security & Privacy
- **Local First:** Trajectories should be read from local filesystem artifacts
or secure local IPC mechanisms where possible, avoiding unnecessary network
calls.
- **Data Sanitization:** The conversion process must not execute any historical
commands; it strictly translates historical records into past-tense context
for the LLM.

View File

@@ -0,0 +1,251 @@
# Trajectory Teleportation: Antigravity to Gemini CLI
This document explains how the Gemini CLI discovers, reads, and converts
Antigravity (Jetski) trajectories to enable session resumption.
## Overview
The teleportation feature allows you to pick up a conversation in the Gemini CLI
that was started in the Antigravity (Jetski) IDE.
## 1. Discovery
The CLI identifies Antigravity sessions by scanning the local filesystem.
- **Storage Location**: `~/.antigravity/conversations`
- **File Format**: Binary Protobuf files with the `.pb` extension.
- **Session IDs**: The filenames (e.g., `f81d4fae-7dec.pb`) serve as the unique
identifiers for resumption.
## 2. Decryption & Parsing
Since Antigravity stores data in a specialized binary format, the CLI uses a
dedicated teleporter bundle:
- **Logic**: `trajectory_teleporter.min.js` (bundled in
`@google/gemini-cli-core`).
- **Process**: The binary `.pb` file is read into a Buffer and passed to the
teleporter's `trajectoryToJson` function, which outputs a standard JavaScript
object.
- **Generation**: To ensure the CLI stays in sync with Jetski's protobuf schema
(`cortex.proto`), this bundle isn't written entirely by hand. It is
auto-generated by compiling the latest schema via `esbuild`. You can rebuild
it anytime by running `npm run build:teleporter` from the CLI root. _(See
[How to Regenerate the Bundle](#how-to-regenerate-the-bundle) for exact
steps)._
## 3. Workspace Identification
To filter sessions by the user's active workspace, the CLI attempts to read the
target workspace from the trajectory.
1. **Primary**: It reads `metadata.workspaces[0].workspaceFolderAbsoluteUri`
from the top-level trajectory metadata. This is the authoritative data
populated by the Jetski Go backend.
2. **Fallback**: For older trajectories without top-level metadata, it attempts
to extract `workspaceUri` from the first user input's
`activeUserState.activeDocument` context.
## 4. Conversion Logic
The conversion layer
([converter.ts](file:///Users/sshon/developments/gemini-cli/packages/core/src/teleportation/converter.ts))
translates the technical "Steps" of an Antigravity trajectory into the CLI's
`ConversationRecord` format:
- **User Input**: Maps `CORTEX_STEP_TYPE_USER_INPUT` (type 14) to `user`
messages.
- **Model Responses**: Maps `CORTEX_STEP_TYPE_PLANNER_RESPONSE` (type 15) to
`gemini` messages.
- **Thoughts & Reasoning**: Extracts reasoning content from the Antigravity step
and populates the `thoughts` array in the CLI record, preserving the model's
logic.
- **Tool Calls**: Maps Antigravity tool execution steps to CLI `ToolCallRecord`
objects. The integration handles these in three main categories:
| Tool Capability | Gemini CLI Tool | Jetski Protobuf Step | Teleportation Status |
| :------------------- | :------------------------------------- | :---------------------------------------------- | :----------------------- |
| **Read File** | `read_file` | `CORTEX_STEP_TYPE_VIEW_FILE` | ✅ Native Mapping |
| **List Directory** | `ls` | `CORTEX_STEP_TYPE_LIST_DIRECTORY` | ✅ Native Mapping |
| **Search Code** | `grep_search` | `CORTEX_STEP_TYPE_GREP_SEARCH` | ✅ Native Mapping |
| **Edit File** | `replace` | `CORTEX_STEP_TYPE_FILE_CHANGE` | ✅ Native Mapping |
| **Search Web** | `google_web_search` | `CORTEX_STEP_TYPE_SEARCH_WEB` | ✅ Native Mapping |
| **Execute Shell** | `run_shell_command` | `CORTEX_STEP_TYPE_RUN_COMMAND` | ✅ Native Mapping |
| **Browser Agent** | _N/A_ \* | `CORTEX_STEP_TYPE_BROWSER_SUBAGENT` | ❌ Dropped |
| **User Input** | `ask_user` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **Read URL Content** | `web_fetch` | `CORTEX_STEP_TYPE_READ_URL_CONTENT` | ❌ Dropped |
| **Find Files** | `glob` | `CORTEX_STEP_TYPE_FIND` | ❌ Dropped |
| **Write File** | `write_file` | `CORTEX_STEP_TYPE_WRITE_TO_FILE` | ❌ Dropped |
| **Read Many Files** | `read_many_files` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **Write Todos** | `write_todos` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **Save Memory** | `save_memory` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **Docs/Skills** | `get_internal_docs` & `activate_skill` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **Plan Mode** | `enter_plan_mode` & `exit_plan_mode` | `CORTEX_STEP_TYPE_GENERIC` | ⚠️ Generic / MCP |
| **IDE Actions** | _N/A_ | `VIEW_CODE_ITEM`, `LINT_DIFF`, `ADD_ANNOTATION` | 🚫 Jetski Only (Dropped) |
**1. Native Mappings** Antigravity has specific protobuf message types for
common tool calls, which map directly to the CLI's native tools:
- `CORTEX_STEP_TYPE_VIEW_FILE` -> `read_file`
- `CORTEX_STEP_TYPE_LIST_DIRECTORY` -> `ls`
- `CORTEX_STEP_TYPE_GREP_SEARCH` -> `grep_search`
- `CORTEX_STEP_TYPE_RUN_COMMAND` -> `run_shell_command`
- `CORTEX_STEP_TYPE_FILE_CHANGE` -> `replace`
- `CORTEX_STEP_TYPE_BROWSER_SUBAGENT` -> (Dropped)
**2. Generic / MCP Tools**
- Jetski relies heavily on `CORTEX_STEP_TYPE_GENERIC` and
`CORTEX_STEP_TYPE_MCP_TOOL` to route non-native or dynamic tools.
- This is fully supported! The CLI reads the `toolName` and `argsJson` directly
from the generic step payload. For instance, the Jetski `ask_user` tool
natively maps to the CLI's `ask_user` tool, and any custom MCP commands are
preserved as-is.
**3. Unsupported Tools** Many isolated actions, sub-agent tools, and
IDE-specific UI interactions are dropped by the teleporter to maintain strict
CLI compatibility and preserve valid context-window state.
**4. Tool UI & Context Representation** When importing dynamic/generic tools,
the CLI UI uses internal synthesis to properly reflect tool usage and outputs
that the user saw in JetSki without blank rendering:
- **Arguments:** Arguments extracted from `argsJson` are synthesized into the
`.description` field enabling the CLI UI to display the exact call arguments
(e.g., specific files or search strings) below the tool name.
- **Output (Result Display):** Tool outputs, like terminal output payloads or
file text, are iteratively extracted from the trajectory steps and rendered
explicitly using `resultDisplay`.
<details>
<summary><b>Click to view exhaustive list of all 75+ dropped Jetski steps</b></summary>
Any step mapped in `cortex.proto` that isn't functionally replicated by the CLI
is skipped. This includes:
- **Browser UI & Automation**: `CORTEX_STEP_TYPE_BROWSER_CLICK_ELEMENT`,
`CORTEX_STEP_TYPE_BROWSER_MOVE_MOUSE`, `CORTEX_STEP_TYPE_BROWSER_SUBAGENT`,
`CORTEX_STEP_TYPE_EXECUTE_BROWSER_JAVASCRIPT`,
`CORTEX_STEP_TYPE_CAPTURE_BROWSER_SCREENSHOT`,
`CORTEX_STEP_TYPE_READ_BROWSER_PAGE`, `CORTEX_STEP_TYPE_BROWSER_GET_DOM`,
`CORTEX_STEP_TYPE_BROWSER_LIST_NETWORK_REQUESTS`,
`CORTEX_STEP_TYPE_BROWSER_SCROLL_UP`, `CORTEX_STEP_TYPE_OPEN_BROWSER_URL`, and
15+ other browser controls.
- **IDE & UI Actions**: `CORTEX_STEP_TYPE_VIEW_CODE_ITEM`,
`CORTEX_STEP_TYPE_LINT_DIFF`, `CORTEX_STEP_TYPE_ADD_ANNOTATION`,
`CORTEX_STEP_TYPE_VIEW_FILE_OUTLINE`, `CORTEX_STEP_TYPE_CODE_SEARCH`,
`CORTEX_STEP_TYPE_FIND_ALL_REFERENCES`, `CORTEX_STEP_TYPE_CODE_ACTION`.
- **Agent Framework**: `CORTEX_STEP_TYPE_TASK_BOUNDARY`,
`CORTEX_STEP_TYPE_NOTIFY_USER`, `CORTEX_STEP_TYPE_INVOKE_SUBAGENT`,
`CORTEX_STEP_TYPE_CHECKPOINT`, `CORTEX_STEP_TYPE_EPHEMERAL_MESSAGE`,
`CORTEX_STEP_TYPE_COMMAND_STATUS`, `CORTEX_STEP_TYPE_SEND_COMMAND_INPUT`.
- **Knowledge & Workspace**: `CORTEX_STEP_TYPE_KNOWLEDGE_GENERATION`,
`CORTEX_STEP_TYPE_KI_INSERTION`, `CORTEX_STEP_TYPE_TRAJECTORY_SEARCH`,
`CORTEX_STEP_TYPE_MQUERY`, `CORTEX_STEP_TYPE_INTERNAL_SEARCH`,
`CORTEX_STEP_TYPE_LIST_RESOURCES`, `CORTEX_STEP_TYPE_WORKSPACE_API`.
- **Misc Commands**: `CORTEX_STEP_TYPE_GIT_COMMIT`,
`CORTEX_STEP_TYPE_GENERATE_IMAGE`, `CORTEX_STEP_TYPE_COMPILE`,
`CORTEX_STEP_TYPE_CLIPBOARD`, `CORTEX_STEP_TYPE_ERROR_MESSAGE`, etc.
</details>
## 5. Session Resumption
Once converted:
1. The record is injected into the CLI's `ChatRecordingService`.
2. Users can continue the conversation seamlessly via the `/resume` command.
## Maintenance & Updates
You are correct that if Antigravity's Protobuf definitions change, the
`trajectory_teleporter.min.js` bundle will need to be updated to maintain
compatibility.
### When to Update
- If new step types are added to Antigravity that the CLI should support in
`converter.ts`.
- If the binary format of the `.pb` files changes.
- If the encryption key or algorithm is rotated.
### How to Regenerate the Bundle
To keep the CLI up to date, you can run the automated build script:
```bash
npm run build:teleporter
```
This runs `packages/core/scripts/build-teleporter.sh`, which automatically:
1. Navigates to the neighboring `Exafunction` directory.
2. Generates the latest Protobuf JS schemas (`pnpm --dir exa/proto_ts build`).
3. Uses `esbuild` to re-bundle the CLI's teleporter script
(`trajectory_teleporter.ts`) against the fresh schemas.
4. Outputs the new `trajectory_teleporter.min.js` directly into the `gemini-cli`
source tree.
> [!TIP] In the long term, this logic could be moved to a shared NPM package
> published from the Antigravity repository, allowing the Gemini CLI to stay
> updated via a simple `npm update`.
## Productionization Roadmap
To safely and seamlessly bring the Jetski trajectory teleportation feature to a
fully production-ready state in the Gemini CLI, several key areas need to be
addressed:
### 1. Security & Key Management
- **Dynamic Key Exchange:** ✅ The CLI now supports loading encryption keys from
`JETSKI_TELEPORT_KEY` environment variables or a local
`~/.gemini/jetski/key.txt` file.
- **Permission Scoping:** Ensure the CLI enforces the same file-access
permission rules (`file_permission_request`) that Jetski enforces so the AI
doesn't suddenly gain destructive permissions when transitioning to the
terminal.
### 2. Architecture & Build Process Decoupling
- **Trajectory Provider Interface:** ✅ The CLI now uses a generic
`TrajectoryProvider` interface, allowing teleportation logic to be decoupled
into extensions.
- **Shared NPM Package:** Publish the compiled Protobufs and parsing logic as a
private internal package (e.g., `@google/cortex-teleporter`). The Gemini CLI
should simply `npm install` this, rather than generating `.min.js` blobs
manually.
- **Schema Versioning:** Add version checks. If the CLI encounters a trajectory
from a newer version of Jetski with breaking protobuf changes, it should
gracefully prompt the user to update the CLI.
### 3. User Experience (UX)
- **Clear UI Indicators:** ✅ Jetski sessions are now grouped in a dedicated tab
in the `/resume` menu.
- **Missing Context Warnings:** ✅ The UI now synthesizes `description` and
`resultDisplay` for generic tool calls to ensure a smooth conversation flow.
- **Seamless Handoff Prompt:** ✅ The CLI now intelligently prompts the user on
startup if a recent Jetski session is found: _"🛸 You have a recent session in
Antigravity. Type /resume agy:<id> to bring it into the terminal."_
### 4. Data Fidelity & Error Handling
- **Graceful Degradation:** If the CLI fails to parse a specific `generic` step
or misses a tool argument, it shouldn't crash the entire resume process. It
should skip the broken step and append a system message warning the model.
- **File State Synchronization:** If a user made uncommitted file edits in
Jetski (e.g., `fileChange` steps that haven't been saved to disk), the CLI
needs to ensure it doesn't accidentally overwrite or hallucinate disk state.
It might be worth requiring a file save before teleporting.
### 5. Performance
- **Lazy Loading / Streaming:** When populating the list of sessions for
`/resume`, the CLI shouldnt decrypt and parse the _entire_ history of every
file. It should only read the `metadata` headers to render the UI picker, and
only parse the full multi-megabyte `ConversationRecord` when a specific
session is selected.
- **Memory Limits:** Set an upper limit on how many historical tool turns the
CLI loads into memory when teleporting to avoid OOM (Out of Memory) crashes in
Node.js.

View File

@@ -0,0 +1,154 @@
# Remote Teleportation Architecture Proposal
## Objective
Prevent leaking proprietary JetSki Protobuf schemas and decryption keys directly
within the public Gemini CLI bundle. When a user requests to resume a JetSki
trajectory, the CLI will interact with an external, secure, or isolated
converter to fetch the standard `ConversationRecord` JSON.
## Guarantees & Constraints
- No Protobuf definitions are packaged or visible in the Gemini CLI
distribution.
- Telemetry/Decryption keys do not need to be hardcoded in the CLI.
- Conversion remains effortless and instantaneous for the end-user.
- **Strict Logic Confidentiality**: Proprietary conversion logic and Protobuf
schemas cannot be readable or reverse-engineered by the user via local caches
or easily accessible client-side code.
---
## Option 1: Local MCP Server (JetSki Daemon Extension)
Since Gemini CLI already natively supports the **Model Context Protocol (MCP)**,
we can utilize JetSki directly as an MCP provider.
### How it works
The JetSki extension (or its already running background daemon) can expose an
MCP tool, for example, `get_trajectory_history`. When the user runs `/resume`,
Gemini CLI invokes this tool via the standard MCP pipeline.
1. **User runs `/resume`**: Gemini CLI lists available trajectories (it can
list .pb files from the local directory or query the MCP server for a
history list).
2. **Conversation Selection**: The user selects a session.
3. **MCP Tool Call**: Gemini calls the MCP tool
`get_trajectory(filePath: string)`.
4. **Local Conversion**: JetSki decrypts, parses, and returns the strictly
CLI-compliant `HistoryItem` JSON payload over stdio/SSE.
5. **No Overhead**: The CLI injects the payload directly into its state without
ever touching a Protobuf.
### Effort & Trade-offs
- **Effort**: Very Low. Gemini CLI already supports MCP out of the box. No new
endpoints, infrastructure, or routing is needed. We only need to write a small
MCP server module using `@modelcontextprotocol/sdk` inside JetSki.
- **Pros**: Zero remote network latency; highly secure (stays on local machine);
incredibly seamless.
- **Cons**: Requires the JetSki service to be installed and running in the
background.
---
## Option 2: Stateless Cloud Function (API/Microservice)
Host the trajectory parsing logic completely server-side.
### How it works
1. **User runs `/resume`**: The CLI scans local `.pb` trajectories.
2. **Cloud Request**: The CLI sends a
`POST https://api.exafunction.com/v1/teleport` request. The body contains
the binary payload (and maybe an authorization token/key).
3. **Cloud Conversion**: The cloud function decrypts, parses, and formats the
trajectory.
4. **Response**: The API responds with the `ConversationRecord` JSON.
### Effort & Trade-offs
- **Effort**: Medium. Requires setting up a secured API endpoint, likely using
Google Cloud Functions or AWS Lambda.
- **Pros**: Completely decouples schema and keys from the end-user's device
entirely. The conversion environment is completely under Exafunction's
control.
- **Cons**: Network roundtrip latency whenever the user converts a session.
Involves sending binary trajectory files (potentially containing source code
snippets) over the network, which could be a privacy concern for some users.
---
## Option 3: Remote "Plugin" Loader (WASM / Transient Javascript)
The CLI downloads the interpreter on-demand, or runs it inside an isolated
Sandbox (like WebAssembly).
### How it works
1. **On-Demand Download**: When `/resume` is first used, the CLI downloads a
private, versioned conversion payload (e.g., `teleporter_v1.2.3.wasm` or an
obfuscated JS bundle) from a secure URL and caches it locally.
2. **Local Execution**: It runs this script locally to perform decryption and
conversion.
3. It keeps the schema out of the open-source CLI bundle, though a determined
user could still reverse engineer the WASM/JS if they inspect their local
caches.
### Effort & Trade-offs
- **Effort**: Medium-High. Requires robust WebAssembly compilation from the
Protobuf definitions, or a dynamic code loading execution chain in Node.js
that doesn't trigger security flags.
- **Pros**: Speed of local processing, decoupled from the main CLI installation.
- **Cons**: Adds complexity to the bundle distribution system.
- **Cons (Security)**: A determined user could reverse engineer the cached
WebAssembly or obfuscated JS bundle to reconstruct the JetSki Protobuf schema,
breaking the logic confidentiality requirement.
---
## Option 4: Remote MCP Server (via SSE)
A remote MCP server would bridge the user's local trajectory files with
Exafunction's parser inside a totally separate host.
### How it works
- We run an SSE MCP service on the Exafunction side.
- Instead of using stdio transport from a local background process (Option 1),
the Gemini CLI establishes an SSE connection to
`https://mcp.exafunction.com/sse`.
- The local CLI still queries the filesystem for the `.pb` trajectories.
- It invokes a remote tool: `parse_trajectory(trajectoryPb: string)` and passes
the local protobuf string as the request argument.
- The remote server unmarshalls, decrypts, and maps the proprietary protobufs
into the standard JSON response, which the CLI renders natively.
### Effort & Trade-offs
- **Effort**: Medium. Gemini CLI already supports SSE network transports for
MCP. You would need to host an MCP SSE server remotely.
- **Pros**: Proprietary mapping logic, credentials, and Protobuf schemas are
hosted totally remote and are decoupled from JetSki OR the CLI. Since it uses
standard MCP, it requires absolutely no specialized HTTP routing or bespoke
protocol headers in the CLI because Gemini naturally maps arguments
dynamically already.
- **Cons**: High network latency (sending large Protobuf strings back and
forth), privacy concerns because user code trajectories are being transmitted
remotely.
---
## Recommendation
**Option 4 (Remote MCP)** or **Option 2 (Cloud Function Isolation)** is the
recommended production approach to ensure strict confidentiality.
By keeping the proprietary deserialization binary completely remote behind an
authentication layer, Gemini CLI ensures that end users cannot observe execution
state, trace unmarshalled arrays, or scrape proprietary JetSki Protobuf
primitives natively. If removing heavy network latency is the highest priority,
**Option 1 (JetSki Local MCP)** remains the most effortless and robust path
forward without modifying the open-source CLI distribution.