mirror of
https://github.com/google-gemini/gemini-cli.git
synced 2026-07-31 20:21:01 -07:00
Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 3d76c80da5 | |||
| d034c4dd96 | |||
| f1a3387160 | |||
| 49534209f2 | |||
| 9e7f52b8f5 | |||
| 30e0ab102a | |||
| 2e03e3aed5 |
@@ -334,8 +334,20 @@ jobs:
|
||||
if: "${{ steps.check_evals.outputs.should_run == 'true' }}"
|
||||
env:
|
||||
GEMINI_API_KEY: '${{ secrets.GEMINI_API_KEY }}'
|
||||
GEMINI_MODEL: 'gemini-3-pro-preview'
|
||||
# Disable Vitest internal retries to avoid double-retrying;
|
||||
# custom retry logic is handled in evals/test-helper.ts
|
||||
VITEST_RETRY: 0
|
||||
run: 'npm run test:always_passing_evals'
|
||||
|
||||
- name: 'Upload Reliability Logs'
|
||||
if: "always() && steps.check_evals.outputs.should_run == 'true'"
|
||||
uses: 'actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02' # ratchet:actions/upload-artifact@v4
|
||||
with:
|
||||
name: 'eval-logs-${{ github.run_id }}-${{ github.run_attempt }}'
|
||||
path: 'evals/logs/api-reliability.jsonl'
|
||||
retention-days: 7
|
||||
|
||||
e2e:
|
||||
name: 'E2E'
|
||||
if: |
|
||||
|
||||
@@ -61,6 +61,8 @@ jobs:
|
||||
GEMINI_MODEL: '${{ matrix.model }}'
|
||||
RUN_EVALS: "${{ github.event.inputs.run_all != 'false' }}"
|
||||
TEST_NAME_PATTERN: '${{ github.event.inputs.test_name_pattern }}'
|
||||
# Disable Vitest internal retries to avoid double-retrying;
|
||||
# custom retry logic is handled in evals/test-helper.ts
|
||||
VITEST_RETRY: 0
|
||||
run: |
|
||||
CMD="npm run test:all_evals"
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
# Gemini CLI examples
|
||||
|
||||
Gemini CLI helps you automate common engineering tasks by combining AI reasoning
|
||||
with local system tools. This document provides examples of how to use the CLI
|
||||
for file management, code analysis, and data transformation.
|
||||
|
||||
<!-- prettier-ignore -->
|
||||
> [!NOTE]
|
||||
> These examples demonstrate potential capabilities. Your actual
|
||||
> results can vary based on the model used and your project environment.
|
||||
|
||||
## Rename your photographs based on content
|
||||
|
||||
You can use Gemini CLI to automate file management tasks that require visual
|
||||
analysis. In this example, Gemini CLI renames images based on their actual
|
||||
subject matter.
|
||||
|
||||
Scenario: You have a folder containing the following files:
|
||||
|
||||
```bash
|
||||
photos/photo1.png
|
||||
photos/photo2.png
|
||||
photos/photo3.png
|
||||
```
|
||||
|
||||
Give Gemini the following prompt:
|
||||
|
||||
```cli
|
||||
Rename the photos in my "photos" directory based on their contents.
|
||||
```
|
||||
|
||||
Result: Gemini asks for permission to rename your files.
|
||||
|
||||
Select **Allow once** and your files are renamed:
|
||||
|
||||
```bash
|
||||
photos/yellow_flowers.png
|
||||
photos/antique_dresser.png
|
||||
photos/green_android_robot.png
|
||||
```
|
||||
|
||||
## Explain a repository by reading its code
|
||||
|
||||
Gemini CLI is effective for rapid codebase exploration. The following example
|
||||
shows how to ask Gemini CLI to fetch, analyze, and summarize a remote project.
|
||||
|
||||
Scenario: You want to understand how a popular open-source utility works by
|
||||
inspecting its code, not just its README.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Clone the 'chalk' repository from https://github.com/chalk/chalk, read its key source files, and explain how it works.
|
||||
```
|
||||
|
||||
Result: Gemini performs a sequence of actions to answer your request.
|
||||
|
||||
1. First, it asks for permission to run `git clone` to download the repository.
|
||||
2. Next, it finds the important source files and asks for permission to read
|
||||
them.
|
||||
3. Finally, after analyzing the code, it provides a summary.
|
||||
|
||||
Gemini CLI returns an explanation based on the actual source code:
|
||||
|
||||
```markdown
|
||||
The `chalk` library is a popular npm package for styling terminal output with
|
||||
colors. After analyzing the source code, here's how it works:
|
||||
|
||||
- **Core functionality:** The main file sets up a chainable API. Each color or
|
||||
modifier (like `bold` or `italic`) is a getter that appends the corresponding
|
||||
ANSI escape code to an internal stack.
|
||||
|
||||
- **Chainable API:** When you call `chalk.red.bold('Hello')`, you are accessing
|
||||
getters. The `red` getter adds the red color code, and the `bold` getter adds
|
||||
the bold code.
|
||||
|
||||
- **Output generation:** When the chain is treated as a string (e.g., in
|
||||
`console.log`), a final `toString()` method is called. This method joins all
|
||||
the stored ANSI codes, wraps them around the input string ('Hello'), and adds
|
||||
a reset code at the end. This produces the final, styled string that the
|
||||
terminal can render.
|
||||
```
|
||||
|
||||
## Combine two spreadsheets into one spreadsheet
|
||||
|
||||
Gemini CLI can process and transform data across multiple files. Use this
|
||||
capability to merge reports or reformat data sets without manual copying.
|
||||
|
||||
Scenario: You have two .csv files: `Revenue - 2023.csv` and
|
||||
`Revenue - 2024.csv`. Each file contains monthly revenue figures.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Combine the two .csv files into a single .csv file, with each year a different column.
|
||||
```
|
||||
|
||||
Result: Gemini CLI reads each file and then asks for permission to write a new
|
||||
file. Provide your permission and Gemini CLI provides the combined data:
|
||||
|
||||
```csv
|
||||
Month,2023,2024
|
||||
January,0,1000
|
||||
February,0,1200
|
||||
March,0,2400
|
||||
April,900,500
|
||||
May,1000,800
|
||||
June,1000,900
|
||||
July,1200,1000
|
||||
August,1800,400
|
||||
September,2000,2000
|
||||
October,2400,3400
|
||||
November,3400,1800
|
||||
December,2100,9000
|
||||
```
|
||||
|
||||
## Run unit tests
|
||||
|
||||
Gemini CLI can generate boilerplate code and tests based on your existing
|
||||
implementation. This example demonstrates how to request code coverage for a
|
||||
JavaScript component.
|
||||
|
||||
Scenario: You've written a simple login page. You wish to write unit tests to
|
||||
ensure that your login page has code coverage.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Write unit tests for Login.js.
|
||||
```
|
||||
|
||||
Result: Gemini CLI asks for permission to write a new file and creates a test
|
||||
for your login page.
|
||||
|
||||
## Next steps
|
||||
|
||||
- Follow the [File management](../cli/tutorials/file-management.md) guide to
|
||||
start working with your codebase.
|
||||
- Follow the [Quickstart](./index.md) to start your first session.
|
||||
- See the [Cheatsheet](../cli/cli-reference.md) for a quick reference of
|
||||
available commands.
|
||||
+127
-1
@@ -62,7 +62,133 @@ Once installed and authenticated, you can start using Gemini CLI by issuing
|
||||
commands and prompts in your terminal. Ask it to generate code, explain files,
|
||||
and more.
|
||||
|
||||
To explore the power of Gemini CLI, see [Gemini CLI examples](./examples.md).
|
||||
<!-- prettier-ignore -->
|
||||
> [!NOTE]
|
||||
> These examples demonstrate potential capabilities. Your actual
|
||||
> results can vary based on the model used and your project environment.
|
||||
|
||||
### Rename your photographs based on content
|
||||
|
||||
You can use Gemini CLI to automate file management tasks that require visual
|
||||
analysis. In this example, Gemini CLI renames images based on their actual
|
||||
subject matter.
|
||||
|
||||
Scenario: You have a folder containing the following files:
|
||||
|
||||
```bash
|
||||
photos/photo1.png
|
||||
photos/photo2.png
|
||||
photos/photo3.png
|
||||
```
|
||||
|
||||
Give Gemini the following prompt:
|
||||
|
||||
```cli
|
||||
Rename the photos in my "photos" directory based on their contents.
|
||||
```
|
||||
|
||||
Result: Gemini asks for permission to rename your files.
|
||||
|
||||
Select **Allow once** and your files are renamed:
|
||||
|
||||
```bash
|
||||
photos/yellow_flowers.png
|
||||
photos/antique_dresser.png
|
||||
photos/green_android_robot.png
|
||||
```
|
||||
|
||||
### Explain a repository by reading its code
|
||||
|
||||
Gemini CLI is effective for rapid codebase exploration. The following example
|
||||
shows how to ask Gemini CLI to fetch, analyze, and summarize a remote project.
|
||||
|
||||
Scenario: You want to understand how a popular open-source utility works by
|
||||
inspecting its code, not just its README.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Clone the 'chalk' repository from https://github.com/chalk/chalk, read its key source files, and explain how it works.
|
||||
```
|
||||
|
||||
Result: Gemini performs a sequence of actions to answer your request.
|
||||
|
||||
1. First, it asks for permission to run `git clone` to download the repository.
|
||||
2. Next, it finds the important source files and asks for permission to read
|
||||
them.
|
||||
3. Finally, after analyzing the code, it provides a summary.
|
||||
|
||||
Gemini CLI returns an explanation based on the actual source code:
|
||||
|
||||
```markdown
|
||||
The `chalk` library is a popular npm package for styling terminal output with
|
||||
colors. After analyzing the source code, here's how it works:
|
||||
|
||||
- **Core functionality:** The main file sets up a chainable API. Each color or
|
||||
modifier (like `bold` or `italic`) is a getter that appends the corresponding
|
||||
ANSI escape code to an internal stack.
|
||||
|
||||
- **Chainable API:** When you call `chalk.red.bold('Hello')`, you are accessing
|
||||
getters. The `red` getter adds the red color code, and the `bold` getter adds
|
||||
the bold code.
|
||||
|
||||
- **Output generation:** When the chain is treated as a string (e.g., in
|
||||
`console.log`), a final `toString()` method is called. This method joins all
|
||||
the stored ANSI codes, wraps them around the input string ('Hello'), and adds
|
||||
a reset code at the end. This produces the final, styled string that the
|
||||
terminal can render.
|
||||
```
|
||||
|
||||
### Combine two spreadsheets into one spreadsheet
|
||||
|
||||
Gemini CLI can process and transform data across multiple files. Use this
|
||||
capability to merge reports or reformat data sets without manual copying.
|
||||
|
||||
Scenario: You have two .csv files: `Revenue - 2023.csv` and
|
||||
`Revenue - 2024.csv`. Each file contains monthly revenue figures.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Combine the two .csv files into a single .csv file, with each year a different column.
|
||||
```
|
||||
|
||||
Result: Gemini CLI reads each file and then asks for permission to write a new
|
||||
file. Provide your permission and Gemini CLI provides the combined data:
|
||||
|
||||
```csv
|
||||
Month,2023,2024
|
||||
January,0,1000
|
||||
February,0,1200
|
||||
March,0,2400
|
||||
April,900,500
|
||||
May,1000,800
|
||||
June,1000,900
|
||||
July,1200,1000
|
||||
August,1800,400
|
||||
September,2000,2000
|
||||
October,2400,3400
|
||||
November,3400,1800
|
||||
December,2100,9000
|
||||
```
|
||||
|
||||
### Run unit tests
|
||||
|
||||
Gemini CLI can generate boilerplate code and tests based on your existing
|
||||
implementation. This example demonstrates how to request code coverage for a
|
||||
JavaScript component.
|
||||
|
||||
Scenario: You've written a simple login page. You wish to write unit tests to
|
||||
ensure that your login page has code coverage.
|
||||
|
||||
Give Gemini CLI the following prompt:
|
||||
|
||||
```cli
|
||||
Write unit tests for Login.js.
|
||||
```
|
||||
|
||||
Result: Gemini CLI asks for permission to write a new file and creates a test
|
||||
for your login page.
|
||||
|
||||
## Check usage and quota
|
||||
|
||||
|
||||
@@ -19,8 +19,6 @@ Jump in to Gemini CLI.
|
||||
on your system.
|
||||
- **[Authentication](./get-started/authentication.md):** Setup instructions for
|
||||
personal and enterprise accounts.
|
||||
- **[Examples](./get-started/examples.md):** Practical examples of Gemini CLI in
|
||||
action.
|
||||
- **[CLI cheatsheet](./cli/cli-reference.md):** A quick reference for common
|
||||
commands and options.
|
||||
- **[Gemini 3 on Gemini CLI](./get-started/gemini-3.md):** Learn about Gemini 3
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
"/docs/faq": "/docs/resources/faq",
|
||||
"/docs/get-started/configuration": "/docs/reference/configuration",
|
||||
"/docs/get-started/configuration-v1": "/docs/reference/configuration",
|
||||
"/docs/get-started/examples": "/docs/get-started/index",
|
||||
"/docs/index": "/docs",
|
||||
"/docs/quota-and-pricing": "/docs/resources/quota-and-pricing",
|
||||
"/docs/tos-privacy": "/docs/resources/tos-privacy",
|
||||
|
||||
@@ -12,7 +12,6 @@
|
||||
"label": "Authentication",
|
||||
"slug": "docs/get-started/authentication"
|
||||
},
|
||||
{ "label": "Examples", "slug": "docs/get-started/examples" },
|
||||
{ "label": "CLI cheatsheet", "slug": "docs/cli/cli-reference" },
|
||||
{
|
||||
"label": "Gemini 3 on Gemini CLI",
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { internalEvalTest } from './test-helper.js';
|
||||
import { TestRig } from '@google/gemini-cli-test-utils';
|
||||
|
||||
// Mock TestRig to control API success/failure
|
||||
vi.mock('@google/gemini-cli-test-utils', () => {
|
||||
return {
|
||||
TestRig: vi.fn().mockImplementation(() => ({
|
||||
setup: vi.fn(),
|
||||
run: vi.fn(),
|
||||
cleanup: vi.fn(),
|
||||
readToolLogs: vi.fn().mockReturnValue([]),
|
||||
_lastRunStderr: '',
|
||||
})),
|
||||
};
|
||||
});
|
||||
|
||||
describe('evalTest reliability logic', () => {
|
||||
const LOG_DIR = path.resolve(process.cwd(), 'evals/logs');
|
||||
const RELIABILITY_LOG = path.join(LOG_DIR, 'api-reliability.jsonl');
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
if (fs.existsSync(RELIABILITY_LOG)) {
|
||||
fs.unlinkSync(RELIABILITY_LOG);
|
||||
}
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (fs.existsSync(RELIABILITY_LOG)) {
|
||||
fs.unlinkSync(RELIABILITY_LOG);
|
||||
}
|
||||
});
|
||||
|
||||
it('should retry 3 times on 500 INTERNAL error and then SKIP', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
|
||||
// Simulate permanent 500 error
|
||||
mockRig.run.mockRejectedValue(new Error('status: INTERNAL - API Down'));
|
||||
|
||||
// Execute the test function directly
|
||||
await internalEvalTest({
|
||||
name: 'test-api-failure',
|
||||
prompt: 'do something',
|
||||
assert: async () => {},
|
||||
});
|
||||
|
||||
// Verify retries: 1 initial + 3 retries = 4 setups/runs
|
||||
expect(mockRig.run).toHaveBeenCalledTimes(4);
|
||||
|
||||
// Verify log content
|
||||
const logContent = fs
|
||||
.readFileSync(RELIABILITY_LOG, 'utf-8')
|
||||
.trim()
|
||||
.split('\n');
|
||||
expect(logContent.length).toBe(4);
|
||||
|
||||
const entries = logContent.map((line) => JSON.parse(line));
|
||||
expect(entries[0].status).toBe('RETRY');
|
||||
expect(entries[0].attempt).toBe(0);
|
||||
expect(entries[3].status).toBe('SKIP');
|
||||
expect(entries[3].attempt).toBe(3);
|
||||
expect(entries[3].testName).toBe('test-api-failure');
|
||||
});
|
||||
|
||||
it('should fail immediately on non-500 errors (like assertion failures)', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
|
||||
// Simulate a real logic error/bug
|
||||
mockRig.run.mockResolvedValue('Success');
|
||||
const assertError = new Error('Assertion failed: expected foo to be bar');
|
||||
|
||||
// Expect the test function to throw immediately
|
||||
await expect(
|
||||
internalEvalTest({
|
||||
name: 'test-logic-failure',
|
||||
prompt: 'do something',
|
||||
assert: async () => {
|
||||
throw assertError;
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow('Assertion failed');
|
||||
|
||||
// Verify NO retries: only 1 attempt
|
||||
expect(mockRig.run).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Verify NO reliability log was created (it's not an API error)
|
||||
expect(fs.existsSync(RELIABILITY_LOG)).toBe(false);
|
||||
});
|
||||
|
||||
it('should recover if a retry succeeds', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
|
||||
// Fail once, then succeed
|
||||
mockRig.run
|
||||
.mockRejectedValueOnce(new Error('status: INTERNAL'))
|
||||
.mockResolvedValueOnce('Success');
|
||||
|
||||
await internalEvalTest({
|
||||
name: 'test-recovery',
|
||||
prompt: 'do something',
|
||||
assert: async () => {},
|
||||
});
|
||||
|
||||
// Ran twice: initial (fail) + retry 1 (success)
|
||||
expect(mockRig.run).toHaveBeenCalledTimes(2);
|
||||
|
||||
// Log should only have the one RETRY entry
|
||||
const logContent = fs
|
||||
.readFileSync(RELIABILITY_LOG, 'utf-8')
|
||||
.trim()
|
||||
.split('\n');
|
||||
expect(logContent.length).toBe(1);
|
||||
expect(JSON.parse(logContent[0]).status).toBe('RETRY');
|
||||
});
|
||||
|
||||
it('should retry 3 times on 503 UNAVAILABLE error and then SKIP', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
|
||||
// Simulate permanent 503 error
|
||||
mockRig.run.mockRejectedValue(
|
||||
new Error('status: UNAVAILABLE - Service Busy'),
|
||||
);
|
||||
|
||||
await internalEvalTest({
|
||||
name: 'test-api-503',
|
||||
prompt: 'do something',
|
||||
assert: async () => {},
|
||||
});
|
||||
|
||||
expect(mockRig.run).toHaveBeenCalledTimes(4);
|
||||
|
||||
const logContent = fs
|
||||
.readFileSync(RELIABILITY_LOG, 'utf-8')
|
||||
.trim()
|
||||
.split('\n');
|
||||
const entries = logContent.map((line) => JSON.parse(line));
|
||||
expect(entries[0].errorCode).toBe('503');
|
||||
expect(entries[3].status).toBe('SKIP');
|
||||
});
|
||||
|
||||
it('should throw if an absolute path is used in files', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
mockRig.testDir = path.resolve(process.cwd(), 'test-dir-tmp');
|
||||
if (!fs.existsSync(mockRig.testDir)) {
|
||||
fs.mkdirSync(mockRig.testDir, { recursive: true });
|
||||
}
|
||||
|
||||
try {
|
||||
await expect(
|
||||
internalEvalTest({
|
||||
name: 'test-absolute-path',
|
||||
prompt: 'do something',
|
||||
files: {
|
||||
'/etc/passwd': 'hacked',
|
||||
},
|
||||
assert: async () => {},
|
||||
}),
|
||||
).rejects.toThrow('Invalid file path in test case: /etc/passwd');
|
||||
} finally {
|
||||
if (fs.existsSync(mockRig.testDir)) {
|
||||
fs.rmSync(mockRig.testDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('should throw if directory traversal is detected in files', async () => {
|
||||
const mockRig = new TestRig() as any;
|
||||
(TestRig as any).mockReturnValue(mockRig);
|
||||
mockRig.testDir = path.resolve(process.cwd(), 'test-dir-tmp');
|
||||
|
||||
// Create a mock test-dir
|
||||
if (!fs.existsSync(mockRig.testDir)) {
|
||||
fs.mkdirSync(mockRig.testDir, { recursive: true });
|
||||
}
|
||||
|
||||
try {
|
||||
await expect(
|
||||
internalEvalTest({
|
||||
name: 'test-traversal',
|
||||
prompt: 'do something',
|
||||
files: {
|
||||
'../sensitive.txt': 'hacked',
|
||||
},
|
||||
assert: async () => {},
|
||||
}),
|
||||
).rejects.toThrow('Invalid file path in test case: ../sensitive.txt');
|
||||
} finally {
|
||||
if (fs.existsSync(mockRig.testDir)) {
|
||||
fs.rmSync(mockRig.testDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
+171
-71
@@ -39,87 +39,34 @@ export * from '@google/gemini-cli-test-utils';
|
||||
export type EvalPolicy = 'ALWAYS_PASSES' | 'USUALLY_PASSES';
|
||||
|
||||
export function evalTest(policy: EvalPolicy, evalCase: EvalCase) {
|
||||
const fn = async () => {
|
||||
runEval(
|
||||
policy,
|
||||
evalCase.name,
|
||||
() => internalEvalTest(evalCase),
|
||||
evalCase.timeout,
|
||||
);
|
||||
}
|
||||
|
||||
export async function internalEvalTest(evalCase: EvalCase) {
|
||||
const maxRetries = 3;
|
||||
let attempt = 0;
|
||||
|
||||
while (attempt <= maxRetries) {
|
||||
const rig = new TestRig();
|
||||
const { logDir, sanitizedName } = await prepareLogDir(evalCase.name);
|
||||
const activityLogFile = path.join(logDir, `${sanitizedName}.jsonl`);
|
||||
const logFile = path.join(logDir, `${sanitizedName}.log`);
|
||||
let isSuccess = false;
|
||||
|
||||
try {
|
||||
rig.setup(evalCase.name, evalCase.params);
|
||||
|
||||
// Symlink node modules to reduce the amount of time needed to
|
||||
// bootstrap test projects.
|
||||
symlinkNodeModules(rig.testDir || '');
|
||||
|
||||
if (evalCase.files) {
|
||||
const acknowledgedAgents: Record<string, Record<string, string>> = {};
|
||||
const projectRoot = fs.realpathSync(rig.testDir!);
|
||||
|
||||
for (const [filePath, content] of Object.entries(evalCase.files)) {
|
||||
const fullPath = path.join(rig.testDir!, filePath);
|
||||
fs.mkdirSync(path.dirname(fullPath), { recursive: true });
|
||||
fs.writeFileSync(fullPath, content);
|
||||
|
||||
// If it's an agent file, calculate hash for acknowledgement
|
||||
if (
|
||||
filePath.startsWith('.gemini/agents/') &&
|
||||
filePath.endsWith('.md')
|
||||
) {
|
||||
const hash = crypto
|
||||
.createHash('sha256')
|
||||
.update(content)
|
||||
.digest('hex');
|
||||
|
||||
try {
|
||||
const agentDefs = await parseAgentMarkdown(fullPath, content);
|
||||
if (agentDefs.length > 0) {
|
||||
const agentName = agentDefs[0].name;
|
||||
if (!acknowledgedAgents[projectRoot]) {
|
||||
acknowledgedAgents[projectRoot] = {};
|
||||
}
|
||||
acknowledgedAgents[projectRoot][agentName] = hash;
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
`Failed to parse agent for test acknowledgement: ${filePath}`,
|
||||
error,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Write acknowledged_agents.json to the home directory
|
||||
if (Object.keys(acknowledgedAgents).length > 0) {
|
||||
const ackPath = path.join(
|
||||
rig.homeDir!,
|
||||
'.gemini',
|
||||
'acknowledgments',
|
||||
'agents.json',
|
||||
);
|
||||
fs.mkdirSync(path.dirname(ackPath), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
ackPath,
|
||||
JSON.stringify(acknowledgedAgents, null, 2),
|
||||
);
|
||||
}
|
||||
|
||||
const execOptions = { cwd: rig.testDir!, stdio: 'inherit' as const };
|
||||
execSync('git init', execOptions);
|
||||
execSync('git config user.email "test@example.com"', execOptions);
|
||||
execSync('git config user.name "Test User"', execOptions);
|
||||
|
||||
// Temporarily disable the interactive editor and git pager
|
||||
// to avoid hanging the tests. It seems the the agent isn't
|
||||
// consistently honoring the instructions to avoid interactive
|
||||
// commands.
|
||||
execSync('git config core.editor "true"', execOptions);
|
||||
execSync('git config core.pager "cat"', execOptions);
|
||||
execSync('git config commit.gpgsign false', execOptions);
|
||||
execSync('git add .', execOptions);
|
||||
execSync('git commit --allow-empty -m "Initial commit"', execOptions);
|
||||
await setupTestFiles(rig, evalCase.files);
|
||||
}
|
||||
|
||||
symlinkNodeModules(rig.testDir || '');
|
||||
|
||||
// If messages are provided, write a session file so --resume can load it.
|
||||
let sessionId: string | undefined;
|
||||
if (evalCase.messages) {
|
||||
@@ -188,6 +135,37 @@ export function evalTest(policy: EvalPolicy, evalCase: EvalCase) {
|
||||
|
||||
await evalCase.assert(rig, result);
|
||||
isSuccess = true;
|
||||
return; // Success! Exit the retry loop.
|
||||
} catch (error: unknown) {
|
||||
const errorMessage =
|
||||
error instanceof Error ? error.message : String(error);
|
||||
const errorCode = getApiErrorCode(errorMessage);
|
||||
|
||||
if (errorCode) {
|
||||
const status = attempt < maxRetries ? 'RETRY' : 'SKIP';
|
||||
logReliabilityEvent(
|
||||
evalCase.name,
|
||||
attempt,
|
||||
status,
|
||||
errorCode,
|
||||
errorMessage,
|
||||
);
|
||||
|
||||
if (attempt < maxRetries) {
|
||||
attempt++;
|
||||
console.warn(
|
||||
`[Eval] Attempt ${attempt} failed with ${errorCode} Error. Retrying...`,
|
||||
);
|
||||
continue; // Retry
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`[Eval] '${evalCase.name}' failed after ${maxRetries} retries due to persistent API errors. Skipping failure to avoid blocking PR.`,
|
||||
);
|
||||
return; // Gracefully exit without failing the test
|
||||
}
|
||||
|
||||
throw error; // Real failure
|
||||
} finally {
|
||||
if (isSuccess) {
|
||||
await fs.promises.unlink(activityLogFile).catch((err) => {
|
||||
@@ -206,9 +184,131 @@ export function evalTest(policy: EvalPolicy, evalCase: EvalCase) {
|
||||
);
|
||||
await rig.cleanup();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function getApiErrorCode(message: string): '500' | '503' | undefined {
|
||||
if (
|
||||
message.includes('status: UNAVAILABLE') ||
|
||||
message.includes('code: 503') ||
|
||||
message.includes('Service Unavailable')
|
||||
) {
|
||||
return '503';
|
||||
}
|
||||
if (
|
||||
message.includes('status: INTERNAL') ||
|
||||
message.includes('code: 500') ||
|
||||
message.includes('Internal error encountered')
|
||||
) {
|
||||
return '500';
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Log reliability event for later harvesting.
|
||||
*
|
||||
* Note: Uses synchronous file I/O to ensure the log is persisted even if the
|
||||
* test process is abruptly terminated by a timeout or CI crash. Performance
|
||||
* impact is negligible compared to long-running evaluation tests.
|
||||
*/
|
||||
function logReliabilityEvent(
|
||||
testName: string,
|
||||
attempt: number,
|
||||
status: 'RETRY' | 'SKIP',
|
||||
errorCode: '500' | '503',
|
||||
errorMessage: string,
|
||||
) {
|
||||
const reliabilityLog = {
|
||||
timestamp: new Date().toISOString(),
|
||||
testName,
|
||||
model: process.env.GEMINI_MODEL || 'unknown',
|
||||
attempt,
|
||||
status,
|
||||
errorCode,
|
||||
error: errorMessage,
|
||||
};
|
||||
|
||||
runEval(policy, evalCase.name, fn, evalCase.timeout);
|
||||
try {
|
||||
const relDir = path.resolve(process.cwd(), 'evals/logs');
|
||||
fs.mkdirSync(relDir, { recursive: true });
|
||||
fs.appendFileSync(
|
||||
path.join(relDir, 'api-reliability.jsonl'),
|
||||
JSON.stringify(reliabilityLog) + '\n',
|
||||
);
|
||||
} catch (logError) {
|
||||
console.error('Failed to write reliability log:', logError);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper to setup test files and git repository.
|
||||
*
|
||||
* Note: While this is an async function (due to parseAgentMarkdown), it
|
||||
* intentionally uses synchronous filesystem and child_process operations
|
||||
* for simplicity and to ensure sequential environment preparation.
|
||||
*/
|
||||
async function setupTestFiles(rig: TestRig, files: Record<string, string>) {
|
||||
const acknowledgedAgents: Record<string, Record<string, string>> = {};
|
||||
const projectRoot = fs.realpathSync(rig.testDir!);
|
||||
|
||||
for (const [filePath, content] of Object.entries(files)) {
|
||||
if (filePath.includes('..') || path.isAbsolute(filePath)) {
|
||||
throw new Error(`Invalid file path in test case: ${filePath}`);
|
||||
}
|
||||
const fullPath = path.join(projectRoot, filePath);
|
||||
if (!fullPath.startsWith(projectRoot)) {
|
||||
throw new Error(`Path traversal detected: ${filePath}`);
|
||||
}
|
||||
|
||||
fs.mkdirSync(path.dirname(fullPath), { recursive: true });
|
||||
fs.writeFileSync(fullPath, content);
|
||||
|
||||
if (filePath.startsWith('.gemini/agents/') && filePath.endsWith('.md')) {
|
||||
const hash = crypto.createHash('sha256').update(content).digest('hex');
|
||||
try {
|
||||
const agentDefs = await parseAgentMarkdown(fullPath, content);
|
||||
if (agentDefs.length > 0) {
|
||||
const agentName = agentDefs[0].name;
|
||||
if (!acknowledgedAgents[projectRoot]) {
|
||||
acknowledgedAgents[projectRoot] = {};
|
||||
}
|
||||
acknowledgedAgents[projectRoot][agentName] = hash;
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
`Failed to parse agent for test acknowledgement: ${filePath}`,
|
||||
error,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (Object.keys(acknowledgedAgents).length > 0) {
|
||||
const ackPath = path.join(
|
||||
rig.homeDir!,
|
||||
'.gemini',
|
||||
'acknowledgments',
|
||||
'agents.json',
|
||||
);
|
||||
fs.mkdirSync(path.dirname(ackPath), { recursive: true });
|
||||
fs.writeFileSync(ackPath, JSON.stringify(acknowledgedAgents, null, 2));
|
||||
}
|
||||
|
||||
const execOptions = { cwd: rig.testDir!, stdio: 'inherit' as const };
|
||||
execSync('git init --initial-branch=main', execOptions);
|
||||
execSync('git config user.email "test@example.com"', execOptions);
|
||||
execSync('git config user.name "Test User"', execOptions);
|
||||
|
||||
// Temporarily disable the interactive editor and git pager
|
||||
// to avoid hanging the tests. It seems the the agent isn't
|
||||
// consistently honoring the instructions to avoid interactive
|
||||
// commands.
|
||||
execSync('git config core.editor "true"', execOptions);
|
||||
execSync('git config core.pager "cat"', execOptions);
|
||||
execSync('git config commit.gpgsign false', execOptions);
|
||||
execSync('git add .', execOptions);
|
||||
execSync('git commit --allow-empty -m "Initial commit"', execOptions);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -16,10 +16,6 @@ export default defineConfig({
|
||||
},
|
||||
test: {
|
||||
testTimeout: 300000, // 5 minutes
|
||||
// Retry in CI but not nightly to avoid blocking on API error.
|
||||
retry: process.env['VITEST_RETRY']
|
||||
? parseInt(process.env['VITEST_RETRY'], 10)
|
||||
: 3,
|
||||
reporters: ['default', 'json'],
|
||||
outputFile: {
|
||||
json: 'evals/logs/report.json',
|
||||
|
||||
@@ -691,6 +691,40 @@ describe('useSlashCompletion', () => {
|
||||
});
|
||||
unmount();
|
||||
});
|
||||
|
||||
it('should rank primary name prefix matches higher than alias prefix matches', async () => {
|
||||
const slashCommands = [
|
||||
createTestCommand({
|
||||
name: 'footer',
|
||||
altNames: ['statusline'],
|
||||
description: 'Configure footer',
|
||||
}),
|
||||
createTestCommand({
|
||||
name: 'stats',
|
||||
altNames: ['usage'],
|
||||
description: 'Check stats',
|
||||
}),
|
||||
];
|
||||
|
||||
const { result, unmount } = await renderHook(() =>
|
||||
useTestHarnessForSlashCompletion(
|
||||
true,
|
||||
'/stat',
|
||||
slashCommands,
|
||||
mockCommandContext,
|
||||
),
|
||||
);
|
||||
|
||||
await resolveMatch();
|
||||
|
||||
await waitFor(() => {
|
||||
// 'stats' should be first because 'stat' is a prefix match on its name
|
||||
// while 'footer' only matches 'stat' via its alias 'statusline'
|
||||
expect(result.current.suggestions[0].label).toBe('stats');
|
||||
expect(result.current.suggestions[1].label).toBe('footer');
|
||||
});
|
||||
unmount();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Sub-Commands', () => {
|
||||
|
||||
@@ -272,13 +272,45 @@ function useCommandSuggestions(
|
||||
}
|
||||
|
||||
if (!signal.aborted) {
|
||||
// Sort potentialSuggestions so that exact match (by name or altName) comes first
|
||||
// Sort potentialSuggestions so that exact name/prefix match comes first,
|
||||
// prioritizing primary name over altNames.
|
||||
const lowerPartial = partial.toLowerCase();
|
||||
const sortedSuggestions = [...potentialSuggestions].sort((a, b) => {
|
||||
const aIsExact = matchesCommand(a, partial);
|
||||
const bIsExact = matchesCommand(b, partial);
|
||||
if (aIsExact && !bIsExact) return -1;
|
||||
if (!aIsExact && bIsExact) return 1;
|
||||
return 0;
|
||||
// 1. Exact name match
|
||||
const aNameExact = a.name.toLowerCase() === lowerPartial;
|
||||
const bNameExact = b.name.toLowerCase() === lowerPartial;
|
||||
if (aNameExact && !bNameExact) return -1;
|
||||
if (!aNameExact && bNameExact) return 1;
|
||||
|
||||
// 2. Exact altName match
|
||||
const aAltExact =
|
||||
a.altNames?.some((alt) => alt.toLowerCase() === lowerPartial) ||
|
||||
false;
|
||||
const bAltExact =
|
||||
b.altNames?.some((alt) => alt.toLowerCase() === lowerPartial) ||
|
||||
false;
|
||||
if (aAltExact && !bAltExact) return -1;
|
||||
if (!aAltExact && bAltExact) return 1;
|
||||
|
||||
// 3. Prefix name match
|
||||
const aNamePrefix = a.name.toLowerCase().startsWith(lowerPartial);
|
||||
const bNamePrefix = b.name.toLowerCase().startsWith(lowerPartial);
|
||||
if (aNamePrefix && !bNamePrefix) return -1;
|
||||
if (!aNamePrefix && bNamePrefix) return 1;
|
||||
|
||||
// 4. Prefix altName match
|
||||
const aAltPrefix =
|
||||
a.altNames?.some((alt) =>
|
||||
alt.toLowerCase().startsWith(lowerPartial),
|
||||
) || false;
|
||||
const bAltPrefix =
|
||||
b.altNames?.some((alt) =>
|
||||
alt.toLowerCase().startsWith(lowerPartial),
|
||||
) || false;
|
||||
if (aAltPrefix && !bAltPrefix) return -1;
|
||||
if (!aAltPrefix && bAltPrefix) return 1;
|
||||
|
||||
return 0; // Maintain FZF score order for other matches
|
||||
});
|
||||
|
||||
const finalSuggestions = sortedSuggestions.map((cmd) => {
|
||||
|
||||
@@ -200,7 +200,10 @@ describe('LocalSubagentInvocation', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(params, signal);
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(
|
||||
params,
|
||||
expect.any(AbortSignal),
|
||||
);
|
||||
|
||||
expect(result.llmContent).toEqual([
|
||||
{
|
||||
@@ -495,10 +498,50 @@ describe('LocalSubagentInvocation', () => {
|
||||
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(
|
||||
params,
|
||||
controller.signal,
|
||||
expect.any(AbortSignal),
|
||||
);
|
||||
});
|
||||
|
||||
it('external AbortSignal propagates through session to abort executor', async () => {
|
||||
// This test validates the wiring: caller signal → abortListener → session.abort()
|
||||
// → internal AbortController → executor's signal aborts.
|
||||
// The previous test mocks the executor to always reject, so it would pass even
|
||||
// if the abort wiring was broken. This test requires the signal to actually fire.
|
||||
const controller = new AbortController();
|
||||
let executorSignal: AbortSignal | undefined;
|
||||
|
||||
mockExecutorInstance.run.mockImplementation(
|
||||
(_p: unknown, sig: AbortSignal) => {
|
||||
executorSignal = sig;
|
||||
return new Promise((_resolve, reject) => {
|
||||
sig.addEventListener('abort', () => {
|
||||
const err = new Error('AbortError');
|
||||
err.name = 'AbortError';
|
||||
reject(err);
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
const executePromise = invocation.execute(
|
||||
controller.signal,
|
||||
updateOutput,
|
||||
);
|
||||
|
||||
// Wait for the executor to start so executorSignal is populated
|
||||
await vi.waitFor(() => {
|
||||
expect(executorSignal).toBeDefined();
|
||||
});
|
||||
|
||||
expect(executorSignal!.aborted).toBe(false);
|
||||
|
||||
// Fire the external signal — this must propagate through to abort the executor
|
||||
controller.abort();
|
||||
|
||||
await expect(executePromise).rejects.toThrow();
|
||||
expect(executorSignal!.aborted).toBe(true);
|
||||
});
|
||||
|
||||
it('should throw an error and bubble cancellation when execution returns ABORTED', async () => {
|
||||
const mockOutput = {
|
||||
result: 'Cancelled by user',
|
||||
|
||||
@@ -5,7 +5,6 @@
|
||||
*/
|
||||
|
||||
import { type AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import { LocalAgentExecutor } from './local-executor.js';
|
||||
import {
|
||||
BaseToolInvocation,
|
||||
type ToolResult,
|
||||
@@ -30,6 +29,8 @@ import {
|
||||
sanitizeToolArgs,
|
||||
sanitizeErrorMessage,
|
||||
} from '../utils/agent-sanitization-utils.js';
|
||||
import { LocalSubagentSession } from './local-subagent-protocol.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
|
||||
const INPUT_PREVIEW_MAX_LENGTH = 50;
|
||||
const DESCRIPTION_MAX_LENGTH = 200;
|
||||
@@ -39,11 +40,10 @@ const MAX_RECENT_ACTIVITY = 3;
|
||||
* Represents a validated, executable instance of a subagent tool.
|
||||
*
|
||||
* This class orchestrates the execution of a defined agent by:
|
||||
* 1. Initializing the {@link LocalAgentExecutor}.
|
||||
* 2. Running the agent's execution loop.
|
||||
* 3. Bridging the agent's streaming activity (e.g., thoughts) to the tool's
|
||||
* live output stream.
|
||||
* 4. Formatting the final result into a {@link ToolResult}.
|
||||
* 1. Using {@link LocalSubagentSession} as the execution engine.
|
||||
* 2. Bridging the agent's streaming activity (e.g., thoughts) to the tool's
|
||||
* live output stream via the session's rawActivityCallback.
|
||||
* 3. Formatting the final result into a {@link ToolResult}.
|
||||
*/
|
||||
export class LocalSubagentInvocation extends BaseToolInvocation<
|
||||
AgentInputs,
|
||||
@@ -54,6 +54,9 @@ export class LocalSubagentInvocation extends BaseToolInvocation<
|
||||
* @param context The agent loop context.
|
||||
* @param params The validated input parameters for the agent.
|
||||
* @param messageBus Message bus for policy enforcement.
|
||||
* @param _toolName Optional override for the tool name.
|
||||
* @param _toolDisplayName Optional override for the tool display name.
|
||||
* @param _onAgentEvent Optional callback for parent session observability.
|
||||
*/
|
||||
constructor(
|
||||
private readonly definition: LocalAgentDefinition,
|
||||
@@ -62,6 +65,7 @@ export class LocalSubagentInvocation extends BaseToolInvocation<
|
||||
messageBus: MessageBus,
|
||||
_toolName?: string,
|
||||
_toolDisplayName?: string,
|
||||
private readonly _onAgentEvent?: (event: AgentEvent) => void,
|
||||
) {
|
||||
super(
|
||||
params,
|
||||
@@ -101,9 +105,170 @@ export class LocalSubagentInvocation extends BaseToolInvocation<
|
||||
): Promise<ToolResult> {
|
||||
let recentActivity: SubagentActivityItem[] = [];
|
||||
|
||||
// Raw SubagentActivityEvent handler — preserves all existing progress display logic.
|
||||
// Passed as rawActivityCallback to LocalSubagentSession so the protocol can call it
|
||||
// before translating to AgentEvents.
|
||||
const onActivity = (activity: SubagentActivityEvent): void => {
|
||||
if (!updateOutput) return;
|
||||
|
||||
let updated = false;
|
||||
|
||||
switch (activity.type) {
|
||||
case 'THOUGHT_CHUNK': {
|
||||
const text = String(activity.data['text']);
|
||||
const lastItem = recentActivity[recentActivity.length - 1];
|
||||
|
||||
if (
|
||||
lastItem &&
|
||||
lastItem.type === 'thought' &&
|
||||
lastItem.status === 'running'
|
||||
) {
|
||||
lastItem.content = sanitizeThoughtContent(text);
|
||||
} else {
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'thought',
|
||||
content: sanitizeThoughtContent(text),
|
||||
status: 'running',
|
||||
});
|
||||
}
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
case 'TOOL_CALL_START': {
|
||||
const name = String(activity.data['name']);
|
||||
const displayName = activity.data['displayName']
|
||||
? sanitizeErrorMessage(String(activity.data['displayName']))
|
||||
: undefined;
|
||||
const description = activity.data['description']
|
||||
? sanitizeErrorMessage(String(activity.data['description']))
|
||||
: undefined;
|
||||
const args = JSON.stringify(sanitizeToolArgs(activity.data['args']));
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'tool_call',
|
||||
content: name,
|
||||
displayName,
|
||||
description,
|
||||
args,
|
||||
status: 'running',
|
||||
});
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
case 'TOOL_CALL_END': {
|
||||
const name = String(activity.data['name']);
|
||||
const data = activity.data['data'];
|
||||
const isError = isToolActivityError(data);
|
||||
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === name &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = isError ? 'error' : 'completed';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'ERROR': {
|
||||
const error = String(activity.data['error']);
|
||||
const errorType = activity.data['errorType'];
|
||||
const sanitizedError = sanitizeErrorMessage(error);
|
||||
const isCancellation =
|
||||
errorType === SubagentActivityErrorType.CANCELLED ||
|
||||
error === SUBAGENT_CANCELLED_ERROR_MESSAGE;
|
||||
const isRejection =
|
||||
errorType === SubagentActivityErrorType.REJECTED ||
|
||||
error.startsWith(SUBAGENT_REJECTED_ERROR_PREFIX);
|
||||
|
||||
const toolName = activity.data['name']
|
||||
? String(activity.data['name'])
|
||||
: undefined;
|
||||
|
||||
if (toolName && (isCancellation || isRejection)) {
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === toolName &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = 'cancelled';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else if (toolName) {
|
||||
// Mark non-rejection/non-cancellation errors as 'error'
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === toolName &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = 'error';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'thought',
|
||||
content:
|
||||
isCancellation || isRejection
|
||||
? sanitizedError
|
||||
: `Error: ${sanitizedError}`,
|
||||
status: isCancellation || isRejection ? 'cancelled' : 'error',
|
||||
});
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
if (updated) {
|
||||
// Keep only the last N items
|
||||
if (recentActivity.length > MAX_RECENT_ACTIVITY) {
|
||||
recentActivity = recentActivity.slice(-MAX_RECENT_ACTIVITY);
|
||||
}
|
||||
|
||||
const progress: SubagentProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName: this.definition.name,
|
||||
recentActivity: [...recentActivity], // Copy to avoid mutation issues
|
||||
state: 'running',
|
||||
};
|
||||
|
||||
updateOutput(progress);
|
||||
}
|
||||
};
|
||||
|
||||
// Create session with the raw activity callback for rich progress display
|
||||
const session = new LocalSubagentSession(
|
||||
this.definition,
|
||||
this.context,
|
||||
this.messageBus,
|
||||
onActivity,
|
||||
);
|
||||
|
||||
// Subscribe for parent session observability (future use)
|
||||
let unsubscribeParent: (() => void) | undefined;
|
||||
if (this._onAgentEvent) {
|
||||
unsubscribeParent = session.subscribe(this._onAgentEvent);
|
||||
}
|
||||
|
||||
// Wire external abort signal to session abort
|
||||
const abortListener = () => void session.abort();
|
||||
signal.addEventListener('abort', abortListener, { once: true });
|
||||
|
||||
try {
|
||||
if (updateOutput) {
|
||||
// Send initial state
|
||||
const initialProgress: SubagentProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName: this.definition.name,
|
||||
@@ -113,158 +278,16 @@ export class LocalSubagentInvocation extends BaseToolInvocation<
|
||||
updateOutput(initialProgress);
|
||||
}
|
||||
|
||||
// Create an activity callback to bridge the executor's events to the
|
||||
// tool's streaming output.
|
||||
const onActivity = (activity: SubagentActivityEvent): void => {
|
||||
if (!updateOutput) return;
|
||||
// Buffer non-query params, then send query as message to start execution
|
||||
const query = String(this.params['query'] ?? '');
|
||||
const otherParams = { ...this.params } as Record<string, unknown>;
|
||||
delete otherParams['query'];
|
||||
if (Object.keys(otherParams).length > 0) {
|
||||
await session.send({ update: { config: otherParams } });
|
||||
}
|
||||
await session.send({ message: [{ type: 'text', text: query }] });
|
||||
|
||||
let updated = false;
|
||||
|
||||
switch (activity.type) {
|
||||
case 'THOUGHT_CHUNK': {
|
||||
const text = String(activity.data['text']);
|
||||
const lastItem = recentActivity[recentActivity.length - 1];
|
||||
|
||||
if (
|
||||
lastItem &&
|
||||
lastItem.type === 'thought' &&
|
||||
lastItem.status === 'running'
|
||||
) {
|
||||
lastItem.content = sanitizeThoughtContent(text);
|
||||
} else {
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'thought',
|
||||
content: sanitizeThoughtContent(text),
|
||||
status: 'running',
|
||||
});
|
||||
}
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
case 'TOOL_CALL_START': {
|
||||
const name = String(activity.data['name']);
|
||||
const displayName = activity.data['displayName']
|
||||
? sanitizeErrorMessage(String(activity.data['displayName']))
|
||||
: undefined;
|
||||
const description = activity.data['description']
|
||||
? sanitizeErrorMessage(String(activity.data['description']))
|
||||
: undefined;
|
||||
const args = JSON.stringify(
|
||||
sanitizeToolArgs(activity.data['args']),
|
||||
);
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'tool_call',
|
||||
content: name,
|
||||
displayName,
|
||||
description,
|
||||
args,
|
||||
status: 'running',
|
||||
});
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
case 'TOOL_CALL_END': {
|
||||
const name = String(activity.data['name']);
|
||||
const data = activity.data['data'];
|
||||
const isError = isToolActivityError(data);
|
||||
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === name &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = isError ? 'error' : 'completed';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'ERROR': {
|
||||
const error = String(activity.data['error']);
|
||||
const errorType = activity.data['errorType'];
|
||||
const sanitizedError = sanitizeErrorMessage(error);
|
||||
const isCancellation =
|
||||
errorType === SubagentActivityErrorType.CANCELLED ||
|
||||
error === SUBAGENT_CANCELLED_ERROR_MESSAGE;
|
||||
const isRejection =
|
||||
errorType === SubagentActivityErrorType.REJECTED ||
|
||||
error.startsWith(SUBAGENT_REJECTED_ERROR_PREFIX);
|
||||
|
||||
const toolName = activity.data['name']
|
||||
? String(activity.data['name'])
|
||||
: undefined;
|
||||
|
||||
if (toolName && (isCancellation || isRejection)) {
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === toolName &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = 'cancelled';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else if (toolName) {
|
||||
// Mark non-rejection/non-cancellation errors as 'error'
|
||||
for (let i = recentActivity.length - 1; i >= 0; i--) {
|
||||
if (
|
||||
recentActivity[i].type === 'tool_call' &&
|
||||
recentActivity[i].content === toolName &&
|
||||
recentActivity[i].status === 'running'
|
||||
) {
|
||||
recentActivity[i].status = 'error';
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
recentActivity.push({
|
||||
id: randomUUID(),
|
||||
type: 'thought',
|
||||
content:
|
||||
isCancellation || isRejection
|
||||
? sanitizedError
|
||||
: `Error: ${sanitizedError}`,
|
||||
status: isCancellation || isRejection ? 'cancelled' : 'error',
|
||||
});
|
||||
updated = true;
|
||||
break;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
if (updated) {
|
||||
// Keep only the last N items
|
||||
if (recentActivity.length > MAX_RECENT_ACTIVITY) {
|
||||
recentActivity = recentActivity.slice(-MAX_RECENT_ACTIVITY);
|
||||
}
|
||||
|
||||
const progress: SubagentProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName: this.definition.name,
|
||||
recentActivity: [...recentActivity], // Copy to avoid mutation issues
|
||||
state: 'running',
|
||||
};
|
||||
|
||||
updateOutput(progress);
|
||||
}
|
||||
};
|
||||
|
||||
const executor = await LocalAgentExecutor.create(
|
||||
this.definition,
|
||||
this.context,
|
||||
onActivity,
|
||||
);
|
||||
|
||||
const output = await executor.run(this.params, signal);
|
||||
const output = await session.getResult();
|
||||
|
||||
if (output.terminate_reason === AgentTerminateMode.ABORTED) {
|
||||
const progress: SubagentProgress = {
|
||||
@@ -359,6 +382,9 @@ ${output.result}`;
|
||||
// We omit the 'error' property so that the UI renders our rich returnDisplay
|
||||
// instead of the raw error message. The llmContent still informs the agent of the failure.
|
||||
};
|
||||
} finally {
|
||||
signal.removeEventListener('abort', abortListener);
|
||||
unsubscribeParent?.();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,776 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { LocalSubagentSession } from './local-subagent-protocol.js';
|
||||
import { LocalAgentExecutor } from './local-executor.js';
|
||||
import {
|
||||
AgentTerminateMode,
|
||||
type LocalAgentDefinition,
|
||||
type SubagentActivityEvent,
|
||||
} from './types.js';
|
||||
import { makeFakeConfig } from '../test-utils/config.js';
|
||||
import { createMockMessageBus } from '../test-utils/mock-message-bus.js';
|
||||
import type { AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
import type { z } from 'zod';
|
||||
import type { Mocked } from 'vitest';
|
||||
|
||||
vi.mock('./local-executor.js');
|
||||
|
||||
const MockLocalAgentExecutor = vi.mocked(LocalAgentExecutor);
|
||||
|
||||
// Captures the onActivity callback passed to LocalAgentExecutor.create().
|
||||
// Set via create.mockImplementation in beforeEach to avoid mock.calls index fragility.
|
||||
let capturedOnActivity: ((activity: SubagentActivityEvent) => void) | undefined;
|
||||
|
||||
const testDefinition: LocalAgentDefinition = {
|
||||
kind: 'local',
|
||||
name: 'TestProtocolAgent',
|
||||
description: 'A test agent for protocol tests.',
|
||||
inputConfig: {
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
task: { type: 'string' },
|
||||
priority: { type: 'number' },
|
||||
},
|
||||
},
|
||||
},
|
||||
modelConfig: { model: 'test', generateContentConfig: {} },
|
||||
runConfig: { maxTimeMinutes: 1 },
|
||||
promptConfig: { systemPrompt: 'test' },
|
||||
};
|
||||
|
||||
const GOAL_OUTPUT = {
|
||||
result: 'Analysis complete.',
|
||||
terminate_reason: AgentTerminateMode.GOAL,
|
||||
};
|
||||
|
||||
describe('LocalSubagentSession (protocol)', () => {
|
||||
let mockContext: AgentLoopContext;
|
||||
let mockMessageBus: MessageBus;
|
||||
let mockExecutorInstance: Mocked<LocalAgentExecutor<z.ZodUnknown>>;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
capturedOnActivity = undefined;
|
||||
|
||||
mockContext = makeFakeConfig() as unknown as AgentLoopContext;
|
||||
mockMessageBus = createMockMessageBus();
|
||||
|
||||
mockExecutorInstance = {
|
||||
run: vi.fn().mockResolvedValue(GOAL_OUTPUT),
|
||||
definition: testDefinition,
|
||||
} as unknown as Mocked<LocalAgentExecutor<z.ZodUnknown>>;
|
||||
|
||||
// Use mockImplementation (not mockResolvedValue) so we can capture onActivity.
|
||||
MockLocalAgentExecutor.create.mockImplementation(
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
async (_def: any, _ctx: any, onActivity: any) => {
|
||||
capturedOnActivity = onActivity;
|
||||
|
||||
return mockExecutorInstance as unknown as LocalAgentExecutor<z.ZodTypeAny>;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Lifecycle events
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('lifecycle events', () => {
|
||||
it('emits agent_start then agent_end(completed) for a GOAL run', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'query' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(events[0].type).toBe('agent_start');
|
||||
expect(events[events.length - 1].type).toBe('agent_end');
|
||||
const endEvent = events[events.length - 1];
|
||||
if (endEvent.type === 'agent_end') {
|
||||
expect(endEvent.reason).toBe('completed');
|
||||
}
|
||||
});
|
||||
|
||||
it('emits agent_start exactly once even if ensureAgentStart called twice internally', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'query' }] });
|
||||
await session.getResult();
|
||||
|
||||
const startEvents = events.filter((e) => e.type === 'agent_start');
|
||||
expect(startEvents).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('emits agent_end exactly once on error path', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
mockExecutorInstance.run.mockRejectedValue(new Error('executor failed'));
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'query' }] });
|
||||
await expect(session.getResult()).rejects.toThrow('executor failed');
|
||||
|
||||
const endEvents = events.filter((e) => e.type === 'agent_end');
|
||||
expect(endEvents).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('all events share the same streamId', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'query' }] });
|
||||
await session.getResult();
|
||||
|
||||
const streamIds = new Set(events.map((e) => e.streamId));
|
||||
expect(streamIds.size).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Config buffering (update + message pattern)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('config buffering', () => {
|
||||
it('merges buffered config with message query', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({
|
||||
update: { config: { task: 'analyze', priority: 5 } },
|
||||
});
|
||||
await session.send({ message: [{ type: 'text', text: 'my query' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(
|
||||
{ task: 'analyze', priority: 5, query: 'my query' },
|
||||
expect.any(AbortSignal),
|
||||
);
|
||||
});
|
||||
|
||||
it('omits query key when message text is empty', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({ update: { config: { task: 'no-query-task' } } });
|
||||
await session.send({ message: [{ type: 'text', text: '' }] });
|
||||
await session.getResult();
|
||||
|
||||
const callArgs = mockExecutorInstance.run.mock.calls[0][0];
|
||||
expect(callArgs).not.toHaveProperty('query');
|
||||
expect(callArgs).toEqual({ task: 'no-query-task' });
|
||||
});
|
||||
|
||||
it('sends only query when no prior update', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'just a query' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(
|
||||
{ query: 'just a query' },
|
||||
expect.any(AbortSignal),
|
||||
);
|
||||
});
|
||||
|
||||
it('multiple update calls are merged', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({ update: { config: { field1: 'a' } } });
|
||||
await session.send({ update: { config: { field2: 'b' } } });
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(mockExecutorInstance.run).toHaveBeenCalledWith(
|
||||
{ field1: 'a', field2: 'b', query: 'q' },
|
||||
expect.any(AbortSignal),
|
||||
);
|
||||
});
|
||||
|
||||
it('update returns streamId: null; message returns a streamId', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const updateResult = await session.send({ update: { config: {} } });
|
||||
expect(updateResult.streamId).toBeNull();
|
||||
|
||||
const messageResult = await session.send({
|
||||
message: [{ type: 'text', text: 'q' }],
|
||||
});
|
||||
expect(messageResult.streamId).not.toBeNull();
|
||||
expect(typeof messageResult.streamId).toBe('string');
|
||||
|
||||
// Await completion to prevent dangling execution affecting subsequent tests
|
||||
await session.getResult();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Activity translation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('activity translation', () => {
|
||||
function makeSession() {
|
||||
const activityEvents: SubagentActivityEvent[] = [];
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
return { session, activityEvents };
|
||||
}
|
||||
|
||||
async function runWithActivities(
|
||||
session: LocalSubagentSession,
|
||||
activities: SubagentActivityEvent[],
|
||||
) {
|
||||
mockExecutorInstance.run.mockImplementation(async () => {
|
||||
// capturedOnActivity is set by the create.mockImplementation in beforeEach
|
||||
// and updated whenever create() is called. By the time run() is called,
|
||||
// capturedOnActivity holds the onActivity closure for the most-recently
|
||||
// created executor — which is the one associated with this session.
|
||||
for (const act of activities) {
|
||||
capturedOnActivity?.(act);
|
||||
}
|
||||
return GOAL_OUTPUT;
|
||||
});
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
return events;
|
||||
}
|
||||
|
||||
it('THOUGHT_CHUNK → message event with thought content', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'THOUGHT_CHUNK',
|
||||
data: { text: 'I am thinking...' },
|
||||
},
|
||||
]);
|
||||
|
||||
const msgEvent = events.find((e) => e.type === 'message');
|
||||
expect(msgEvent).toBeDefined();
|
||||
if (msgEvent?.type === 'message') {
|
||||
expect(msgEvent.role).toBe('agent');
|
||||
expect(msgEvent.content).toContainEqual({
|
||||
type: 'thought',
|
||||
thought: 'I am thinking...',
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('TOOL_CALL_START → tool_request event', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'TOOL_CALL_START',
|
||||
data: { callId: 'call-123', name: 'read_file', args: { path: '/a' } },
|
||||
},
|
||||
]);
|
||||
|
||||
const reqEvent = events.find((e) => e.type === 'tool_request');
|
||||
expect(reqEvent).toBeDefined();
|
||||
if (reqEvent?.type === 'tool_request') {
|
||||
expect(reqEvent.requestId).toBe('call-123');
|
||||
expect(reqEvent.name).toBe('read_file');
|
||||
expect(reqEvent.args).toEqual({ path: '/a' });
|
||||
}
|
||||
});
|
||||
|
||||
it('TOOL_CALL_END → tool_response event', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'TOOL_CALL_END',
|
||||
data: { id: 'call-123', name: 'read_file', output: 'file contents' },
|
||||
},
|
||||
]);
|
||||
|
||||
const respEvent = events.find((e) => e.type === 'tool_response');
|
||||
expect(respEvent).toBeDefined();
|
||||
if (respEvent?.type === 'tool_response') {
|
||||
expect(respEvent.requestId).toBe('call-123');
|
||||
expect(respEvent.name).toBe('read_file');
|
||||
expect(respEvent.content).toContainEqual({
|
||||
type: 'text',
|
||||
text: 'file contents',
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('ERROR activity → error event with INTERNAL status, fatal: false', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'ERROR',
|
||||
data: { error: 'something went wrong' },
|
||||
},
|
||||
]);
|
||||
|
||||
const errEvent = events.find((e) => e.type === 'error');
|
||||
expect(errEvent).toBeDefined();
|
||||
if (errEvent?.type === 'error') {
|
||||
expect(errEvent.status).toBe('INTERNAL');
|
||||
expect(errEvent.message).toBe('something went wrong');
|
||||
expect(errEvent.fatal).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('unknown activity type → no events emitted', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
type: 'UNKNOWN_TYPE' as any,
|
||||
data: {},
|
||||
},
|
||||
]);
|
||||
|
||||
// Only agent_start and agent_end should be present
|
||||
const nonLifecycle = events.filter(
|
||||
(e) => e.type !== 'agent_start' && e.type !== 'agent_end',
|
||||
);
|
||||
expect(nonLifecycle).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('TOOL_CALL_START with non-object args defaults to {}', async () => {
|
||||
const { session } = makeSession();
|
||||
const events = await runWithActivities(session, [
|
||||
{
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'TOOL_CALL_START',
|
||||
data: { callId: 'x', name: 'tool', args: null },
|
||||
},
|
||||
]);
|
||||
|
||||
const reqEvent = events.find((e) => e.type === 'tool_request');
|
||||
if (reqEvent?.type === 'tool_request') {
|
||||
expect(reqEvent.args).toEqual({});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// getResult() promise
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('getResult()', () => {
|
||||
it('resolves with OutputObject on GOAL termination', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
const output = await session.getResult();
|
||||
|
||||
expect(output.result).toBe('Analysis complete.');
|
||||
expect(output.terminate_reason).toBe(AgentTerminateMode.GOAL);
|
||||
});
|
||||
|
||||
it('rejects when executor throws', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
mockExecutorInstance.run.mockRejectedValue(new Error('executor error'));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow('executor error');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// rawActivityCallback
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('rawActivityCallback', () => {
|
||||
it('receives raw SubagentActivityEvent before AgentEvent translation', async () => {
|
||||
const rawActivities: SubagentActivityEvent[] = [];
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
(activity) => rawActivities.push(activity),
|
||||
);
|
||||
|
||||
const thoughtActivity: SubagentActivityEvent = {
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'THOUGHT_CHUNK',
|
||||
data: { text: 'raw thought' },
|
||||
};
|
||||
|
||||
mockExecutorInstance.run.mockImplementation(async () => {
|
||||
const onActivity = MockLocalAgentExecutor.create.mock.calls[0]?.[2];
|
||||
onActivity?.(thoughtActivity);
|
||||
return GOAL_OUTPUT;
|
||||
});
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(rawActivities).toHaveLength(1);
|
||||
expect(rawActivities[0]).toBe(thoughtActivity);
|
||||
});
|
||||
|
||||
it('is called before AgentEvent translation (raw arrives first)', async () => {
|
||||
const callOrder: string[] = [];
|
||||
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
() => callOrder.push('raw'),
|
||||
);
|
||||
|
||||
session.subscribe((e) => {
|
||||
if (e.type === 'message') callOrder.push('translated');
|
||||
});
|
||||
|
||||
mockExecutorInstance.run.mockImplementation(async () => {
|
||||
const onActivity = MockLocalAgentExecutor.create.mock.calls[0]?.[2];
|
||||
onActivity?.({
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'THOUGHT_CHUNK',
|
||||
data: { text: 'thought' },
|
||||
});
|
||||
return GOAL_OUTPUT;
|
||||
});
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(callOrder).toEqual(['raw', 'translated']);
|
||||
});
|
||||
|
||||
it('is optional — no callback causes no error', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
// no rawActivityCallback
|
||||
);
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).resolves.toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Subscription
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('subscription', () => {
|
||||
it('unsubscribe stops event delivery', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const received: AgentEvent[] = [];
|
||||
const unsub = session.subscribe((e) => received.push(e));
|
||||
unsub();
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(received).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('multiple subscribers all receive events', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const received1: AgentEvent[] = [];
|
||||
const received2: AgentEvent[] = [];
|
||||
session.subscribe((e) => received1.push(e));
|
||||
session.subscribe((e) => received2.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(received1.length).toBeGreaterThan(0);
|
||||
expect(received1).toEqual(received2);
|
||||
});
|
||||
|
||||
it('events array accumulates all emitted events', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(session.events.length).toBeGreaterThanOrEqual(2); // at least agent_start + agent_end
|
||||
expect(session.events[0].type).toBe('agent_start');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Terminate mode mapping
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('terminate mode → StreamEndReason mapping', () => {
|
||||
const cases: Array<[AgentTerminateMode, string]> = [
|
||||
[AgentTerminateMode.GOAL, 'completed'],
|
||||
[AgentTerminateMode.TIMEOUT, 'max_time'],
|
||||
[AgentTerminateMode.MAX_TURNS, 'max_turns'],
|
||||
[AgentTerminateMode.ABORTED, 'aborted'],
|
||||
[AgentTerminateMode.ERROR, 'failed'],
|
||||
[AgentTerminateMode.ERROR_NO_COMPLETE_TASK_CALL, 'failed'],
|
||||
];
|
||||
|
||||
for (const [terminateMode, expectedReason] of cases) {
|
||||
it(`${terminateMode} → agent_end(reason:'${expectedReason}')`, async () => {
|
||||
mockExecutorInstance.run.mockResolvedValue({
|
||||
result: 'done',
|
||||
terminate_reason: terminateMode,
|
||||
});
|
||||
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult().catch(() => {
|
||||
// ABORTED results in rejection — catch to let test complete
|
||||
});
|
||||
|
||||
const endEvent = events.find((e) => e.type === 'agent_end');
|
||||
expect(endEvent).toBeDefined();
|
||||
if (endEvent?.type === 'agent_end') {
|
||||
expect(endEvent.reason).toBe(expectedReason);
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Abort
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('abort()', () => {
|
||||
it('abort() causes agent_end(reason:aborted)', async () => {
|
||||
// Make run() wait until aborted
|
||||
let abortSignal: AbortSignal | undefined;
|
||||
mockExecutorInstance.run.mockImplementation(
|
||||
(_params: unknown, signal: AbortSignal) => {
|
||||
abortSignal = signal;
|
||||
return new Promise((_resolve, reject) => {
|
||||
signal.addEventListener('abort', () => {
|
||||
const err = new Error('AbortError');
|
||||
err.name = 'AbortError';
|
||||
reject(err);
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
void session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
|
||||
// Wait for executor to be created and run started
|
||||
await vi.waitFor(() => {
|
||||
expect(abortSignal).toBeDefined();
|
||||
});
|
||||
|
||||
await session.abort();
|
||||
|
||||
await expect(session.getResult()).rejects.toThrow();
|
||||
|
||||
const endEvent = events.find((e) => e.type === 'agent_end');
|
||||
expect(endEvent).toBeDefined();
|
||||
if (endEvent?.type === 'agent_end') {
|
||||
expect(endEvent.reason).toBe('aborted');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Full event sequence
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('full event sequence', () => {
|
||||
it('emits agent_start → message(thought) → tool_request → tool_response → agent_end in order', async () => {
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
mockExecutorInstance.run.mockImplementation(async () => {
|
||||
const onActivity = MockLocalAgentExecutor.create.mock.calls[0]?.[2];
|
||||
onActivity?.({
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'THOUGHT_CHUNK',
|
||||
data: { text: 'thinking' },
|
||||
});
|
||||
onActivity?.({
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'TOOL_CALL_START',
|
||||
data: { callId: 'c1', name: 'tool', args: {} },
|
||||
});
|
||||
onActivity?.({
|
||||
isSubagentActivityEvent: true,
|
||||
agentName: 'TestProtocolAgent',
|
||||
type: 'TOOL_CALL_END',
|
||||
data: { id: 'c1', name: 'tool', output: 'result' },
|
||||
});
|
||||
return GOAL_OUTPUT;
|
||||
});
|
||||
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'go' }] });
|
||||
await session.getResult();
|
||||
|
||||
const types = events.map((e) => e.type);
|
||||
expect(types).toEqual([
|
||||
'agent_start',
|
||||
'message',
|
||||
'tool_request',
|
||||
'tool_response',
|
||||
'agent_end',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Concurrent send() guard
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('concurrent send() guard', () => {
|
||||
it('calling send() while a stream is active throws', async () => {
|
||||
let abortSignal: AbortSignal | undefined;
|
||||
mockExecutorInstance.run.mockImplementation(
|
||||
(_params: unknown, signal: AbortSignal) => {
|
||||
abortSignal = signal;
|
||||
return new Promise((_resolve, reject) => {
|
||||
// Reject when aborted so getResult() can settle during cleanup
|
||||
signal.addEventListener('abort', () => {
|
||||
const err = new Error('AbortError');
|
||||
err.name = 'AbortError';
|
||||
reject(err);
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
const session = new LocalSubagentSession(
|
||||
testDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
void session.send({ message: [{ type: 'text', text: 'first' }] });
|
||||
|
||||
// Wait for execution to start
|
||||
await vi.waitFor(() => {
|
||||
expect(abortSignal).toBeDefined();
|
||||
});
|
||||
|
||||
// Second send() while first stream is active must throw
|
||||
await expect(
|
||||
session.send({ message: [{ type: 'text', text: 'second' }] }),
|
||||
).rejects.toThrow('cannot be called while a stream is active');
|
||||
|
||||
// Clean up: abort to unblock the hanging executor
|
||||
await session.abort();
|
||||
await session.getResult().catch(() => {});
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,427 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
/**
|
||||
* @fileoverview LocalSubagentProtocol — wraps LocalAgentExecutor behind the
|
||||
* AgentProtocol interface, translating SubagentActivityEvent callbacks into
|
||||
* AgentEvents and exposing the executor result via getResult().
|
||||
*
|
||||
* Pattern mirrors LegacyAgentProtocol, but the loop body runs
|
||||
* LocalAgentExecutor instead of GeminiClient.sendMessageStream().
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import type { AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import { AgentSession } from '../agent/agent-session.js';
|
||||
import type {
|
||||
AgentProtocol,
|
||||
AgentSend,
|
||||
AgentEvent,
|
||||
StreamEndReason,
|
||||
Unsubscribe,
|
||||
ContentPart,
|
||||
} from '../agent/types.js';
|
||||
import { LocalAgentExecutor } from './local-executor.js';
|
||||
import {
|
||||
AgentTerminateMode,
|
||||
type LocalAgentDefinition,
|
||||
type AgentInputs,
|
||||
type OutputObject,
|
||||
type SubagentActivityEvent,
|
||||
} from './types.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function isAbortLikeError(err: unknown): boolean {
|
||||
return err instanceof Error && err.name === 'AbortError';
|
||||
}
|
||||
|
||||
function mapTerminateMode(mode: AgentTerminateMode): StreamEndReason {
|
||||
switch (mode) {
|
||||
case AgentTerminateMode.GOAL:
|
||||
return 'completed';
|
||||
case AgentTerminateMode.TIMEOUT:
|
||||
return 'max_time';
|
||||
case AgentTerminateMode.MAX_TURNS:
|
||||
return 'max_turns';
|
||||
case AgentTerminateMode.ABORTED:
|
||||
return 'aborted';
|
||||
case AgentTerminateMode.ERROR:
|
||||
case AgentTerminateMode.ERROR_NO_COMPLETE_TASK_CALL:
|
||||
default:
|
||||
return 'failed';
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// LocalSubagentProtocol
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
class LocalSubagentProtocol implements AgentProtocol {
|
||||
private _events: AgentEvent[] = [];
|
||||
private _subscribers = new Set<(event: AgentEvent) => void>();
|
||||
private _streamId: string = randomUUID();
|
||||
private _eventCounter = 0;
|
||||
private _agentStartEmitted = false;
|
||||
private _agentEndEmitted = false;
|
||||
private _activeStreamId: string | undefined;
|
||||
private _abortController = new AbortController();
|
||||
|
||||
// Result promise wiring
|
||||
private _resultResolve!: (output: OutputObject) => void;
|
||||
private _resultReject!: (err: unknown) => void;
|
||||
private readonly _resultPromise: Promise<OutputObject>;
|
||||
|
||||
// Buffered config from send({update})
|
||||
private _bufferedConfig: Record<string, unknown> = {};
|
||||
|
||||
constructor(
|
||||
private readonly definition: LocalAgentDefinition,
|
||||
private readonly context: AgentLoopContext,
|
||||
_messageBus: MessageBus,
|
||||
private readonly _rawActivityCallback?: (
|
||||
activity: SubagentActivityEvent,
|
||||
) => void,
|
||||
) {
|
||||
this._resultPromise = new Promise<OutputObject>((resolve, reject) => {
|
||||
this._resultResolve = resolve;
|
||||
this._resultReject = reject;
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AgentProtocol interface
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
get events(): readonly AgentEvent[] {
|
||||
return this._events;
|
||||
}
|
||||
|
||||
subscribe(callback: (event: AgentEvent) => void): Unsubscribe {
|
||||
this._subscribers.add(callback);
|
||||
return () => {
|
||||
this._subscribers.delete(callback);
|
||||
};
|
||||
}
|
||||
|
||||
async send(payload: AgentSend): Promise<{ streamId: string | null }> {
|
||||
if ('update' in payload && payload.update) {
|
||||
// Buffer config for use when message send arrives
|
||||
if (payload.update.config) {
|
||||
this._bufferedConfig = {
|
||||
...this._bufferedConfig,
|
||||
...payload.update.config,
|
||||
};
|
||||
}
|
||||
return { streamId: null };
|
||||
}
|
||||
|
||||
if ('message' in payload && payload.message) {
|
||||
if (this._activeStreamId) {
|
||||
throw new Error(
|
||||
'LocalSubagentProtocol.send() cannot be called while a stream is active.',
|
||||
);
|
||||
}
|
||||
|
||||
// Extract query text from the message ContentParts
|
||||
const queryText = payload.message
|
||||
.filter((p): p is ContentPart & { type: 'text' } => p.type === 'text')
|
||||
.map((p) => p.text)
|
||||
.join('');
|
||||
|
||||
// Only include 'query' in params when the message text is non-empty,
|
||||
// so that callers that pass all fields via update.config are not affected.
|
||||
const params: AgentInputs = {
|
||||
...this._bufferedConfig,
|
||||
...(queryText.length > 0 ? { query: queryText } : {}),
|
||||
};
|
||||
|
||||
this._beginNewStream();
|
||||
const streamId = this._streamId;
|
||||
|
||||
// Schedule execution in a macrotask so send() resolves before agent_start
|
||||
setTimeout(() => {
|
||||
void this._runExecutionInBackground(params);
|
||||
}, 0);
|
||||
|
||||
return { streamId };
|
||||
}
|
||||
|
||||
// action and elicitations are not supported
|
||||
return { streamId: null };
|
||||
}
|
||||
|
||||
async abort(): Promise<void> {
|
||||
this._abortController.abort();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Protocol-specific: result access
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Resolves when the executor completes, with the raw OutputObject.
|
||||
* Used by LocalSubagentInvocation to build the ToolResult.
|
||||
*/
|
||||
getResult(): Promise<OutputObject> {
|
||||
return this._resultPromise;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core: execution
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
private _beginNewStream(): void {
|
||||
this._streamId = randomUUID();
|
||||
this._eventCounter = 0;
|
||||
this._abortController = new AbortController();
|
||||
this._agentStartEmitted = false;
|
||||
this._agentEndEmitted = false;
|
||||
this._activeStreamId = this._streamId;
|
||||
}
|
||||
|
||||
private async _runExecutionInBackground(params: AgentInputs): Promise<void> {
|
||||
this._ensureAgentStart();
|
||||
try {
|
||||
await this._runExecution(params);
|
||||
} catch (err: unknown) {
|
||||
if (this._abortController.signal.aborted || isAbortLikeError(err)) {
|
||||
this._ensureAgentEnd('aborted');
|
||||
this._resultReject(err);
|
||||
} else {
|
||||
this._emitErrorAndAgentEnd(err);
|
||||
this._resultReject(err);
|
||||
}
|
||||
this._clearActiveStream();
|
||||
}
|
||||
}
|
||||
|
||||
private async _runExecution(params: AgentInputs): Promise<void> {
|
||||
const signal = this._abortController.signal;
|
||||
|
||||
const onActivity = (activity: SubagentActivityEvent): void => {
|
||||
// Forward raw activity to invocation-level callback (for rich SubagentProgress display)
|
||||
this._rawActivityCallback?.(activity);
|
||||
this._emit(this._translateActivity(activity));
|
||||
};
|
||||
|
||||
const executor = await LocalAgentExecutor.create(
|
||||
this.definition,
|
||||
this.context,
|
||||
onActivity,
|
||||
);
|
||||
|
||||
const output = await executor.run(params, signal);
|
||||
|
||||
if (
|
||||
output.terminate_reason === AgentTerminateMode.ABORTED ||
|
||||
signal.aborted
|
||||
) {
|
||||
this._finishStream('aborted');
|
||||
} else {
|
||||
this._finishStream(mapTerminateMode(output.terminate_reason));
|
||||
}
|
||||
|
||||
this._resultResolve(output);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Activity → AgentEvent translation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
private _translateActivity(activity: SubagentActivityEvent): AgentEvent[] {
|
||||
switch (activity.type) {
|
||||
case 'THOUGHT_CHUNK': {
|
||||
const rawText = activity.data['text'];
|
||||
const text = String(rawText ?? '');
|
||||
return [
|
||||
this._makeEvent('message', {
|
||||
role: 'agent',
|
||||
content: [{ type: 'thought', thought: text }],
|
||||
}),
|
||||
];
|
||||
}
|
||||
case 'TOOL_CALL_START': {
|
||||
const rawCallId = activity.data['callId'];
|
||||
const callId = String(rawCallId ?? randomUUID());
|
||||
const rawName = activity.data['name'];
|
||||
const name = String(rawName ?? 'unknown');
|
||||
const rawArgs = activity.data['args'];
|
||||
const args: Record<string, unknown> =
|
||||
rawArgs !== null &&
|
||||
typeof rawArgs === 'object' &&
|
||||
!Array.isArray(rawArgs)
|
||||
? // eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion
|
||||
(rawArgs as Record<string, unknown>)
|
||||
: {};
|
||||
return [
|
||||
this._makeEvent('tool_request', {
|
||||
requestId: callId,
|
||||
name,
|
||||
args,
|
||||
}),
|
||||
];
|
||||
}
|
||||
case 'TOOL_CALL_END': {
|
||||
const rawId = activity.data['id'];
|
||||
const requestId = String(rawId ?? randomUUID());
|
||||
const rawName = activity.data['name'];
|
||||
const name = String(rawName ?? 'unknown');
|
||||
const rawOutput = activity.data['output'];
|
||||
const output = String(rawOutput ?? '');
|
||||
return [
|
||||
this._makeEvent('tool_response', {
|
||||
requestId,
|
||||
name,
|
||||
content: [{ type: 'text', text: output }],
|
||||
}),
|
||||
];
|
||||
}
|
||||
case 'ERROR': {
|
||||
const rawError = activity.data['error'];
|
||||
const errorMsg = String(rawError ?? 'Unknown error');
|
||||
return [
|
||||
this._makeEvent('error', {
|
||||
status: 'INTERNAL',
|
||||
message: errorMsg,
|
||||
fatal: false,
|
||||
}),
|
||||
];
|
||||
}
|
||||
default:
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal helpers (mirrors LegacyAgentProtocol)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
private _emit(events: AgentEvent[]): void {
|
||||
if (events.length === 0) return;
|
||||
const subscribers = [...this._subscribers];
|
||||
for (const event of events) {
|
||||
this._events.push(event);
|
||||
if (event.type === 'agent_end') {
|
||||
this._agentEndEmitted = true;
|
||||
}
|
||||
for (const sub of subscribers) {
|
||||
sub(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private _clearActiveStream(): void {
|
||||
this._activeStreamId = undefined;
|
||||
}
|
||||
|
||||
private _ensureAgentStart(): void {
|
||||
if (!this._agentStartEmitted) {
|
||||
this._agentStartEmitted = true;
|
||||
this._emit([this._makeEvent('agent_start', {})]);
|
||||
}
|
||||
}
|
||||
|
||||
private _ensureAgentEnd(reason: StreamEndReason = 'completed'): void {
|
||||
if (!this._agentEndEmitted && this._agentStartEmitted) {
|
||||
this._emit([this._makeEvent('agent_end', { reason })]);
|
||||
}
|
||||
}
|
||||
|
||||
private _finishStream(
|
||||
reason: StreamEndReason,
|
||||
data?: Record<string, unknown>,
|
||||
): void {
|
||||
if (data && !this._agentEndEmitted) {
|
||||
this._emit([this._makeEvent('agent_end', { reason, data })]);
|
||||
this._agentEndEmitted = true;
|
||||
} else {
|
||||
this._ensureAgentEnd(reason);
|
||||
}
|
||||
this._clearActiveStream();
|
||||
}
|
||||
|
||||
private _emitErrorAndAgentEnd(err: unknown): void {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
this._ensureAgentStart();
|
||||
|
||||
const meta: Record<string, unknown> = {};
|
||||
if (err instanceof Error) {
|
||||
meta['errorName'] = err.constructor.name;
|
||||
if ('exitCode' in err && typeof err.exitCode === 'number') {
|
||||
meta['exitCode'] = err.exitCode;
|
||||
}
|
||||
if ('code' in err) {
|
||||
meta['code'] = err.code;
|
||||
}
|
||||
}
|
||||
|
||||
this._emit([
|
||||
this._makeEvent('error', {
|
||||
status: 'INTERNAL',
|
||||
message,
|
||||
fatal: true,
|
||||
...(Object.keys(meta).length > 0 ? { _meta: meta } : {}),
|
||||
}),
|
||||
]);
|
||||
this._ensureAgentEnd('failed');
|
||||
}
|
||||
|
||||
private _nextEventFields() {
|
||||
return {
|
||||
id: `${this._streamId}-${this._eventCounter++}`,
|
||||
timestamp: new Date().toISOString(),
|
||||
streamId: this._streamId,
|
||||
};
|
||||
}
|
||||
|
||||
private _makeEvent<T extends AgentEvent['type']>(
|
||||
type: T,
|
||||
payload: Omit<AgentEvent<T>, 'id' | 'timestamp' | 'streamId' | 'type'>,
|
||||
): AgentEvent {
|
||||
// eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion
|
||||
return {
|
||||
...this._nextEventFields(),
|
||||
type,
|
||||
...payload,
|
||||
} as AgentEvent;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public export
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class LocalSubagentSession extends AgentSession {
|
||||
private readonly _localProtocol: LocalSubagentProtocol;
|
||||
|
||||
constructor(
|
||||
definition: LocalAgentDefinition,
|
||||
context: AgentLoopContext,
|
||||
messageBus: MessageBus,
|
||||
rawActivityCallback?: (activity: SubagentActivityEvent) => void,
|
||||
) {
|
||||
const protocol = new LocalSubagentProtocol(
|
||||
definition,
|
||||
context,
|
||||
messageBus,
|
||||
rawActivityCallback,
|
||||
);
|
||||
super(protocol);
|
||||
this._localProtocol = protocol;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the raw executor OutputObject once execution completes.
|
||||
* Used by LocalSubagentInvocation to build the ToolResult.
|
||||
*/
|
||||
getResult(): Promise<OutputObject> {
|
||||
return this._localProtocol.getResult();
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
type ToolConfirmationOutcome,
|
||||
type ToolResult,
|
||||
type ToolCallConfirmationDetails,
|
||||
type ToolLiveOutput,
|
||||
} from '../tools/tools.js';
|
||||
import {
|
||||
DEFAULT_QUERY_STRING,
|
||||
@@ -16,44 +17,23 @@ import {
|
||||
type RemoteAgentDefinition,
|
||||
type AgentInputs,
|
||||
type SubagentProgress,
|
||||
getAgentCardLoadOptions,
|
||||
getRemoteAgentTargetUrl,
|
||||
} from './types.js';
|
||||
import { type AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
import type {
|
||||
A2AClientManager,
|
||||
SendMessageResult,
|
||||
} from './a2a-client-manager.js';
|
||||
import { extractIdsFromResponse, A2AResultReassembler } from './a2aUtils.js';
|
||||
import type { AuthenticationHandler } from '@a2a-js/sdk/client';
|
||||
import { debugLogger } from '../utils/debugLogger.js';
|
||||
import type { AnsiOutput } from '../utils/terminalSerializer.js';
|
||||
import { A2AAuthProviderFactory } from './auth-provider/factory.js';
|
||||
import { A2AAgentError } from './a2a-errors.js';
|
||||
import { RemoteSubagentSession } from './remote-subagent-protocol.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
|
||||
/**
|
||||
* A tool invocation that proxies to a remote A2A agent.
|
||||
*
|
||||
* This implementation bypasses the local `LocalAgentExecutor` loop and directly
|
||||
* invokes the configured A2A tool.
|
||||
* This implementation delegates execution to {@link RemoteSubagentSession},
|
||||
* which wraps the A2A client streaming behind the AgentProtocol interface.
|
||||
*/
|
||||
export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
RemoteAgentInputs,
|
||||
ToolResult
|
||||
> {
|
||||
// Persist state across ephemeral invocation instances.
|
||||
private static readonly sessionState = new Map<
|
||||
string,
|
||||
{ contextId?: string; taskId?: string }
|
||||
>();
|
||||
// State for the ongoing conversation with the remote agent
|
||||
private contextId: string | undefined;
|
||||
private taskId: string | undefined;
|
||||
|
||||
private readonly clientManager: A2AClientManager;
|
||||
private authHandler: AuthenticationHandler | undefined;
|
||||
|
||||
constructor(
|
||||
private readonly definition: RemoteAgentDefinition,
|
||||
private readonly context: AgentLoopContext,
|
||||
@@ -61,6 +41,7 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
messageBus: MessageBus,
|
||||
_toolName?: string,
|
||||
_toolDisplayName?: string,
|
||||
private readonly _onAgentEvent?: (event: AgentEvent) => void,
|
||||
) {
|
||||
const query = params['query'] ?? DEFAULT_QUERY_STRING;
|
||||
if (typeof query !== 'string') {
|
||||
@@ -75,43 +56,19 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
_toolName ?? definition.name,
|
||||
_toolDisplayName ?? definition.displayName,
|
||||
);
|
||||
const clientManager = this.context.config.getA2AClientManager();
|
||||
if (!clientManager) {
|
||||
|
||||
// Validate that A2AClientManager is available at construction time
|
||||
if (!this.context.config.getA2AClientManager()) {
|
||||
throw new Error(
|
||||
`Failed to initialize RemoteAgentInvocation for '${definition.name}': A2AClientManager is not available.`,
|
||||
);
|
||||
}
|
||||
this.clientManager = clientManager;
|
||||
}
|
||||
|
||||
getDescription(): string {
|
||||
return `Calling remote agent ${this.definition.displayName ?? this.definition.name}`;
|
||||
}
|
||||
|
||||
private async getAuthHandler(): Promise<AuthenticationHandler | undefined> {
|
||||
if (this.authHandler) {
|
||||
return this.authHandler;
|
||||
}
|
||||
|
||||
if (this.definition.auth) {
|
||||
const targetUrl = getRemoteAgentTargetUrl(this.definition);
|
||||
const provider = await A2AAuthProviderFactory.create({
|
||||
authConfig: this.definition.auth,
|
||||
agentName: this.definition.name,
|
||||
targetUrl,
|
||||
agentCardUrl: this.definition.agentCardUrl,
|
||||
});
|
||||
if (!provider) {
|
||||
throw new Error(
|
||||
`Failed to create auth provider for agent '${this.definition.name}'`,
|
||||
);
|
||||
}
|
||||
this.authHandler = provider;
|
||||
}
|
||||
|
||||
return this.authHandler;
|
||||
}
|
||||
|
||||
protected override async getConfirmationDetails(
|
||||
_abortSignal: AbortSignal,
|
||||
): Promise<ToolCallConfirmationDetails | false> {
|
||||
@@ -128,13 +85,33 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
|
||||
async execute(
|
||||
_signal: AbortSignal,
|
||||
updateOutput?: (output: string | AnsiOutput | SubagentProgress) => void,
|
||||
updateOutput?: (output: ToolLiveOutput) => void,
|
||||
): Promise<ToolResult> {
|
||||
// 1. Ensure the agent is loaded (cached by manager)
|
||||
// We assume the user has provided an access token via some mechanism (TODO),
|
||||
// or we rely on ADC.
|
||||
const reassembler = new A2AResultReassembler();
|
||||
const agentName = this.definition.displayName ?? this.definition.name;
|
||||
const session = new RemoteSubagentSession(
|
||||
this.definition,
|
||||
this.context,
|
||||
this.messageBus,
|
||||
);
|
||||
|
||||
// Wire external abort signal to session abort
|
||||
const abortListener = () => void session.abort();
|
||||
_signal.addEventListener('abort', abortListener, { once: true });
|
||||
|
||||
// Subscribe for parent session observability (future use)
|
||||
let unsubscribeParent: (() => void) | undefined;
|
||||
if (this._onAgentEvent) {
|
||||
unsubscribeParent = session.subscribe(this._onAgentEvent);
|
||||
}
|
||||
|
||||
// Subscribe to message events for live SubagentProgress updates
|
||||
const unsubscribeProgress = session.subscribe((event: AgentEvent) => {
|
||||
if (event.type === 'message' && updateOutput) {
|
||||
const currentProgress = session.getLatestProgress();
|
||||
if (currentProgress) updateOutput(currentProgress);
|
||||
}
|
||||
});
|
||||
|
||||
try {
|
||||
if (updateOutput) {
|
||||
updateOutput({
|
||||
@@ -152,97 +129,25 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
});
|
||||
}
|
||||
|
||||
const priorState = RemoteAgentInvocation.sessionState.get(
|
||||
this.definition.name,
|
||||
);
|
||||
if (priorState) {
|
||||
this.contextId = priorState.contextId;
|
||||
this.taskId = priorState.taskId;
|
||||
}
|
||||
await session.send({
|
||||
message: [{ type: 'text', text: this.params.query }],
|
||||
});
|
||||
|
||||
const authHandler = await this.getAuthHandler();
|
||||
|
||||
if (!this.clientManager.getClient(this.definition.name)) {
|
||||
await this.clientManager.loadAgent(
|
||||
this.definition.name,
|
||||
getAgentCardLoadOptions(this.definition),
|
||||
authHandler,
|
||||
);
|
||||
}
|
||||
|
||||
const message = this.params.query;
|
||||
|
||||
const stream = this.clientManager.sendMessageStream(
|
||||
this.definition.name,
|
||||
message,
|
||||
{
|
||||
contextId: this.contextId,
|
||||
taskId: this.taskId,
|
||||
signal: _signal,
|
||||
},
|
||||
);
|
||||
|
||||
let finalResponse: SendMessageResult | undefined;
|
||||
|
||||
for await (const chunk of stream) {
|
||||
if (_signal.aborted) {
|
||||
throw new Error('Operation aborted');
|
||||
}
|
||||
finalResponse = chunk;
|
||||
reassembler.update(chunk);
|
||||
|
||||
if (updateOutput) {
|
||||
updateOutput({
|
||||
isSubagentProgress: true,
|
||||
agentName,
|
||||
state: 'running',
|
||||
recentActivity: reassembler.toActivityItems(),
|
||||
result: reassembler.toString(),
|
||||
});
|
||||
}
|
||||
|
||||
const {
|
||||
contextId: newContextId,
|
||||
taskId: newTaskId,
|
||||
clearTaskId,
|
||||
} = extractIdsFromResponse(chunk);
|
||||
|
||||
if (newContextId) {
|
||||
this.contextId = newContextId;
|
||||
}
|
||||
|
||||
this.taskId = clearTaskId ? undefined : (newTaskId ?? this.taskId);
|
||||
}
|
||||
|
||||
if (!finalResponse) {
|
||||
throw new Error('No response from remote agent.');
|
||||
}
|
||||
|
||||
const finalOutput = reassembler.toString();
|
||||
|
||||
debugLogger.debug(
|
||||
`[RemoteAgent] Final response from ${this.definition.name}:\n${JSON.stringify(finalResponse, null, 2)}`,
|
||||
);
|
||||
|
||||
const finalProgress: SubagentProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName,
|
||||
state: 'completed',
|
||||
result: finalOutput,
|
||||
recentActivity: reassembler.toActivityItems(),
|
||||
};
|
||||
const result = await session.getResult();
|
||||
|
||||
// Emit final completed progress
|
||||
if (updateOutput) {
|
||||
updateOutput(finalProgress);
|
||||
const finalProgress = session.getLatestProgress();
|
||||
if (finalProgress) updateOutput(finalProgress);
|
||||
}
|
||||
|
||||
return {
|
||||
llmContent: [{ text: finalOutput }],
|
||||
returnDisplay: finalProgress,
|
||||
};
|
||||
return result;
|
||||
} catch (error: unknown) {
|
||||
const partialOutput = reassembler.toString();
|
||||
// Surface structured, user-friendly error messages.
|
||||
const partialProgress = session.getLatestProgress();
|
||||
const partialOutput =
|
||||
typeof partialProgress?.result === 'string'
|
||||
? partialProgress.result
|
||||
: '';
|
||||
const errorMessage = this.formatExecutionError(error);
|
||||
const fullDisplay = partialOutput
|
||||
? `${partialOutput}\n\n${errorMessage}`
|
||||
@@ -253,7 +158,7 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
agentName,
|
||||
state: 'error',
|
||||
result: fullDisplay,
|
||||
recentActivity: reassembler.toActivityItems(),
|
||||
recentActivity: partialProgress?.recentActivity ?? [],
|
||||
};
|
||||
|
||||
if (updateOutput) {
|
||||
@@ -265,11 +170,9 @@ export class RemoteAgentInvocation extends BaseToolInvocation<
|
||||
returnDisplay: errorProgress,
|
||||
};
|
||||
} finally {
|
||||
// Persist state even on partial failures or aborts to maintain conversational continuity.
|
||||
RemoteAgentInvocation.sessionState.set(this.definition.name, {
|
||||
contextId: this.contextId,
|
||||
taskId: this.taskId,
|
||||
});
|
||||
_signal.removeEventListener('abort', abortListener);
|
||||
unsubscribeProgress();
|
||||
unsubscribeParent?.();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,776 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import {
|
||||
describe,
|
||||
it,
|
||||
expect,
|
||||
vi,
|
||||
beforeEach,
|
||||
afterEach,
|
||||
type Mock,
|
||||
} from 'vitest';
|
||||
import { RemoteSubagentSession } from './remote-subagent-protocol.js';
|
||||
|
||||
import { A2AAuthProviderFactory } from './auth-provider/factory.js';
|
||||
import type { RemoteAgentDefinition, SubagentProgress } from './types.js';
|
||||
import { createMockMessageBus } from '../test-utils/mock-message-bus.js';
|
||||
import type { AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
import type { Config } from '../config/config.js';
|
||||
import type { A2AAuthProvider } from './auth-provider/types.js';
|
||||
|
||||
// Mock A2AClientManager at module level
|
||||
vi.mock('./a2a-client-manager.js', () => ({
|
||||
A2AClientManager: vi.fn().mockImplementation(() => ({
|
||||
getClient: vi.fn(),
|
||||
loadAgent: vi.fn(),
|
||||
sendMessageStream: vi.fn(),
|
||||
})),
|
||||
}));
|
||||
|
||||
// Mock A2AAuthProviderFactory
|
||||
vi.mock('./auth-provider/factory.js', () => ({
|
||||
A2AAuthProviderFactory: {
|
||||
create: vi.fn(),
|
||||
},
|
||||
}));
|
||||
|
||||
const mockDefinition: RemoteAgentDefinition = {
|
||||
name: 'test-remote-agent',
|
||||
kind: 'remote',
|
||||
agentCardUrl: 'http://test-agent/card',
|
||||
displayName: 'Test Remote Agent',
|
||||
description: 'A test remote agent',
|
||||
inputConfig: {
|
||||
inputSchema: { type: 'object' },
|
||||
},
|
||||
};
|
||||
|
||||
function makeChunk(text: string) {
|
||||
return {
|
||||
kind: 'message' as const,
|
||||
messageId: `msg-${Math.random()}`,
|
||||
role: 'agent' as const,
|
||||
parts: [{ kind: 'text' as const, text }],
|
||||
};
|
||||
}
|
||||
|
||||
describe('RemoteSubagentSession (protocol)', () => {
|
||||
let mockClientManager: {
|
||||
getClient: Mock;
|
||||
loadAgent: Mock;
|
||||
sendMessageStream: Mock;
|
||||
};
|
||||
let mockContext: AgentLoopContext;
|
||||
let mockMessageBus: ReturnType<typeof createMockMessageBus>;
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
|
||||
// Static session state is not cleared between tests — each test uses a
|
||||
// unique agent name to avoid cross-test contamination.
|
||||
|
||||
mockClientManager = {
|
||||
getClient: vi.fn().mockReturnValue(undefined), // client not yet loaded
|
||||
loadAgent: vi.fn().mockResolvedValue(undefined),
|
||||
sendMessageStream: vi.fn(),
|
||||
};
|
||||
|
||||
const mockConfig = {
|
||||
getA2AClientManager: vi.fn().mockReturnValue(mockClientManager),
|
||||
injectionService: {
|
||||
getLatestInjectionIndex: vi.fn().mockReturnValue(0),
|
||||
},
|
||||
} as unknown as Config;
|
||||
|
||||
mockContext = { config: mockConfig } as unknown as AgentLoopContext;
|
||||
mockMessageBus = createMockMessageBus();
|
||||
|
||||
// Default: sendMessageStream yields one chunk with "Hello"
|
||||
mockClientManager.sendMessageStream.mockImplementation(async function* () {
|
||||
yield makeChunk('Hello');
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
// Helper: run a session with the default or custom stream and collect events
|
||||
async function runSession(
|
||||
definition: RemoteAgentDefinition = mockDefinition,
|
||||
query = 'test query',
|
||||
) {
|
||||
const session = new RemoteSubagentSession(
|
||||
definition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
await session.send({ message: [{ type: 'text', text: query }] });
|
||||
const result = await session.getResult();
|
||||
return { session, events, result };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Lifecycle events
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('lifecycle events', () => {
|
||||
it('emits agent_start then agent_end(completed) on success', async () => {
|
||||
const { events } = await runSession();
|
||||
|
||||
const types = events.map((e) => e.type);
|
||||
expect(types[0]).toBe('agent_start');
|
||||
expect(types[types.length - 1]).toBe('agent_end');
|
||||
const end = events[events.length - 1];
|
||||
if (end.type === 'agent_end') {
|
||||
expect(end.reason).toBe('completed');
|
||||
}
|
||||
});
|
||||
|
||||
it('emits agent_start exactly once', async () => {
|
||||
const { events } = await runSession();
|
||||
expect(events.filter((e) => e.type === 'agent_start')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('emits agent_end exactly once on error path', async () => {
|
||||
mockClientManager.sendMessageStream.mockReturnValue({
|
||||
[Symbol.asyncIterator]() {
|
||||
return {
|
||||
async next(): Promise<IteratorResult<never>> {
|
||||
throw new Error('stream error');
|
||||
},
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow('stream error');
|
||||
|
||||
expect(events.filter((e) => e.type === 'agent_end')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('all events share the same streamId', async () => {
|
||||
const { events } = await runSession();
|
||||
const streamIds = new Set(events.map((e) => e.streamId));
|
||||
expect(streamIds.size).toBe(1);
|
||||
});
|
||||
|
||||
it('message returns a non-null streamId; unsupported payload returns null', async () => {
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const updateResult = await session.send({
|
||||
update: { config: { key: 'val' } },
|
||||
});
|
||||
expect(updateResult.streamId).toBeNull();
|
||||
|
||||
const messageResult = await session.send({
|
||||
message: [{ type: 'text', text: 'q' }],
|
||||
});
|
||||
expect(messageResult.streamId).not.toBeNull();
|
||||
// complete the session to avoid dangling execution
|
||||
await session.getResult();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Chunk → AgentEvent translation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('chunk → AgentEvent translation', () => {
|
||||
it('each A2A chunk produces a message event with current accumulated text', async () => {
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
yield makeChunk('Hello');
|
||||
yield makeChunk(' world');
|
||||
},
|
||||
);
|
||||
|
||||
const { events } = await runSession();
|
||||
|
||||
const msgEvents = events.filter((e) => e.type === 'message');
|
||||
expect(msgEvents.length).toBeGreaterThanOrEqual(1);
|
||||
// Final message event should contain the accumulated text
|
||||
const lastMsg = msgEvents[msgEvents.length - 1];
|
||||
if (lastMsg?.type === 'message') {
|
||||
const textContent = lastMsg.content.find((c) => c.type === 'text');
|
||||
expect(textContent).toBeDefined();
|
||||
if (textContent?.type === 'text') {
|
||||
expect(textContent.text).toContain('Hello');
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('getLatestProgress() is updated per chunk with state running', async () => {
|
||||
let capturedProgress: SubagentProgress | undefined;
|
||||
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
yield makeChunk('Partial');
|
||||
},
|
||||
);
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
session.subscribe((e) => {
|
||||
if (e.type === 'message') {
|
||||
capturedProgress = session.getLatestProgress();
|
||||
}
|
||||
});
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
// During streaming, progress should be 'running'
|
||||
expect(capturedProgress).toBeDefined();
|
||||
// Note: by the time we check, progress may be 'completed'.
|
||||
// During the message event, it was 'running'.
|
||||
expect(capturedProgress?.isSubagentProgress).toBe(true);
|
||||
expect(capturedProgress?.agentName).toBe('Test Remote Agent');
|
||||
});
|
||||
|
||||
it('getLatestProgress() state is completed after getResult() resolves', async () => {
|
||||
const { session } = await runSession();
|
||||
const progress = session.getLatestProgress();
|
||||
expect(progress?.state).toBe('completed');
|
||||
expect(progress?.result).toBe('Hello');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// getResult() promise
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('getResult()', () => {
|
||||
it('resolves with ToolResult containing llmContent and SubagentProgress returnDisplay', async () => {
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
yield makeChunk('Result text');
|
||||
},
|
||||
);
|
||||
|
||||
const { result } = await runSession();
|
||||
|
||||
expect(result.llmContent).toEqual([{ text: 'Result text' }]);
|
||||
const display = result.returnDisplay as SubagentProgress;
|
||||
expect(display.isSubagentProgress).toBe(true);
|
||||
expect(display.state).toBe('completed');
|
||||
expect(display.result).toBe('Result text');
|
||||
expect(display.agentName).toBe('Test Remote Agent');
|
||||
});
|
||||
|
||||
it('rejects when stream throws a non-A2A error', async () => {
|
||||
mockClientManager.sendMessageStream.mockReturnValue({
|
||||
[Symbol.asyncIterator]() {
|
||||
return {
|
||||
async next(): Promise<IteratorResult<never>> {
|
||||
throw new Error('network failure');
|
||||
},
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow();
|
||||
});
|
||||
|
||||
it('resolves even with empty stream (empty final output)', async () => {
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
// yield nothing
|
||||
},
|
||||
);
|
||||
|
||||
const { result } = await runSession();
|
||||
expect(result.llmContent).toEqual([{ text: '' }]);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Session state persistence
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('session state persistence', () => {
|
||||
it('second call reuses contextId captured from first call', async () => {
|
||||
const agentName = 'persistent-agent';
|
||||
const persistDef: RemoteAgentDefinition = {
|
||||
...mockDefinition,
|
||||
name: agentName,
|
||||
};
|
||||
|
||||
let callCount = 0;
|
||||
mockClientManager.sendMessageStream.mockImplementation(async function* (
|
||||
_name: string,
|
||||
_query: string,
|
||||
opts: { contextId?: string },
|
||||
) {
|
||||
callCount++;
|
||||
if (callCount === 1) {
|
||||
// First call: return a chunk that yields a contextId
|
||||
yield {
|
||||
kind: 'message' as const,
|
||||
messageId: 'msg-1',
|
||||
role: 'agent' as const,
|
||||
contextId: 'ctx-from-server',
|
||||
parts: [{ kind: 'text' as const, text: 'First response' }],
|
||||
};
|
||||
} else {
|
||||
// Second call: caller should have passed the contextId
|
||||
expect(opts.contextId).toBe('ctx-from-server');
|
||||
yield makeChunk('Second response');
|
||||
}
|
||||
});
|
||||
|
||||
// First call
|
||||
const session1 = new RemoteSubagentSession(
|
||||
persistDef,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session1.send({ message: [{ type: 'text', text: 'first' }] });
|
||||
await session1.getResult();
|
||||
|
||||
// Second call — different session but same agent name → should reuse contextId
|
||||
const session2 = new RemoteSubagentSession(
|
||||
persistDef,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session2.send({ message: [{ type: 'text', text: 'second' }] });
|
||||
await session2.getResult();
|
||||
|
||||
expect(callCount).toBe(2);
|
||||
});
|
||||
|
||||
it('different agent names have independent session state', async () => {
|
||||
const def1: RemoteAgentDefinition = {
|
||||
...mockDefinition,
|
||||
name: 'agent-alpha',
|
||||
};
|
||||
const def2: RemoteAgentDefinition = {
|
||||
...mockDefinition,
|
||||
name: 'agent-beta',
|
||||
};
|
||||
|
||||
const capturedContextIds: Array<string | undefined> = [];
|
||||
mockClientManager.sendMessageStream.mockImplementation(async function* (
|
||||
_name: string,
|
||||
_query: string,
|
||||
opts: { contextId?: string },
|
||||
) {
|
||||
capturedContextIds.push(opts.contextId);
|
||||
yield {
|
||||
kind: 'message' as const,
|
||||
messageId: 'msg-1',
|
||||
role: 'agent' as const,
|
||||
contextId: `ctx-for-${_name}`,
|
||||
parts: [{ kind: 'text' as const, text: 'ok' }],
|
||||
};
|
||||
});
|
||||
|
||||
const session1 = new RemoteSubagentSession(
|
||||
def1,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session1.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session1.getResult();
|
||||
|
||||
const session2 = new RemoteSubagentSession(
|
||||
def2,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session2.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session2.getResult();
|
||||
|
||||
// Both start with no contextId (different agents, different state entries)
|
||||
expect(capturedContextIds[0]).toBeUndefined();
|
||||
expect(capturedContextIds[1]).toBeUndefined();
|
||||
});
|
||||
|
||||
it('taskId is cleared when a terminal-state task chunk is received', async () => {
|
||||
// A task chunk with a terminal status sets clearTaskId=true, which
|
||||
// should clear this.taskId so it is NOT passed on the next call.
|
||||
const agentName = 'clearTaskId-agent';
|
||||
const def: RemoteAgentDefinition = { ...mockDefinition, name: agentName };
|
||||
|
||||
let callCount = 0;
|
||||
const capturedTaskIds: Array<string | undefined> = [];
|
||||
|
||||
mockClientManager.sendMessageStream.mockImplementation(async function* (
|
||||
_n: string,
|
||||
_q: string,
|
||||
opts: { taskId?: string },
|
||||
) {
|
||||
callCount++;
|
||||
capturedTaskIds.push(opts.taskId);
|
||||
if (callCount === 1) {
|
||||
// First call: yield a task chunk with taskId + terminal status → clearTaskId
|
||||
yield {
|
||||
kind: 'task' as const,
|
||||
id: 'task-123',
|
||||
contextId: 'ctx-1',
|
||||
status: { state: 'completed' as const },
|
||||
};
|
||||
} else {
|
||||
yield makeChunk('done');
|
||||
}
|
||||
});
|
||||
|
||||
const session1 = new RemoteSubagentSession(
|
||||
def,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session1.send({ message: [{ type: 'text', text: 'first' }] });
|
||||
await session1.getResult();
|
||||
|
||||
const session2 = new RemoteSubagentSession(
|
||||
def,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session2.send({ message: [{ type: 'text', text: 'second' }] });
|
||||
await session2.getResult();
|
||||
|
||||
expect(callCount).toBe(2);
|
||||
// First call starts with no taskId
|
||||
expect(capturedTaskIds[0]).toBeUndefined();
|
||||
// Second call: taskId was cleared because terminal-state task chunk was received
|
||||
expect(capturedTaskIds[1]).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Auth setup
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('auth setup', () => {
|
||||
it('no auth → loadAgent called without auth handler', async () => {
|
||||
await runSession();
|
||||
|
||||
expect(mockClientManager.loadAgent).toHaveBeenCalledWith(
|
||||
'test-remote-agent',
|
||||
{ type: 'url', url: 'http://test-agent/card' },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it('definition.auth present → A2AAuthProviderFactory.create called', async () => {
|
||||
const authDef: RemoteAgentDefinition = {
|
||||
...mockDefinition,
|
||||
name: 'auth-agent',
|
||||
auth: {
|
||||
type: 'http' as const,
|
||||
scheme: 'Bearer' as const,
|
||||
token: 'secret',
|
||||
},
|
||||
};
|
||||
|
||||
const mockProvider = {
|
||||
type: 'http' as const,
|
||||
headers: vi.fn().mockResolvedValue({ Authorization: 'Bearer secret' }),
|
||||
shouldRetryWithHeaders: vi.fn(),
|
||||
} as unknown as A2AAuthProvider;
|
||||
(A2AAuthProviderFactory.create as Mock).mockResolvedValue(mockProvider);
|
||||
|
||||
await runSession(authDef, 'q');
|
||||
|
||||
expect(A2AAuthProviderFactory.create).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
agentName: 'auth-agent',
|
||||
agentCardUrl: 'http://test-agent/card',
|
||||
}),
|
||||
);
|
||||
expect(mockClientManager.loadAgent).toHaveBeenCalledWith(
|
||||
'auth-agent',
|
||||
expect.any(Object),
|
||||
mockProvider,
|
||||
);
|
||||
});
|
||||
|
||||
it('auth factory returns undefined → throws error that rejects getResult()', async () => {
|
||||
const authDef: RemoteAgentDefinition = {
|
||||
...mockDefinition,
|
||||
name: 'failing-auth-agent',
|
||||
auth: {
|
||||
type: 'http' as const,
|
||||
scheme: 'Bearer' as const,
|
||||
token: 'secret',
|
||||
},
|
||||
};
|
||||
|
||||
(A2AAuthProviderFactory.create as Mock).mockResolvedValue(undefined);
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
authDef,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow(
|
||||
"Failed to create auth provider for agent 'failing-auth-agent'",
|
||||
);
|
||||
});
|
||||
|
||||
it('agent already loaded → loadAgent not called again', async () => {
|
||||
// Return a client object (truthy) so getClient returns defined
|
||||
mockClientManager.getClient.mockReturnValue({});
|
||||
|
||||
await runSession();
|
||||
|
||||
expect(mockClientManager.loadAgent).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Error handling
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('error handling', () => {
|
||||
it('stream error → error event + agent_end(failed)', async () => {
|
||||
mockClientManager.sendMessageStream.mockReturnValue({
|
||||
[Symbol.asyncIterator]() {
|
||||
return {
|
||||
async next(): Promise<IteratorResult<never>> {
|
||||
throw new Error('network error');
|
||||
},
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow();
|
||||
|
||||
const errEvent = events.find((e) => e.type === 'error');
|
||||
expect(errEvent).toBeDefined();
|
||||
|
||||
const endEvent = events.find((e) => e.type === 'agent_end');
|
||||
expect(endEvent).toBeDefined();
|
||||
if (endEvent?.type === 'agent_end') {
|
||||
expect(endEvent.reason).toBe('failed');
|
||||
}
|
||||
});
|
||||
|
||||
it('missing A2AClientManager → rejects getResult()', async () => {
|
||||
const mockConfig = {
|
||||
getA2AClientManager: vi.fn().mockReturnValue(undefined),
|
||||
injectionService: {
|
||||
getLatestInjectionIndex: vi.fn().mockReturnValue(0),
|
||||
},
|
||||
} as unknown as Config;
|
||||
const noClientContext = {
|
||||
config: mockConfig,
|
||||
} as unknown as AgentLoopContext;
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
noClientContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await expect(session.getResult()).rejects.toThrow(
|
||||
'A2AClientManager not available',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Subscription
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('subscription', () => {
|
||||
it('unsubscribe stops event delivery', async () => {
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const received: AgentEvent[] = [];
|
||||
const unsub = session.subscribe((e) => received.push(e));
|
||||
unsub();
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(received).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('multiple subscribers all receive events', async () => {
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const events1: AgentEvent[] = [];
|
||||
const events2: AgentEvent[] = [];
|
||||
session.subscribe((e) => events1.push(e));
|
||||
session.subscribe((e) => events2.push(e));
|
||||
|
||||
await session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
await session.getResult();
|
||||
|
||||
expect(events1.length).toBeGreaterThan(0);
|
||||
expect(events1).toEqual(events2);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Abort
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('abort()', () => {
|
||||
it('abort() causes agent_end(reason:aborted)', async () => {
|
||||
let resolveChunk: (() => void) | undefined;
|
||||
|
||||
// Stream that blocks until we abort
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
// Hang until aborted
|
||||
await new Promise<void>((resolve) => {
|
||||
resolveChunk = resolve;
|
||||
});
|
||||
yield makeChunk('Too late');
|
||||
},
|
||||
);
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
|
||||
void session.send({ message: [{ type: 'text', text: 'q' }] });
|
||||
|
||||
// Wait for agent_start to be emitted before aborting
|
||||
await vi.waitFor(() => {
|
||||
expect(events.some((e) => e.type === 'agent_start')).toBe(true);
|
||||
});
|
||||
|
||||
await session.abort();
|
||||
|
||||
// Resolve the hanging chunk generator so it can check the signal
|
||||
resolveChunk?.();
|
||||
|
||||
await expect(session.getResult()).rejects.toThrow();
|
||||
|
||||
const endEvent = events.find((e) => e.type === 'agent_end');
|
||||
expect(endEvent).toBeDefined();
|
||||
if (endEvent?.type === 'agent_end') {
|
||||
expect(endEvent.reason).toBe('aborted');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// sendMessageStream call args
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('sendMessageStream call arguments', () => {
|
||||
it('passes the query string from the message payload', async () => {
|
||||
await runSession(mockDefinition, 'my specific query');
|
||||
|
||||
expect(mockClientManager.sendMessageStream).toHaveBeenCalledWith(
|
||||
'test-remote-agent',
|
||||
'my specific query',
|
||||
expect.objectContaining({ signal: expect.any(Object) }),
|
||||
);
|
||||
});
|
||||
|
||||
it('uses DEFAULT_QUERY_STRING when message text is empty', async () => {
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
await session.send({ message: [{ type: 'text', text: '' }] });
|
||||
await session.getResult();
|
||||
|
||||
// DEFAULT_QUERY_STRING = 'Get Started!'
|
||||
expect(mockClientManager.sendMessageStream).toHaveBeenCalledWith(
|
||||
'test-remote-agent',
|
||||
'Get Started!',
|
||||
expect.objectContaining({ signal: expect.any(Object) }),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Concurrent send() guard
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('concurrent send() guard', () => {
|
||||
it('calling send() while a stream is active throws', async () => {
|
||||
let resolveChunk!: () => void;
|
||||
|
||||
mockClientManager.sendMessageStream.mockImplementation(
|
||||
async function* () {
|
||||
// Block until test releases the chunk
|
||||
await new Promise<void>((resolve) => {
|
||||
resolveChunk = resolve;
|
||||
});
|
||||
yield makeChunk('late');
|
||||
},
|
||||
);
|
||||
|
||||
const session = new RemoteSubagentSession(
|
||||
mockDefinition,
|
||||
mockContext,
|
||||
mockMessageBus,
|
||||
);
|
||||
|
||||
void session.send({ message: [{ type: 'text', text: 'first' }] });
|
||||
|
||||
// Wait for the stream to actually start (agent_start emitted)
|
||||
const events: AgentEvent[] = [];
|
||||
session.subscribe((e) => events.push(e));
|
||||
await vi.waitFor(() => {
|
||||
expect(events.some((e) => e.type === 'agent_start')).toBe(true);
|
||||
});
|
||||
|
||||
// Second send() while first stream is active must throw
|
||||
await expect(
|
||||
session.send({ message: [{ type: 'text', text: 'second' }] }),
|
||||
).rejects.toThrow('cannot be called while a stream is active');
|
||||
|
||||
// Clean up: release the blocked generator so getResult() can settle
|
||||
resolveChunk();
|
||||
await session.getResult().catch(() => {});
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,461 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
/**
|
||||
* @fileoverview RemoteSubagentProtocol — wraps A2A remote agent streaming
|
||||
* behind the AgentProtocol interface.
|
||||
*
|
||||
* Pattern mirrors LocalSubagentProtocol and LegacyAgentProtocol, but the loop
|
||||
* body drives A2AClientManager instead of LocalAgentExecutor.
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import type { AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import { AgentSession } from '../agent/agent-session.js';
|
||||
import type {
|
||||
AgentProtocol,
|
||||
AgentSend,
|
||||
AgentEvent,
|
||||
StreamEndReason,
|
||||
Unsubscribe,
|
||||
ContentPart,
|
||||
} from '../agent/types.js';
|
||||
import type { ToolResult } from '../tools/tools.js';
|
||||
import {
|
||||
DEFAULT_QUERY_STRING,
|
||||
type RemoteAgentDefinition,
|
||||
type SubagentProgress,
|
||||
getRemoteAgentTargetUrl,
|
||||
getAgentCardLoadOptions,
|
||||
} from './types.js';
|
||||
import { A2AResultReassembler, extractIdsFromResponse } from './a2aUtils.js';
|
||||
import type { AuthenticationHandler } from '@a2a-js/sdk/client';
|
||||
import { A2AAuthProviderFactory } from './auth-provider/factory.js';
|
||||
import { A2AAgentError } from './a2a-errors.js';
|
||||
import { debugLogger } from '../utils/debugLogger.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function isAbortLikeError(err: unknown): boolean {
|
||||
return err instanceof Error && err.name === 'AbortError';
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RemoteSubagentProtocol
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
class RemoteSubagentProtocol implements AgentProtocol {
|
||||
private _events: AgentEvent[] = [];
|
||||
private _subscribers = new Set<(event: AgentEvent) => void>();
|
||||
private _streamId: string = randomUUID();
|
||||
private _eventCounter = 0;
|
||||
private _agentStartEmitted = false;
|
||||
private _agentEndEmitted = false;
|
||||
private _activeStreamId: string | undefined;
|
||||
private _abortController = new AbortController();
|
||||
|
||||
// Session state persisted across sends (mirrors RemoteAgentInvocation)
|
||||
private contextId: string | undefined;
|
||||
private taskId: string | undefined;
|
||||
private authHandler: AuthenticationHandler | undefined;
|
||||
|
||||
// Agent display name (for SubagentProgress construction)
|
||||
private readonly _agentName: string;
|
||||
|
||||
// Latest SubagentProgress — updated per chunk, used for error recovery
|
||||
private _latestProgress: SubagentProgress | undefined;
|
||||
|
||||
// Result promise wiring
|
||||
private _resultResolve!: (result: ToolResult) => void;
|
||||
private _resultReject!: (err: unknown) => void;
|
||||
private readonly _resultPromise: Promise<ToolResult>;
|
||||
|
||||
constructor(
|
||||
private readonly definition: RemoteAgentDefinition,
|
||||
private readonly context: AgentLoopContext,
|
||||
_messageBus: MessageBus,
|
||||
) {
|
||||
this._agentName = definition.displayName ?? definition.name;
|
||||
this._resultPromise = new Promise<ToolResult>((resolve, reject) => {
|
||||
this._resultResolve = resolve;
|
||||
this._resultReject = reject;
|
||||
});
|
||||
|
||||
// Restore persisted session state (mirrors static map in RemoteAgentInvocation)
|
||||
const priorState = RemoteSubagentProtocol._sessionState.get(
|
||||
definition.name,
|
||||
);
|
||||
if (priorState) {
|
||||
this.contextId = priorState.contextId;
|
||||
this.taskId = priorState.taskId;
|
||||
}
|
||||
}
|
||||
|
||||
// Per-agent session state, mirrors RemoteAgentInvocation.sessionState
|
||||
private static readonly _sessionState = new Map<
|
||||
string,
|
||||
{ contextId?: string; taskId?: string }
|
||||
>();
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AgentProtocol interface
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
get events(): readonly AgentEvent[] {
|
||||
return this._events;
|
||||
}
|
||||
|
||||
subscribe(callback: (event: AgentEvent) => void): Unsubscribe {
|
||||
this._subscribers.add(callback);
|
||||
return () => {
|
||||
this._subscribers.delete(callback);
|
||||
};
|
||||
}
|
||||
|
||||
async send(payload: AgentSend): Promise<{ streamId: string | null }> {
|
||||
if ('message' in payload && payload.message) {
|
||||
if (this._activeStreamId) {
|
||||
throw new Error(
|
||||
'RemoteSubagentProtocol.send() cannot be called while a stream is active.',
|
||||
);
|
||||
}
|
||||
|
||||
const query =
|
||||
payload.message
|
||||
.filter((p): p is ContentPart & { type: 'text' } => p.type === 'text')
|
||||
.map((p) => p.text)
|
||||
.join('') || DEFAULT_QUERY_STRING;
|
||||
|
||||
this._beginNewStream();
|
||||
const streamId = this._streamId;
|
||||
|
||||
setTimeout(() => {
|
||||
void this._runStreamInBackground(query);
|
||||
}, 0);
|
||||
|
||||
return { streamId };
|
||||
}
|
||||
|
||||
// update/action/elicitations not used for remote agents
|
||||
return { streamId: null };
|
||||
}
|
||||
|
||||
async abort(): Promise<void> {
|
||||
this._abortController.abort();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Protocol-specific: result access
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
getResult(): Promise<ToolResult> {
|
||||
return this._resultPromise;
|
||||
}
|
||||
|
||||
getLatestProgress(): SubagentProgress | undefined {
|
||||
return this._latestProgress;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core: A2A streaming
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
private _beginNewStream(): void {
|
||||
this._streamId = randomUUID();
|
||||
this._eventCounter = 0;
|
||||
this._abortController = new AbortController();
|
||||
this._agentStartEmitted = false;
|
||||
this._agentEndEmitted = false;
|
||||
this._activeStreamId = this._streamId;
|
||||
}
|
||||
|
||||
private async _runStreamInBackground(query: string): Promise<void> {
|
||||
this._ensureAgentStart();
|
||||
try {
|
||||
await this._runStream(query);
|
||||
} catch (err: unknown) {
|
||||
// Save state before rejecting so callers see updated contextId/taskId
|
||||
// immediately after the returned promise settles.
|
||||
this._saveSessionState();
|
||||
if (this._abortController.signal.aborted || isAbortLikeError(err)) {
|
||||
this._ensureAgentEnd('aborted');
|
||||
this._resultReject(err);
|
||||
} else {
|
||||
this._emitErrorAndAgentEnd(err);
|
||||
this._resultReject(err);
|
||||
}
|
||||
this._clearActiveStream();
|
||||
}
|
||||
}
|
||||
|
||||
private _saveSessionState(): void {
|
||||
RemoteSubagentProtocol._sessionState.set(this.definition.name, {
|
||||
contextId: this.contextId,
|
||||
taskId: this.taskId,
|
||||
});
|
||||
}
|
||||
|
||||
private async _runStream(query: string): Promise<void> {
|
||||
const clientManager = this.context.config.getA2AClientManager();
|
||||
if (!clientManager) {
|
||||
throw new Error(
|
||||
`RemoteSubagentProtocol: A2AClientManager not available for '${this.definition.name}'.`,
|
||||
);
|
||||
}
|
||||
|
||||
const authHandler = await this._getAuthHandler();
|
||||
if (!clientManager.getClient(this.definition.name)) {
|
||||
await clientManager.loadAgent(
|
||||
this.definition.name,
|
||||
getAgentCardLoadOptions(this.definition),
|
||||
authHandler,
|
||||
);
|
||||
}
|
||||
|
||||
const reassembler = new A2AResultReassembler();
|
||||
let prevText = '';
|
||||
|
||||
const stream = clientManager.sendMessageStream(
|
||||
this.definition.name,
|
||||
query,
|
||||
{
|
||||
contextId: this.contextId,
|
||||
taskId: this.taskId,
|
||||
signal: this._abortController.signal,
|
||||
},
|
||||
);
|
||||
|
||||
for await (const chunk of stream) {
|
||||
if (this._abortController.signal.aborted) {
|
||||
this._saveSessionState();
|
||||
this._finishStream('aborted');
|
||||
this._resultReject(new Error('Operation aborted'));
|
||||
this._clearActiveStream();
|
||||
return;
|
||||
}
|
||||
|
||||
reassembler.update(chunk);
|
||||
|
||||
const {
|
||||
contextId: newContextId,
|
||||
taskId: newTaskId,
|
||||
clearTaskId,
|
||||
} = extractIdsFromResponse(chunk);
|
||||
if (newContextId) this.contextId = newContextId;
|
||||
this.taskId = clearTaskId ? undefined : (newTaskId ?? this.taskId);
|
||||
|
||||
// Update latest progress snapshot (for invocation's error recovery)
|
||||
this._latestProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName: this._agentName,
|
||||
state: 'running',
|
||||
recentActivity: reassembler.toActivityItems(),
|
||||
result: reassembler.toString(),
|
||||
};
|
||||
|
||||
// Emit delta as a message event
|
||||
const currentText = reassembler.toString();
|
||||
const delta = currentText.slice(prevText.length);
|
||||
if (delta) {
|
||||
this._emit([
|
||||
this._makeEvent('message', {
|
||||
role: 'agent',
|
||||
content: [{ type: 'text', text: currentText }],
|
||||
}),
|
||||
]);
|
||||
prevText = currentText;
|
||||
}
|
||||
}
|
||||
|
||||
const finalOutput = reassembler.toString();
|
||||
debugLogger.debug(
|
||||
`[RemoteSubagentProtocol] ${this.definition.name} finished, output length: ${finalOutput.length}`,
|
||||
);
|
||||
|
||||
const finalProgress: SubagentProgress = {
|
||||
isSubagentProgress: true,
|
||||
agentName: this._agentName,
|
||||
state: 'completed',
|
||||
result: finalOutput,
|
||||
recentActivity: reassembler.toActivityItems(),
|
||||
};
|
||||
this._latestProgress = finalProgress;
|
||||
|
||||
this._finishStream('completed');
|
||||
|
||||
// Save state before resolving so callers see updated contextId/taskId
|
||||
// immediately after the returned promise settles.
|
||||
this._saveSessionState();
|
||||
this._resultResolve({
|
||||
llmContent: [{ text: finalOutput }],
|
||||
returnDisplay: finalProgress,
|
||||
});
|
||||
}
|
||||
|
||||
private async _getAuthHandler(): Promise<AuthenticationHandler | undefined> {
|
||||
if (this.authHandler) return this.authHandler;
|
||||
if (!this.definition.auth) return undefined;
|
||||
|
||||
const targetUrl = getRemoteAgentTargetUrl(this.definition);
|
||||
const provider = await A2AAuthProviderFactory.create({
|
||||
authConfig: this.definition.auth,
|
||||
agentName: this.definition.name,
|
||||
targetUrl,
|
||||
agentCardUrl: this.definition.agentCardUrl,
|
||||
});
|
||||
if (!provider) {
|
||||
throw new Error(
|
||||
`Failed to create auth provider for agent '${this.definition.name}'`,
|
||||
);
|
||||
}
|
||||
this.authHandler = provider;
|
||||
return this.authHandler;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
private _emit(events: AgentEvent[]): void {
|
||||
if (events.length === 0) return;
|
||||
const subscribers = [...this._subscribers];
|
||||
for (const event of events) {
|
||||
this._events.push(event);
|
||||
if (event.type === 'agent_end') {
|
||||
this._agentEndEmitted = true;
|
||||
}
|
||||
for (const sub of subscribers) {
|
||||
sub(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private _clearActiveStream(): void {
|
||||
this._activeStreamId = undefined;
|
||||
}
|
||||
|
||||
private _ensureAgentStart(): void {
|
||||
if (!this._agentStartEmitted) {
|
||||
this._agentStartEmitted = true;
|
||||
this._emit([this._makeEvent('agent_start', {})]);
|
||||
}
|
||||
}
|
||||
|
||||
private _ensureAgentEnd(reason: StreamEndReason = 'completed'): void {
|
||||
if (!this._agentEndEmitted && this._agentStartEmitted) {
|
||||
this._emit([this._makeEvent('agent_end', { reason })]);
|
||||
}
|
||||
}
|
||||
|
||||
private _finishStream(
|
||||
reason: StreamEndReason,
|
||||
data?: Record<string, unknown>,
|
||||
): void {
|
||||
if (data && !this._agentEndEmitted) {
|
||||
this._emit([this._makeEvent('agent_end', { reason, data })]);
|
||||
this._agentEndEmitted = true;
|
||||
} else {
|
||||
this._ensureAgentEnd(reason);
|
||||
}
|
||||
this._clearActiveStream();
|
||||
}
|
||||
|
||||
private _emitErrorAndAgentEnd(err: unknown): void {
|
||||
const message = this._formatError(err);
|
||||
this._ensureAgentStart();
|
||||
|
||||
const meta: Record<string, unknown> = {};
|
||||
if (err instanceof Error) {
|
||||
meta['errorName'] = err.constructor.name;
|
||||
}
|
||||
|
||||
this._emit([
|
||||
this._makeEvent('error', {
|
||||
status: 'INTERNAL',
|
||||
message,
|
||||
fatal: true,
|
||||
...(Object.keys(meta).length > 0 ? { _meta: meta } : {}),
|
||||
}),
|
||||
]);
|
||||
this._ensureAgentEnd('failed');
|
||||
}
|
||||
|
||||
private _formatError(error: unknown): string {
|
||||
if (error instanceof A2AAgentError) {
|
||||
return error.userMessage;
|
||||
}
|
||||
return `Error calling remote agent: ${error instanceof Error ? error.message : String(error)}`;
|
||||
}
|
||||
|
||||
private _nextEventFields() {
|
||||
return {
|
||||
id: `${this._streamId}-${this._eventCounter++}`,
|
||||
timestamp: new Date().toISOString(),
|
||||
streamId: this._streamId,
|
||||
};
|
||||
}
|
||||
|
||||
private _makeEvent<T extends AgentEvent['type']>(
|
||||
type: T,
|
||||
payload: Omit<AgentEvent<T>, 'id' | 'timestamp' | 'streamId' | 'type'>,
|
||||
): AgentEvent {
|
||||
// eslint-disable-next-line @typescript-eslint/no-unsafe-type-assertion
|
||||
return {
|
||||
...this._nextEventFields(),
|
||||
type,
|
||||
...payload,
|
||||
} as AgentEvent;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public export
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class RemoteSubagentSession extends AgentSession {
|
||||
private readonly _remoteProtocol: RemoteSubagentProtocol;
|
||||
|
||||
constructor(
|
||||
definition: RemoteAgentDefinition,
|
||||
context: AgentLoopContext,
|
||||
messageBus: MessageBus,
|
||||
) {
|
||||
const protocol = new RemoteSubagentProtocol(
|
||||
definition,
|
||||
context,
|
||||
messageBus,
|
||||
);
|
||||
super(protocol);
|
||||
this._remoteProtocol = protocol;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the ToolResult once the remote agent stream completes.
|
||||
* Used by RemoteAgentInvocation to return the result.
|
||||
*/
|
||||
getResult(): Promise<ToolResult> {
|
||||
return this._remoteProtocol.getResult();
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the most recent SubagentProgress snapshot, updated per streaming
|
||||
* chunk. Useful for constructing error progress when getResult() rejects.
|
||||
*/
|
||||
getLatestProgress(): SubagentProgress | undefined {
|
||||
return this._remoteProtocol.getLatestProgress();
|
||||
}
|
||||
|
||||
/**
|
||||
* Convenience: start execution with a query string.
|
||||
* Equivalent to send({message: [{type:'text', text: query}]}).
|
||||
*/
|
||||
async startWithQuery(query: string): Promise<{ streamId: string | null }> {
|
||||
return this.send({ message: [{ type: 'text', text: query }] });
|
||||
}
|
||||
}
|
||||
@@ -139,6 +139,7 @@ describe('SubagentToolWrapper', () => {
|
||||
mockMessageBus,
|
||||
mockDefinition.name,
|
||||
mockDefinition.displayName,
|
||||
undefined, // onAgentEvent (not set when wrapper has no onAgentEvent)
|
||||
);
|
||||
});
|
||||
|
||||
@@ -164,6 +165,7 @@ describe('SubagentToolWrapper', () => {
|
||||
specificMessageBus,
|
||||
mockDefinition.name,
|
||||
mockDefinition.displayName,
|
||||
undefined, // onAgentEvent (not set when wrapper has no onAgentEvent)
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ import { RemoteAgentInvocation } from './remote-invocation.js';
|
||||
import { BrowserAgentInvocation } from './browser/browserAgentInvocation.js';
|
||||
import { BROWSER_AGENT_NAME } from './browser/browserAgentDefinition.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
|
||||
/**
|
||||
* A tool wrapper that dynamically exposes a subagent as a standard,
|
||||
@@ -41,6 +42,7 @@ export class SubagentToolWrapper extends BaseDeclarativeTool<
|
||||
private readonly definition: AgentDefinition,
|
||||
private readonly context: AgentLoopContext,
|
||||
messageBus: MessageBus,
|
||||
private readonly onAgentEvent?: (event: AgentEvent) => void,
|
||||
) {
|
||||
super(
|
||||
definition.name,
|
||||
@@ -80,6 +82,7 @@ export class SubagentToolWrapper extends BaseDeclarativeTool<
|
||||
effectiveMessageBus,
|
||||
_toolName,
|
||||
_toolDisplayName,
|
||||
this.onAgentEvent,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -101,6 +104,7 @@ export class SubagentToolWrapper extends BaseDeclarativeTool<
|
||||
effectiveMessageBus,
|
||||
_toolName,
|
||||
_toolDisplayName,
|
||||
this.onAgentEvent,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -121,6 +121,7 @@ describe('SubAgentInvocation', () => {
|
||||
testDefinition,
|
||||
mockConfig,
|
||||
mockMessageBus,
|
||||
undefined, // onAgentEvent (not set on SubAgentInvocation)
|
||||
);
|
||||
});
|
||||
|
||||
@@ -165,6 +166,7 @@ describe('SubAgentInvocation', () => {
|
||||
testRemoteDefinition,
|
||||
mockConfig,
|
||||
mockMessageBus,
|
||||
undefined, // onAgentEvent (not set on SubAgentInvocation)
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ import { type AgentLoopContext } from '../config/agent-loop-context.js';
|
||||
import type { MessageBus } from '../confirmation-bus/message-bus.js';
|
||||
import type { AgentDefinition, AgentInputs } from './types.js';
|
||||
import { SubagentToolWrapper } from './subagent-tool-wrapper.js';
|
||||
import type { AgentEvent } from '../agent/types.js';
|
||||
import { SchemaValidator } from '../utils/schemaValidator.js';
|
||||
import { formatUserHintsForModel } from '../utils/fastAckHelper.js';
|
||||
import { runInDevTraceSpan } from '../telemetry/trace.js';
|
||||
@@ -33,6 +34,7 @@ export class SubagentTool extends BaseDeclarativeTool<AgentInputs, ToolResult> {
|
||||
private readonly definition: AgentDefinition,
|
||||
private readonly context: AgentLoopContext,
|
||||
messageBus: MessageBus,
|
||||
private readonly onAgentEvent?: (event: AgentEvent) => void,
|
||||
) {
|
||||
const inputSchema = definition.inputConfig.inputSchema;
|
||||
|
||||
@@ -116,6 +118,7 @@ export class SubagentTool extends BaseDeclarativeTool<AgentInputs, ToolResult> {
|
||||
messageBus,
|
||||
_toolName,
|
||||
_toolDisplayName,
|
||||
this.onAgentEvent,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -130,6 +133,7 @@ class SubAgentInvocation extends BaseToolInvocation<AgentInputs, ToolResult> {
|
||||
messageBus: MessageBus,
|
||||
_toolName?: string,
|
||||
_toolDisplayName?: string,
|
||||
private readonly onAgentEvent?: (event: AgentEvent) => void,
|
||||
) {
|
||||
super(
|
||||
params,
|
||||
@@ -229,6 +233,7 @@ class SubAgentInvocation extends BaseToolInvocation<AgentInputs, ToolResult> {
|
||||
definition,
|
||||
this.context,
|
||||
this.messageBus,
|
||||
this.onAgentEvent,
|
||||
);
|
||||
|
||||
return wrapper.build(agentArgs);
|
||||
|
||||
@@ -187,6 +187,8 @@ export * from './agents/agent-scheduler.js';
|
||||
// Export agent session interface
|
||||
export * from './agent/agent-session.js';
|
||||
export * from './agent/legacy-agent-session.js';
|
||||
export { LocalSubagentSession } from './agents/local-subagent-protocol.js';
|
||||
export { RemoteSubagentSession } from './agents/remote-subagent-protocol.js';
|
||||
export * from './agent/event-translator.js';
|
||||
export * from './agent/content-utils.js';
|
||||
// Agent event types — namespaced to avoid collisions with existing exports
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { LinuxSandboxManager } from './LinuxSandboxManager.js';
|
||||
import * as sandboxManager from '../../services/sandboxManager.js';
|
||||
import type { SandboxRequest } from '../../services/sandboxManager.js';
|
||||
import fs from 'node:fs';
|
||||
|
||||
@@ -18,14 +17,16 @@ vi.mock('node:fs', async () => {
|
||||
// @ts-expect-error - Property 'default' does not exist on type 'typeof import("node:fs")'
|
||||
...actual.default,
|
||||
existsSync: vi.fn(() => true),
|
||||
realpathSync: vi.fn((p: string | Buffer) => p.toString()),
|
||||
realpathSync: vi.fn((p) => p.toString()),
|
||||
statSync: vi.fn(() => ({ isDirectory: () => true }) as fs.Stats),
|
||||
mkdirSync: vi.fn(),
|
||||
openSync: vi.fn(),
|
||||
closeSync: vi.fn(),
|
||||
writeFileSync: vi.fn(),
|
||||
},
|
||||
existsSync: vi.fn(() => true),
|
||||
realpathSync: vi.fn((p: string | Buffer) => p.toString()),
|
||||
realpathSync: vi.fn((p) => p.toString()),
|
||||
statSync: vi.fn(() => ({ isDirectory: () => true }) as fs.Stats),
|
||||
mkdirSync: vi.fn(),
|
||||
openSync: vi.fn(),
|
||||
closeSync: vi.fn(),
|
||||
@@ -48,8 +49,12 @@ describe('LinuxSandboxManager', () => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const getBwrapArgs = async (req: SandboxRequest) => {
|
||||
const result = await manager.prepareCommand(req);
|
||||
const getBwrapArgs = async (
|
||||
req: SandboxRequest,
|
||||
customManager?: LinuxSandboxManager,
|
||||
) => {
|
||||
const mgr = customManager || manager;
|
||||
const result = await mgr.prepareCommand(req);
|
||||
expect(result.program).toBe('sh');
|
||||
expect(result.args[0]).toBe('-c');
|
||||
expect(result.args[1]).toBe(
|
||||
@@ -60,41 +65,6 @@ describe('LinuxSandboxManager', () => {
|
||||
return result.args.slice(4);
|
||||
};
|
||||
|
||||
/**
|
||||
* Helper to verify only the dynamic, policy-based binds (e.g. allowedPaths, forbiddenPaths).
|
||||
* It asserts that the base workspace and governance files are present exactly once,
|
||||
* then strips them away, leaving only the dynamic binds for a focused, non-brittle assertion.
|
||||
*/
|
||||
const expectDynamicBinds = (
|
||||
bwrapArgs: string[],
|
||||
expectedDynamicBinds: string[],
|
||||
) => {
|
||||
const bindsIndex = bwrapArgs.indexOf('--seccomp');
|
||||
const allBinds = bwrapArgs.slice(bwrapArgs.indexOf('--bind'), bindsIndex);
|
||||
|
||||
const baseBinds = [
|
||||
'--bind',
|
||||
workspace,
|
||||
workspace,
|
||||
'--ro-bind',
|
||||
`${workspace}/.gitignore`,
|
||||
`${workspace}/.gitignore`,
|
||||
'--ro-bind',
|
||||
`${workspace}/.geminiignore`,
|
||||
`${workspace}/.geminiignore`,
|
||||
'--ro-bind',
|
||||
`${workspace}/.git`,
|
||||
`${workspace}/.git`,
|
||||
];
|
||||
|
||||
// Verify the base binds are present exactly at the beginning
|
||||
expect(allBinds.slice(0, baseBinds.length)).toEqual(baseBinds);
|
||||
|
||||
// Extract the remaining dynamic binds
|
||||
const dynamicBinds = allBinds.slice(baseBinds.length);
|
||||
expect(dynamicBinds).toEqual(expectedDynamicBinds);
|
||||
};
|
||||
|
||||
describe('prepareCommand', () => {
|
||||
it('should correctly format the base command and args', async () => {
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
@@ -117,7 +87,7 @@ describe('LinuxSandboxManager', () => {
|
||||
'/proc',
|
||||
'--tmpfs',
|
||||
'/tmp',
|
||||
'--bind',
|
||||
'--ro-bind-try',
|
||||
workspace,
|
||||
workspace,
|
||||
'--ro-bind',
|
||||
@@ -137,6 +107,73 @@ describe('LinuxSandboxManager', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it('binds workspace read-write when readonly is false', async () => {
|
||||
const customManager = new LinuxSandboxManager({
|
||||
workspace,
|
||||
modeConfig: { readonly: false },
|
||||
});
|
||||
const bwrapArgs = await getBwrapArgs(
|
||||
{
|
||||
command: 'ls',
|
||||
args: [],
|
||||
cwd: workspace,
|
||||
env: {},
|
||||
},
|
||||
customManager,
|
||||
);
|
||||
|
||||
expect(bwrapArgs).toContain('--bind-try');
|
||||
expect(bwrapArgs).toContain(workspace);
|
||||
});
|
||||
|
||||
it('maps network permissions to --share-net', async () => {
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'curl',
|
||||
args: [],
|
||||
cwd: workspace,
|
||||
env: {},
|
||||
policy: { additionalPermissions: { network: true } },
|
||||
});
|
||||
|
||||
expect(bwrapArgs).toContain('--share-net');
|
||||
});
|
||||
|
||||
it('maps explicit write permissions to --bind-try', async () => {
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'touch',
|
||||
args: [],
|
||||
cwd: workspace,
|
||||
env: {},
|
||||
policy: {
|
||||
additionalPermissions: {
|
||||
fileSystem: { write: ['/home/user/workspace/out/dir'] },
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const index = bwrapArgs.indexOf('--bind-try');
|
||||
expect(index).not.toBe(-1);
|
||||
expect(bwrapArgs[index + 1]).toBe('/home/user/workspace/out/dir');
|
||||
});
|
||||
|
||||
it('rejects overrides in plan mode', async () => {
|
||||
const customManager = new LinuxSandboxManager({
|
||||
workspace,
|
||||
modeConfig: { allowOverrides: false },
|
||||
});
|
||||
await expect(
|
||||
customManager.prepareCommand({
|
||||
command: 'ls',
|
||||
args: [],
|
||||
cwd: workspace,
|
||||
env: {},
|
||||
policy: { additionalPermissions: { network: true } },
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
/Cannot override readonly\/network\/filesystem restrictions in Plan mode/,
|
||||
);
|
||||
});
|
||||
|
||||
it('should correctly pass through the cwd to the resulting command', async () => {
|
||||
const req: SandboxRequest = {
|
||||
command: 'ls',
|
||||
@@ -184,12 +221,7 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
expect(bwrapArgs).toContain('--unshare-user');
|
||||
expect(bwrapArgs).toContain('--unshare-ipc');
|
||||
expect(bwrapArgs).toContain('--unshare-pid');
|
||||
expect(bwrapArgs).toContain('--unshare-uts');
|
||||
expect(bwrapArgs).toContain('--unshare-cgroup');
|
||||
expect(bwrapArgs).not.toContain('--unshare-all');
|
||||
expect(bwrapArgs).toContain('--share-net');
|
||||
});
|
||||
|
||||
describe('governance files', () => {
|
||||
@@ -252,15 +284,32 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
// Verify the specific bindings were added correctly
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
expect(bwrapArgs).toContain('--bind-try');
|
||||
expect(bwrapArgs[bwrapArgs.indexOf('/tmp/cache') - 1]).toBe(
|
||||
'--bind-try',
|
||||
'/tmp/cache',
|
||||
'/tmp/cache',
|
||||
);
|
||||
expect(bwrapArgs[bwrapArgs.indexOf('/opt/tools') - 1]).toBe(
|
||||
'--bind-try',
|
||||
'/opt/tools',
|
||||
'/opt/tools',
|
||||
]);
|
||||
);
|
||||
});
|
||||
|
||||
it('should not grant read-write access to allowedPaths inside the workspace when readonly mode is active', async () => {
|
||||
const manager = new LinuxSandboxManager({
|
||||
workspace,
|
||||
modeConfig: { readonly: true },
|
||||
});
|
||||
const result = await manager.prepareCommand({
|
||||
command: 'ls',
|
||||
args: [],
|
||||
cwd: workspace,
|
||||
env: {},
|
||||
policy: {
|
||||
allowedPaths: [workspace + '/subdirectory'],
|
||||
},
|
||||
});
|
||||
const bwrapArgs = result.args;
|
||||
const bindIndex = bwrapArgs.indexOf(workspace + '/subdirectory');
|
||||
expect(bwrapArgs[bindIndex - 1]).toBe('--ro-bind-try');
|
||||
});
|
||||
|
||||
it('should not bind the workspace twice even if it has a trailing slash in allowedPaths', async () => {
|
||||
@@ -274,23 +323,20 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
// Should only contain the primary workspace bind and governance files, not the second workspace bind with a trailing slash
|
||||
expectDynamicBinds(bwrapArgs, []);
|
||||
const binds = bwrapArgs.filter((a) => a === workspace);
|
||||
expect(binds.length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('forbiddenPaths', () => {
|
||||
it('should parameterize forbidden paths and explicitly deny them', async () => {
|
||||
vi.spyOn(fs.promises, 'stat').mockImplementation(async (p) => {
|
||||
// Mock /tmp/cache as a directory, and /opt/secret.txt as a file
|
||||
vi.mocked(fs.statSync).mockImplementation((p) => {
|
||||
if (p.toString().includes('cache')) {
|
||||
return { isDirectory: () => true } as fs.Stats;
|
||||
}
|
||||
return { isDirectory: () => false } as fs.Stats;
|
||||
});
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(async (p) =>
|
||||
p.toString(),
|
||||
);
|
||||
vi.mocked(fs.realpathSync).mockImplementation((p) => p.toString());
|
||||
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'ls',
|
||||
@@ -302,27 +348,22 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
'--tmpfs',
|
||||
'/tmp/cache',
|
||||
'--remount-ro',
|
||||
'/tmp/cache',
|
||||
'--ro-bind-try',
|
||||
'/dev/null',
|
||||
'/opt/secret.txt',
|
||||
]);
|
||||
const cacheIndex = bwrapArgs.indexOf('/tmp/cache');
|
||||
expect(bwrapArgs[cacheIndex - 1]).toBe('--tmpfs');
|
||||
|
||||
const secretIndex = bwrapArgs.indexOf('/opt/secret.txt');
|
||||
expect(bwrapArgs[secretIndex - 2]).toBe('--ro-bind');
|
||||
expect(bwrapArgs[secretIndex - 1]).toBe('/dev/null');
|
||||
});
|
||||
|
||||
it('resolves forbidden symlink paths to their real paths', async () => {
|
||||
vi.spyOn(fs.promises, 'stat').mockImplementation(
|
||||
async () => ({ isDirectory: () => false }) as fs.Stats,
|
||||
);
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => {
|
||||
if (p === '/tmp/forbidden-symlink') return '/opt/real-target.txt';
|
||||
return p.toString();
|
||||
},
|
||||
vi.mocked(fs.statSync).mockImplementation(
|
||||
() => ({ isDirectory: () => false }) as fs.Stats,
|
||||
);
|
||||
vi.mocked(fs.realpathSync).mockImplementation((p) => {
|
||||
if (p === '/tmp/forbidden-symlink') return '/opt/real-target.txt';
|
||||
return p.toString();
|
||||
});
|
||||
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'ls',
|
||||
@@ -334,24 +375,18 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
// Should explicitly mask both the resolved path and the original symlink path
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
'--ro-bind-try',
|
||||
'/dev/null',
|
||||
'/opt/real-target.txt',
|
||||
'--ro-bind-try',
|
||||
'/dev/null',
|
||||
'/tmp/forbidden-symlink',
|
||||
]);
|
||||
const secretIndex = bwrapArgs.indexOf('/opt/real-target.txt');
|
||||
expect(bwrapArgs[secretIndex - 2]).toBe('--ro-bind');
|
||||
expect(bwrapArgs[secretIndex - 1]).toBe('/dev/null');
|
||||
});
|
||||
|
||||
it('explicitly denies non-existent forbidden paths to prevent creation', async () => {
|
||||
const error = new Error('File not found') as NodeJS.ErrnoException;
|
||||
error.code = 'ENOENT';
|
||||
vi.spyOn(fs.promises, 'stat').mockRejectedValue(error);
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(async (p) =>
|
||||
p.toString(),
|
||||
);
|
||||
vi.mocked(fs.statSync).mockImplementation(() => {
|
||||
throw error;
|
||||
});
|
||||
vi.mocked(fs.realpathSync).mockImplementation((p) => p.toString());
|
||||
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'ls',
|
||||
@@ -363,23 +398,19 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
'--symlink',
|
||||
'/.forbidden',
|
||||
'/tmp/not-here.txt',
|
||||
]);
|
||||
const idx = bwrapArgs.indexOf('/tmp/not-here.txt');
|
||||
expect(bwrapArgs[idx - 2]).toBe('--symlink');
|
||||
expect(bwrapArgs[idx - 1]).toBe('/dev/null');
|
||||
});
|
||||
|
||||
it('masks directory symlinks with tmpfs for both paths', async () => {
|
||||
vi.spyOn(fs.promises, 'stat').mockImplementation(
|
||||
async () => ({ isDirectory: () => true }) as fs.Stats,
|
||||
);
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => {
|
||||
if (p === '/tmp/dir-link') return '/opt/real-dir';
|
||||
return p.toString();
|
||||
},
|
||||
vi.mocked(fs.statSync).mockImplementation(
|
||||
() => ({ isDirectory: () => true }) as fs.Stats,
|
||||
);
|
||||
vi.mocked(fs.realpathSync).mockImplementation((p) => {
|
||||
if (p === '/tmp/dir-link') return '/opt/real-dir';
|
||||
return p.toString();
|
||||
});
|
||||
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'ls',
|
||||
@@ -391,25 +422,15 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
'--tmpfs',
|
||||
'/opt/real-dir',
|
||||
'--remount-ro',
|
||||
'/opt/real-dir',
|
||||
'--tmpfs',
|
||||
'/tmp/dir-link',
|
||||
'--remount-ro',
|
||||
'/tmp/dir-link',
|
||||
]);
|
||||
const idx = bwrapArgs.indexOf('/opt/real-dir');
|
||||
expect(bwrapArgs[idx - 1]).toBe('--tmpfs');
|
||||
});
|
||||
|
||||
it('should override allowed paths if a path is also in forbidden paths', async () => {
|
||||
vi.spyOn(fs.promises, 'stat').mockImplementation(
|
||||
async () => ({ isDirectory: () => true }) as fs.Stats,
|
||||
);
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(async (p) =>
|
||||
p.toString(),
|
||||
vi.mocked(fs.statSync).mockImplementation(
|
||||
() => ({ isDirectory: () => true }) as fs.Stats,
|
||||
);
|
||||
vi.mocked(fs.realpathSync).mockImplementation((p) => p.toString());
|
||||
|
||||
const bwrapArgs = await getBwrapArgs({
|
||||
command: 'ls',
|
||||
@@ -422,15 +443,12 @@ describe('LinuxSandboxManager', () => {
|
||||
},
|
||||
});
|
||||
|
||||
expectDynamicBinds(bwrapArgs, [
|
||||
'--bind-try',
|
||||
'/tmp/conflict',
|
||||
'/tmp/conflict',
|
||||
'--tmpfs',
|
||||
'/tmp/conflict',
|
||||
'--remount-ro',
|
||||
'/tmp/conflict',
|
||||
]);
|
||||
const bindTryIdx = bwrapArgs.indexOf('--bind-try');
|
||||
const tmpfsIdx = bwrapArgs.lastIndexOf('--tmpfs');
|
||||
|
||||
expect(bwrapArgs[bindTryIdx + 1]).toBe('/tmp/conflict');
|
||||
expect(bwrapArgs[tmpfsIdx + 1]).toBe('/tmp/conflict');
|
||||
expect(tmpfsIdx).toBeGreaterThan(bindTryIdx);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import { debugLogger } from '../../utils/debugLogger.js';
|
||||
import { join, dirname, normalize } from 'node:path';
|
||||
import os from 'node:os';
|
||||
import {
|
||||
@@ -12,15 +13,25 @@ import {
|
||||
type GlobalSandboxOptions,
|
||||
type SandboxRequest,
|
||||
type SandboxedCommand,
|
||||
type SandboxPermissions,
|
||||
GOVERNANCE_FILES,
|
||||
sanitizePaths,
|
||||
tryRealpath,
|
||||
} from '../../services/sandboxManager.js';
|
||||
import {
|
||||
sanitizeEnvironment,
|
||||
getSecureSanitizationConfig,
|
||||
} from '../../services/environmentSanitization.js';
|
||||
import { isNodeError } from '../../utils/errors.js';
|
||||
import { type SandboxPolicyManager } from '../../policy/sandboxPolicyManager.js';
|
||||
import {
|
||||
isStrictlyApproved,
|
||||
verifySandboxOverrides,
|
||||
getCommandName,
|
||||
} from '../utils/commandUtils.js';
|
||||
import {
|
||||
tryRealpath,
|
||||
resolveGitWorktreePaths,
|
||||
isErrnoException,
|
||||
} from '../utils/fsUtils.js';
|
||||
|
||||
let cachedBpfPath: string | undefined;
|
||||
|
||||
@@ -102,13 +113,24 @@ function touch(filePath: string, isDirectory: boolean) {
|
||||
import {
|
||||
isKnownSafeCommand,
|
||||
isDangerousCommand,
|
||||
} from '../macos/commandSafety.js';
|
||||
} from '../utils/commandSafety.js';
|
||||
|
||||
/**
|
||||
* A SandboxManager implementation for Linux that uses Bubblewrap (bwrap).
|
||||
*/
|
||||
|
||||
export interface LinuxSandboxOptions extends GlobalSandboxOptions {
|
||||
modeConfig?: {
|
||||
readonly?: boolean;
|
||||
network?: boolean;
|
||||
approvedTools?: string[];
|
||||
allowOverrides?: boolean;
|
||||
};
|
||||
policyManager?: SandboxPolicyManager;
|
||||
}
|
||||
|
||||
export class LinuxSandboxManager implements SandboxManager {
|
||||
constructor(private readonly options: GlobalSandboxOptions) {}
|
||||
constructor(private readonly options: LinuxSandboxOptions) {}
|
||||
|
||||
isKnownSafeCommand(args: string[]): boolean {
|
||||
return isKnownSafeCommand(args);
|
||||
@@ -119,6 +141,41 @@ export class LinuxSandboxManager implements SandboxManager {
|
||||
}
|
||||
|
||||
async prepareCommand(req: SandboxRequest): Promise<SandboxedCommand> {
|
||||
const isReadonlyMode = this.options.modeConfig?.readonly ?? true;
|
||||
const allowOverrides = this.options.modeConfig?.allowOverrides ?? true;
|
||||
|
||||
verifySandboxOverrides(allowOverrides, req.policy);
|
||||
|
||||
const commandName = await getCommandName(req);
|
||||
const isApproved = allowOverrides
|
||||
? await isStrictlyApproved(req, this.options.modeConfig?.approvedTools)
|
||||
: false;
|
||||
const workspaceWrite = !isReadonlyMode || isApproved;
|
||||
const networkAccess =
|
||||
this.options.modeConfig?.network ?? req.policy?.networkAccess ?? false;
|
||||
|
||||
const persistentPermissions = allowOverrides
|
||||
? this.options.policyManager?.getCommandPermissions(commandName)
|
||||
: undefined;
|
||||
|
||||
const mergedAdditional: SandboxPermissions = {
|
||||
fileSystem: {
|
||||
read: [
|
||||
...(persistentPermissions?.fileSystem?.read ?? []),
|
||||
...(req.policy?.additionalPermissions?.fileSystem?.read ?? []),
|
||||
],
|
||||
write: [
|
||||
...(persistentPermissions?.fileSystem?.write ?? []),
|
||||
...(req.policy?.additionalPermissions?.fileSystem?.write ?? []),
|
||||
],
|
||||
},
|
||||
network:
|
||||
networkAccess ||
|
||||
persistentPermissions?.network ||
|
||||
req.policy?.additionalPermissions?.network ||
|
||||
false,
|
||||
};
|
||||
|
||||
const sanitizationConfig = getSecureSanitizationConfig(
|
||||
req.policy?.sanitizationConfig,
|
||||
);
|
||||
@@ -126,13 +183,142 @@ export class LinuxSandboxManager implements SandboxManager {
|
||||
const sanitizedEnv = sanitizeEnvironment(req.env, sanitizationConfig);
|
||||
|
||||
const bwrapArgs: string[] = [
|
||||
...this.getNetworkArgs(req),
|
||||
...this.getBaseArgs(),
|
||||
...this.getGovernanceArgs(),
|
||||
...this.getAllowedPathsArgs(req.policy?.allowedPaths),
|
||||
...(await this.getForbiddenPathsArgs(req.policy?.forbiddenPaths)),
|
||||
'--unshare-all',
|
||||
'--new-session', // Isolate session
|
||||
'--die-with-parent', // Prevent orphaned runaway processes
|
||||
];
|
||||
|
||||
if (mergedAdditional.network) {
|
||||
bwrapArgs.push('--share-net');
|
||||
}
|
||||
|
||||
bwrapArgs.push(
|
||||
'--ro-bind',
|
||||
'/',
|
||||
'/',
|
||||
'--dev', // Creates a safe, minimal /dev (replaces --dev-bind)
|
||||
'/dev',
|
||||
'--proc', // Creates a fresh procfs for the unshared PID namespace
|
||||
'/proc',
|
||||
'--tmpfs', // Provides an isolated, writable /tmp directory
|
||||
'/tmp',
|
||||
);
|
||||
|
||||
const workspacePath = tryRealpath(this.options.workspace);
|
||||
|
||||
const bindFlag = workspaceWrite ? '--bind-try' : '--ro-bind-try';
|
||||
|
||||
if (workspaceWrite) {
|
||||
bwrapArgs.push(
|
||||
'--bind-try',
|
||||
this.options.workspace,
|
||||
this.options.workspace,
|
||||
);
|
||||
if (workspacePath !== this.options.workspace) {
|
||||
bwrapArgs.push('--bind-try', workspacePath, workspacePath);
|
||||
}
|
||||
} else {
|
||||
bwrapArgs.push(
|
||||
'--ro-bind-try',
|
||||
this.options.workspace,
|
||||
this.options.workspace,
|
||||
);
|
||||
if (workspacePath !== this.options.workspace) {
|
||||
bwrapArgs.push('--ro-bind-try', workspacePath, workspacePath);
|
||||
}
|
||||
}
|
||||
|
||||
const { worktreeGitDir, mainGitDir } =
|
||||
resolveGitWorktreePaths(workspacePath);
|
||||
if (worktreeGitDir) {
|
||||
bwrapArgs.push(bindFlag, worktreeGitDir, worktreeGitDir);
|
||||
}
|
||||
if (mainGitDir) {
|
||||
bwrapArgs.push(bindFlag, mainGitDir, mainGitDir);
|
||||
}
|
||||
|
||||
const allowedPaths = sanitizePaths(req.policy?.allowedPaths) || [];
|
||||
const normalizedWorkspace = normalize(workspacePath).replace(/\/$/, '');
|
||||
for (const allowedPath of allowedPaths) {
|
||||
const resolved = tryRealpath(allowedPath);
|
||||
if (!fs.existsSync(resolved)) continue;
|
||||
const normalizedAllowedPath = normalize(resolved).replace(/\/$/, '');
|
||||
if (normalizedAllowedPath !== normalizedWorkspace) {
|
||||
if (
|
||||
!workspaceWrite &&
|
||||
normalizedAllowedPath.startsWith(normalizedWorkspace + '/')
|
||||
) {
|
||||
bwrapArgs.push('--ro-bind-try', resolved, resolved);
|
||||
} else {
|
||||
bwrapArgs.push('--bind-try', resolved, resolved);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const additionalReads =
|
||||
sanitizePaths(mergedAdditional.fileSystem?.read) || [];
|
||||
for (const p of additionalReads) {
|
||||
try {
|
||||
const safeResolvedPath = tryRealpath(p);
|
||||
bwrapArgs.push('--ro-bind-try', safeResolvedPath, safeResolvedPath);
|
||||
} catch (e: unknown) {
|
||||
debugLogger.warn(e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
}
|
||||
|
||||
const additionalWrites =
|
||||
sanitizePaths(mergedAdditional.fileSystem?.write) || [];
|
||||
for (const p of additionalWrites) {
|
||||
try {
|
||||
const safeResolvedPath = tryRealpath(p);
|
||||
bwrapArgs.push('--bind-try', safeResolvedPath, safeResolvedPath);
|
||||
} catch (e: unknown) {
|
||||
debugLogger.warn(e instanceof Error ? e.message : String(e));
|
||||
}
|
||||
}
|
||||
|
||||
for (const file of GOVERNANCE_FILES) {
|
||||
const filePath = join(this.options.workspace, file.path);
|
||||
touch(filePath, file.isDirectory);
|
||||
const realPath = tryRealpath(filePath);
|
||||
bwrapArgs.push('--ro-bind', filePath, filePath);
|
||||
if (realPath !== filePath) {
|
||||
bwrapArgs.push('--ro-bind', realPath, realPath);
|
||||
}
|
||||
}
|
||||
|
||||
const forbiddenPaths = sanitizePaths(req.policy?.forbiddenPaths) || [];
|
||||
for (const p of forbiddenPaths) {
|
||||
let resolved: string;
|
||||
try {
|
||||
resolved = tryRealpath(p); // Forbidden paths should still resolve to block the real path
|
||||
if (!fs.existsSync(resolved)) continue;
|
||||
} catch (e: unknown) {
|
||||
debugLogger.warn(
|
||||
`Failed to resolve forbidden path ${p}: ${e instanceof Error ? e.message : String(e)}`,
|
||||
);
|
||||
bwrapArgs.push('--ro-bind', '/dev/null', p);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const stat = fs.statSync(resolved);
|
||||
if (stat.isDirectory()) {
|
||||
bwrapArgs.push('--tmpfs', resolved, '--remount-ro', resolved);
|
||||
} else {
|
||||
bwrapArgs.push('--ro-bind', '/dev/null', resolved);
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
if (isErrnoException(e) && e.code === 'ENOENT') {
|
||||
bwrapArgs.push('--symlink', '/dev/null', resolved);
|
||||
} else {
|
||||
debugLogger.warn(
|
||||
`Failed to stat forbidden path ${resolved}: ${e instanceof Error ? e.message : String(e)}`,
|
||||
);
|
||||
bwrapArgs.push('--ro-bind', '/dev/null', resolved);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const bpfPath = getSeccompBpfPath();
|
||||
|
||||
bwrapArgs.push('--seccomp', '9');
|
||||
@@ -153,142 +339,4 @@ export class LinuxSandboxManager implements SandboxManager {
|
||||
cwd: req.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates arguments for network isolation.
|
||||
*/
|
||||
private getNetworkArgs(req: SandboxRequest): string[] {
|
||||
return req.policy?.networkAccess
|
||||
? [
|
||||
'--unshare-user',
|
||||
'--unshare-ipc',
|
||||
'--unshare-pid',
|
||||
'--unshare-uts',
|
||||
'--unshare-cgroup',
|
||||
]
|
||||
: ['--unshare-all'];
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates the base bubblewrap arguments for isolation.
|
||||
*/
|
||||
private getBaseArgs(): string[] {
|
||||
return [
|
||||
'--new-session', // Isolate session
|
||||
'--die-with-parent', // Prevent orphaned runaway processes
|
||||
'--ro-bind',
|
||||
'/',
|
||||
'/',
|
||||
'--dev', // Creates a safe, minimal /dev (replaces --dev-bind)
|
||||
'/dev',
|
||||
'--proc', // Creates a fresh procfs for the unshared PID namespace
|
||||
'/proc',
|
||||
'--tmpfs', // Provides an isolated, writable /tmp directory
|
||||
'/tmp',
|
||||
// Note: --dev /dev sets up /dev/pts automatically
|
||||
'--bind',
|
||||
this.options.workspace,
|
||||
this.options.workspace,
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates arguments for protected governance files.
|
||||
*/
|
||||
private getGovernanceArgs(): string[] {
|
||||
const args: string[] = [];
|
||||
// Protected governance files are bind-mounted as read-only, even if the workspace is RW.
|
||||
// We ensure they exist on the host and resolve real paths to prevent symlink bypasses.
|
||||
// In bwrap, later binds override earlier ones for the same path.
|
||||
for (const file of GOVERNANCE_FILES) {
|
||||
const filePath = join(this.options.workspace, file.path);
|
||||
touch(filePath, file.isDirectory);
|
||||
|
||||
const realPath = fs.realpathSync(filePath);
|
||||
|
||||
args.push('--ro-bind', filePath, filePath);
|
||||
if (realPath !== filePath) {
|
||||
args.push('--ro-bind', realPath, realPath);
|
||||
}
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates arguments for allowed paths.
|
||||
*/
|
||||
private getAllowedPathsArgs(allowedPaths?: string[]): string[] {
|
||||
const args: string[] = [];
|
||||
const paths = sanitizePaths(allowedPaths) || [];
|
||||
const normalizedWorkspace = this.normalizePath(this.options.workspace);
|
||||
|
||||
for (const p of paths) {
|
||||
if (this.normalizePath(p) !== normalizedWorkspace) {
|
||||
args.push('--bind-try', p, p);
|
||||
}
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates arguments for forbidden paths.
|
||||
*/
|
||||
private async getForbiddenPathsArgs(
|
||||
forbiddenPaths?: string[],
|
||||
): Promise<string[]> {
|
||||
const args: string[] = [];
|
||||
const paths = sanitizePaths(forbiddenPaths) || [];
|
||||
|
||||
for (const p of paths) {
|
||||
try {
|
||||
const originalPath = this.normalizePath(p);
|
||||
const resolvedPath = await tryRealpath(originalPath);
|
||||
|
||||
// Mask the resolved path to prevent access to the underlying file.
|
||||
const resolvedMask = await this.getMaskArgs(resolvedPath);
|
||||
args.push(...resolvedMask);
|
||||
|
||||
// If the original path was a symlink, mask it as well to prevent access
|
||||
// through the link itself.
|
||||
if (resolvedPath !== originalPath) {
|
||||
const originalMask = await this.getMaskArgs(originalPath);
|
||||
args.push(...originalMask);
|
||||
}
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`Failed to deny access to forbidden path: ${p}. ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates bubblewrap arguments to mask a forbidden path.
|
||||
*/
|
||||
private async getMaskArgs(path: string): Promise<string[]> {
|
||||
try {
|
||||
const stats = await fs.promises.stat(path);
|
||||
|
||||
if (stats.isDirectory()) {
|
||||
// Directories are masked by mounting an empty, read-only tmpfs.
|
||||
return ['--tmpfs', path, '--remount-ro', path];
|
||||
}
|
||||
// Existing files are masked by binding them to /dev/null.
|
||||
return ['--ro-bind-try', '/dev/null', path];
|
||||
} catch (e) {
|
||||
if (isNodeError(e) && e.code === 'ENOENT') {
|
||||
// Non-existent paths are masked by a broken symlink. This prevents
|
||||
// creation within the sandbox while avoiding host remnants.
|
||||
return ['--symlink', '/.forbidden', path];
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
private normalizePath(p: string): string {
|
||||
return normalize(p).replace(/\/$/, '');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -38,7 +38,7 @@ describe('MacOsSandboxManager', () => {
|
||||
manager = new MacOsSandboxManager({ workspace: mockWorkspace });
|
||||
|
||||
// Mock the seatbelt args builder to isolate manager tests
|
||||
vi.spyOn(seatbeltArgsBuilder, 'buildSeatbeltArgs').mockResolvedValue([
|
||||
vi.spyOn(seatbeltArgsBuilder, 'buildSeatbeltArgs').mockReturnValue([
|
||||
'-p',
|
||||
'(mock profile)',
|
||||
'-D',
|
||||
|
||||
@@ -24,8 +24,9 @@ import {
|
||||
isKnownSafeCommand,
|
||||
isDangerousCommand,
|
||||
isStrictlyApproved,
|
||||
} from './commandSafety.js';
|
||||
} from '../utils/commandSafety.js';
|
||||
import { type SandboxPolicyManager } from '../../policy/sandboxPolicyManager.js';
|
||||
import { verifySandboxOverrides } from '../utils/commandUtils.js';
|
||||
|
||||
export interface MacOsSandboxOptions extends GlobalSandboxOptions {
|
||||
/** The current sandbox mode behavior from config. */
|
||||
@@ -70,17 +71,7 @@ export class MacOsSandboxManager implements SandboxManager {
|
||||
const allowOverrides = this.options.modeConfig?.allowOverrides ?? true;
|
||||
|
||||
// Reject override attempts in plan mode
|
||||
if (!allowOverrides && req.policy?.additionalPermissions) {
|
||||
const perms = req.policy.additionalPermissions;
|
||||
if (
|
||||
perms.network ||
|
||||
(perms.fileSystem?.write && perms.fileSystem.write.length > 0)
|
||||
) {
|
||||
throw new Error(
|
||||
'Sandbox request rejected: Cannot override readonly/network restrictions in Plan mode.',
|
||||
);
|
||||
}
|
||||
}
|
||||
verifySandboxOverrides(allowOverrides, req.policy);
|
||||
|
||||
// If not in readonly mode OR it's a strictly approved pipeline, allow workspace writes
|
||||
const isApproved = allowOverrides
|
||||
@@ -120,7 +111,7 @@ export class MacOsSandboxManager implements SandboxManager {
|
||||
false,
|
||||
};
|
||||
|
||||
const sandboxArgs = await buildSeatbeltArgs({
|
||||
const sandboxArgs = buildSeatbeltArgs({
|
||||
workspace: this.options.workspace,
|
||||
allowedPaths: [...(req.policy?.allowedPaths || [])],
|
||||
forbiddenPaths: req.policy?.forbiddenPaths,
|
||||
|
||||
@@ -3,25 +3,31 @@
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { describe, it, expect, vi, afterEach } from 'vitest';
|
||||
import { buildSeatbeltArgs } from './seatbeltArgsBuilder.js';
|
||||
import * as sandboxManager from '../../services/sandboxManager.js';
|
||||
import * as fsUtils from '../utils/fsUtils.js';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
|
||||
vi.mock('../utils/fsUtils.js', async () => {
|
||||
const actual = await vi.importActual('../utils/fsUtils.js');
|
||||
return {
|
||||
...actual,
|
||||
tryRealpath: vi.fn((p) => p),
|
||||
resolveGitWorktreePaths: vi.fn(() => ({})),
|
||||
};
|
||||
});
|
||||
|
||||
describe('seatbeltArgsBuilder', () => {
|
||||
beforeEach(() => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe('buildSeatbeltArgs', () => {
|
||||
it('should build a strict allowlist profile allowing the workspace via param', async () => {
|
||||
// Mock tryRealpath to just return the path for testing
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => p,
|
||||
);
|
||||
it('should build a strict allowlist profile allowing the workspace via param', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p);
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/Users/test/workspace',
|
||||
});
|
||||
|
||||
@@ -38,11 +44,9 @@ describe('seatbeltArgsBuilder', () => {
|
||||
expect(args).toContain(`TMPDIR=${os.tmpdir()}`);
|
||||
});
|
||||
|
||||
it('should allow network when networkAccess is true', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => p,
|
||||
);
|
||||
const args = await buildSeatbeltArgs({
|
||||
it('should allow network when networkAccess is true', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p);
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
networkAccess: true,
|
||||
});
|
||||
@@ -51,10 +55,8 @@ describe('seatbeltArgsBuilder', () => {
|
||||
});
|
||||
|
||||
describe('governance files', () => {
|
||||
it('should inject explicit deny rules for governance files', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(async (p) =>
|
||||
p.toString(),
|
||||
);
|
||||
it('should inject explicit deny rules for governance files', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p.toString());
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true);
|
||||
vi.spyOn(fs, 'lstatSync').mockImplementation(
|
||||
(p) =>
|
||||
@@ -64,35 +66,29 @@ describe('seatbeltArgsBuilder', () => {
|
||||
}) as unknown as fs.Stats,
|
||||
);
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
workspace: '/Users/test/workspace',
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test/workspace',
|
||||
});
|
||||
const profile = args[1];
|
||||
|
||||
// .gitignore should be a literal deny
|
||||
expect(args).toContain('-D');
|
||||
expect(args).toContain(
|
||||
'GOVERNANCE_FILE_0=/Users/test/workspace/.gitignore',
|
||||
);
|
||||
expect(args).toContain('GOVERNANCE_FILE_0=/test/workspace/.gitignore');
|
||||
expect(profile).toContain(
|
||||
'(deny file-write* (literal (param "GOVERNANCE_FILE_0")))',
|
||||
);
|
||||
|
||||
// .git should be a subpath deny
|
||||
expect(args).toContain('GOVERNANCE_FILE_2=/Users/test/workspace/.git');
|
||||
expect(args).toContain('GOVERNANCE_FILE_2=/test/workspace/.git');
|
||||
expect(profile).toContain(
|
||||
'(deny file-write* (subpath (param "GOVERNANCE_FILE_2")))',
|
||||
);
|
||||
});
|
||||
|
||||
it('should protect both the symlink and the real path if they differ', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => {
|
||||
if (p === '/test/workspace/.gitignore')
|
||||
return '/test/real/.gitignore';
|
||||
return p.toString();
|
||||
},
|
||||
);
|
||||
it('should protect both the symlink and the real path if they differ', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => {
|
||||
if (p === '/test/workspace/.gitignore')
|
||||
return '/test/real/.gitignore';
|
||||
return p.toString();
|
||||
});
|
||||
vi.spyOn(fs, 'existsSync').mockReturnValue(true);
|
||||
vi.spyOn(fs, 'lstatSync').mockImplementation(
|
||||
() =>
|
||||
@@ -102,7 +98,7 @@ describe('seatbeltArgsBuilder', () => {
|
||||
}) as unknown as fs.Stats,
|
||||
);
|
||||
|
||||
const args = await buildSeatbeltArgs({ workspace: '/test/workspace' });
|
||||
const args = buildSeatbeltArgs({ workspace: '/test/workspace' });
|
||||
const profile = args[1];
|
||||
|
||||
expect(args).toContain('GOVERNANCE_FILE_0=/test/workspace/.gitignore');
|
||||
@@ -117,15 +113,13 @@ describe('seatbeltArgsBuilder', () => {
|
||||
});
|
||||
|
||||
describe('allowedPaths', () => {
|
||||
it('should parameterize allowed paths and normalize them', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => {
|
||||
if (p === '/test/symlink') return '/test/real_path';
|
||||
return p;
|
||||
},
|
||||
);
|
||||
it('should parameterize allowed paths and normalize them', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => {
|
||||
if (p === '/test/symlink') return '/test/real_path';
|
||||
return p;
|
||||
});
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
allowedPaths: ['/custom/path1', '/test/symlink'],
|
||||
});
|
||||
@@ -141,12 +135,10 @@ describe('seatbeltArgsBuilder', () => {
|
||||
});
|
||||
|
||||
describe('forbiddenPaths', () => {
|
||||
it('should parameterize forbidden paths and explicitly deny them', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => p,
|
||||
);
|
||||
it('should parameterize forbidden paths and explicitly deny them', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p);
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
forbiddenPaths: ['/secret/path'],
|
||||
});
|
||||
@@ -161,22 +153,21 @@ describe('seatbeltArgsBuilder', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('resolves forbidden symlink paths to their real paths', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => {
|
||||
if (p === '/test/symlink') return '/test/real_path';
|
||||
return p;
|
||||
},
|
||||
);
|
||||
it('resolves forbidden symlink paths to their real paths', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => {
|
||||
if (p === '/test/symlink' || p === '/test/missing-dir') {
|
||||
return '/test/real_path';
|
||||
}
|
||||
return p;
|
||||
});
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
forbiddenPaths: ['/test/symlink'],
|
||||
});
|
||||
|
||||
const profile = args[1];
|
||||
|
||||
// The builder should resolve the symlink and explicitly deny the real target path
|
||||
expect(args).toContain('-D');
|
||||
expect(args).toContain('FORBIDDEN_PATH_0=/test/real_path');
|
||||
expect(profile).toContain(
|
||||
@@ -184,12 +175,10 @@ describe('seatbeltArgsBuilder', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('explicitly denies non-existent forbidden paths to prevent creation', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => p,
|
||||
);
|
||||
it('explicitly denies non-existent forbidden paths to prevent creation', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p);
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
forbiddenPaths: ['/test/missing-dir/missing-file.txt'],
|
||||
});
|
||||
@@ -205,12 +194,10 @@ describe('seatbeltArgsBuilder', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('should override allowed paths if a path is also in forbidden paths', async () => {
|
||||
vi.spyOn(sandboxManager, 'tryRealpath').mockImplementation(
|
||||
async (p) => p,
|
||||
);
|
||||
it('should override allowed paths if a path is also in forbidden paths', () => {
|
||||
vi.mocked(fsUtils.tryRealpath).mockImplementation((p) => p);
|
||||
|
||||
const args = await buildSeatbeltArgs({
|
||||
const args = buildSeatbeltArgs({
|
||||
workspace: '/test',
|
||||
allowedPaths: ['/custom/path1'],
|
||||
forbiddenPaths: ['/custom/path1'],
|
||||
@@ -226,8 +213,6 @@ describe('seatbeltArgsBuilder', () => {
|
||||
expect(profile).toContain(allowString);
|
||||
expect(profile).toContain(denyString);
|
||||
|
||||
// Verify ordering: The explicit deny must appear AFTER the explicit allow in the profile string
|
||||
// Seatbelt rules are evaluated in order where the latest rule matching a path wins
|
||||
const allowIndex = profile.indexOf(allowString);
|
||||
const denyIndex = profile.indexOf(denyString);
|
||||
expect(denyIndex).toBeGreaterThan(allowIndex);
|
||||
|
||||
@@ -15,8 +15,8 @@ import {
|
||||
type SandboxPermissions,
|
||||
sanitizePaths,
|
||||
GOVERNANCE_FILES,
|
||||
tryRealpath,
|
||||
} from '../../services/sandboxManager.js';
|
||||
import { tryRealpath, resolveGitWorktreePaths } from '../utils/fsUtils.js';
|
||||
|
||||
/**
|
||||
* Options for building macOS Seatbelt arguments.
|
||||
@@ -44,13 +44,11 @@ export interface SeatbeltArgsOptions {
|
||||
* Returns arguments up to the end of sandbox-exec configuration (e.g. ['-p', '<profile>', '-D', ...])
|
||||
* Does not include the final '--' separator or the command to run.
|
||||
*/
|
||||
export async function buildSeatbeltArgs(
|
||||
options: SeatbeltArgsOptions,
|
||||
): Promise<string[]> {
|
||||
export function buildSeatbeltArgs(options: SeatbeltArgsOptions): string[] {
|
||||
let profile = BASE_SEATBELT_PROFILE + '\n';
|
||||
const args: string[] = [];
|
||||
|
||||
const workspacePath = await tryRealpath(options.workspace);
|
||||
const workspacePath = tryRealpath(options.workspace);
|
||||
args.push('-D', `WORKSPACE=${workspacePath}`);
|
||||
args.push('-D', `WORKSPACE_RAW=${options.workspace}`);
|
||||
profile += `(allow file-read* (subpath (param "WORKSPACE_RAW")))\n`;
|
||||
@@ -67,7 +65,7 @@ export async function buildSeatbeltArgs(
|
||||
// (Seatbelt evaluates rules in order, later rules win for same path).
|
||||
for (let i = 0; i < GOVERNANCE_FILES.length; i++) {
|
||||
const governanceFile = path.join(workspacePath, GOVERNANCE_FILES[i].path);
|
||||
const realGovernanceFile = await tryRealpath(governanceFile);
|
||||
const realGovernanceFile = tryRealpath(governanceFile);
|
||||
|
||||
// Determine if it should be treated as a directory (subpath) or a file (literal).
|
||||
// .git is generally a directory, while ignore files are literals.
|
||||
@@ -92,42 +90,20 @@ export async function buildSeatbeltArgs(
|
||||
}
|
||||
|
||||
// Auto-detect and support git worktrees by granting read and write access to the underlying git directory
|
||||
try {
|
||||
const gitPath = path.join(workspacePath, '.git');
|
||||
const gitStat = fs.lstatSync(gitPath);
|
||||
if (gitStat.isFile()) {
|
||||
const gitContent = fs.readFileSync(gitPath, 'utf8');
|
||||
const match = gitContent.match(/^gitdir:\s*(.+)$/m);
|
||||
if (match && match[1]) {
|
||||
let worktreeGitDir = match[1].trim();
|
||||
if (!path.isAbsolute(worktreeGitDir)) {
|
||||
worktreeGitDir = path.resolve(workspacePath, worktreeGitDir);
|
||||
}
|
||||
const resolvedWorktreeGitDir = await tryRealpath(worktreeGitDir);
|
||||
|
||||
// Grant write access to the worktree's specific .git directory
|
||||
args.push('-D', `WORKTREE_GIT_DIR=${resolvedWorktreeGitDir}`);
|
||||
profile += `(allow file-read* file-write* (subpath (param "WORKTREE_GIT_DIR")))\n`;
|
||||
|
||||
// Grant write access to the main repository's .git directory (objects, refs, etc. are shared)
|
||||
// resolvedWorktreeGitDir is usually like: /path/to/main-repo/.git/worktrees/worktree-name
|
||||
const mainGitDir = await tryRealpath(
|
||||
path.dirname(path.dirname(resolvedWorktreeGitDir)),
|
||||
);
|
||||
if (mainGitDir && mainGitDir.endsWith('.git')) {
|
||||
args.push('-D', `MAIN_GIT_DIR=${mainGitDir}`);
|
||||
profile += `(allow file-read* file-write* (subpath (param "MAIN_GIT_DIR")))\n`;
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (_e) {
|
||||
// Ignore if .git doesn't exist, isn't readable, etc.
|
||||
const { worktreeGitDir, mainGitDir } = resolveGitWorktreePaths(workspacePath);
|
||||
if (worktreeGitDir) {
|
||||
args.push('-D', `WORKTREE_GIT_DIR=${worktreeGitDir}`);
|
||||
profile += `(allow file-read* file-write* (subpath (param "WORKTREE_GIT_DIR")))\n`;
|
||||
}
|
||||
if (mainGitDir) {
|
||||
args.push('-D', `MAIN_GIT_DIR=${mainGitDir}`);
|
||||
profile += `(allow file-read* file-write* (subpath (param "MAIN_GIT_DIR")))\n`;
|
||||
}
|
||||
|
||||
const tmpPath = await tryRealpath(os.tmpdir());
|
||||
const tmpPath = tryRealpath(os.tmpdir());
|
||||
args.push('-D', `TMPDIR=${tmpPath}`);
|
||||
|
||||
const nodeRootPath = await tryRealpath(
|
||||
const nodeRootPath = tryRealpath(
|
||||
path.dirname(path.dirname(process.execPath)),
|
||||
);
|
||||
args.push('-D', `NODE_ROOT=${nodeRootPath}`);
|
||||
@@ -142,7 +118,7 @@ export async function buildSeatbeltArgs(
|
||||
for (const p of paths) {
|
||||
if (!p.trim()) continue;
|
||||
try {
|
||||
let resolved = await tryRealpath(p);
|
||||
let resolved = tryRealpath(p);
|
||||
|
||||
// If this is a 'bin' directory (like /usr/local/bin or homebrew/bin),
|
||||
// also grant read access to its parent directory so that symlinked
|
||||
@@ -165,8 +141,10 @@ export async function buildSeatbeltArgs(
|
||||
|
||||
// Handle allowedPaths
|
||||
const allowedPaths = sanitizePaths(options.allowedPaths) || [];
|
||||
const resolvedAllowedPaths: string[] = [];
|
||||
for (let i = 0; i < allowedPaths.length; i++) {
|
||||
const allowedPath = await tryRealpath(allowedPaths[i]);
|
||||
const allowedPath = tryRealpath(allowedPaths[i]);
|
||||
resolvedAllowedPaths.push(allowedPath);
|
||||
args.push('-D', `ALLOWED_PATH_${i}=${allowedPath}`);
|
||||
profile += `(allow file-read* file-write* (subpath (param "ALLOWED_PATH_${i}")))\n`;
|
||||
}
|
||||
@@ -176,7 +154,7 @@ export async function buildSeatbeltArgs(
|
||||
const { read, write } = options.additionalPermissions.fileSystem;
|
||||
if (read) {
|
||||
for (let i = 0; i < read.length; i++) {
|
||||
const resolved = await tryRealpath(read[i]);
|
||||
const resolved = tryRealpath(read[i]);
|
||||
const paramName = `ADDITIONAL_READ_${i}`;
|
||||
args.push('-D', `${paramName}=${resolved}`);
|
||||
let isFile = false;
|
||||
@@ -194,7 +172,7 @@ export async function buildSeatbeltArgs(
|
||||
}
|
||||
if (write) {
|
||||
for (let i = 0; i < write.length; i++) {
|
||||
const resolved = await tryRealpath(write[i]);
|
||||
const resolved = tryRealpath(write[i]);
|
||||
const paramName = `ADDITIONAL_WRITE_${i}`;
|
||||
args.push('-D', `${paramName}=${resolved}`);
|
||||
let isFile = false;
|
||||
@@ -215,7 +193,7 @@ export async function buildSeatbeltArgs(
|
||||
// Handle forbiddenPaths
|
||||
const forbiddenPaths = sanitizePaths(options.forbiddenPaths) || [];
|
||||
for (let i = 0; i < forbiddenPaths.length; i++) {
|
||||
const forbiddenPath = await tryRealpath(forbiddenPaths[i]);
|
||||
const forbiddenPath = tryRealpath(forbiddenPaths[i]);
|
||||
args.push('-D', `FORBIDDEN_PATH_${i}=${forbiddenPath}`);
|
||||
profile += `(deny file-read* file-write* (subpath (param "FORBIDDEN_PATH_${i}")))\n`;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { type SandboxRequest } from '../../services/sandboxManager.js';
|
||||
import {
|
||||
getCommandRoots,
|
||||
initializeShellParsers,
|
||||
splitCommands,
|
||||
stripShellWrapper,
|
||||
} from '../../utils/shell-utils.js';
|
||||
import { isKnownSafeCommand } from './commandSafety.js';
|
||||
import { parse as shellParse } from 'shell-quote';
|
||||
import path from 'node:path';
|
||||
|
||||
export async function isStrictlyApproved(
|
||||
req: SandboxRequest,
|
||||
approvedTools?: string[],
|
||||
): Promise<boolean> {
|
||||
if (!approvedTools || approvedTools.length === 0) {
|
||||
return false;
|
||||
}
|
||||
|
||||
await initializeShellParsers();
|
||||
|
||||
const fullCmd = [req.command, ...req.args].join(' ');
|
||||
const stripped = stripShellWrapper(fullCmd);
|
||||
|
||||
const roots = getCommandRoots(stripped);
|
||||
if (roots.length === 0) return false;
|
||||
|
||||
const allRootsApproved = roots.every((root) => approvedTools.includes(root));
|
||||
if (allRootsApproved) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const pipelineCommands = splitCommands(stripped);
|
||||
if (pipelineCommands.length === 0) return false;
|
||||
|
||||
for (const cmdString of pipelineCommands) {
|
||||
const parsedArgs = shellParse(cmdString).map(String);
|
||||
if (!isKnownSafeCommand(parsedArgs)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function getCommandName(req: SandboxRequest): Promise<string> {
|
||||
await initializeShellParsers();
|
||||
const fullCmd = [req.command, ...req.args].join(' ');
|
||||
const stripped = stripShellWrapper(fullCmd);
|
||||
const roots = getCommandRoots(stripped).filter(
|
||||
(r) => r !== 'shopt' && r !== 'set',
|
||||
);
|
||||
if (roots.length > 0) {
|
||||
return roots[0];
|
||||
}
|
||||
return path.basename(req.command);
|
||||
}
|
||||
|
||||
export function verifySandboxOverrides(
|
||||
allowOverrides: boolean,
|
||||
policy: SandboxRequest['policy'],
|
||||
) {
|
||||
if (!allowOverrides) {
|
||||
if (
|
||||
policy?.networkAccess ||
|
||||
policy?.allowedPaths?.length ||
|
||||
policy?.additionalPermissions?.network ||
|
||||
policy?.additionalPermissions?.fileSystem?.read?.length ||
|
||||
policy?.additionalPermissions?.fileSystem?.write?.length
|
||||
) {
|
||||
throw new Error(
|
||||
'Sandbox request rejected: Cannot override readonly/network/filesystem restrictions in Plan mode.',
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Google LLC
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
export function isErrnoException(e: unknown): e is NodeJS.ErrnoException {
|
||||
return e instanceof Error && 'code' in e;
|
||||
}
|
||||
|
||||
export function tryRealpath(p: string): string {
|
||||
try {
|
||||
return fs.realpathSync(p);
|
||||
} catch (_e) {
|
||||
if (isErrnoException(_e) && _e.code === 'ENOENT') {
|
||||
const parentDir = path.dirname(p);
|
||||
if (parentDir === p) {
|
||||
return p;
|
||||
}
|
||||
return path.join(tryRealpath(parentDir), path.basename(p));
|
||||
}
|
||||
throw _e;
|
||||
}
|
||||
}
|
||||
|
||||
export function resolveGitWorktreePaths(workspacePath: string): {
|
||||
worktreeGitDir?: string;
|
||||
mainGitDir?: string;
|
||||
} {
|
||||
try {
|
||||
const gitPath = path.join(workspacePath, '.git');
|
||||
const gitStat = fs.lstatSync(gitPath);
|
||||
if (gitStat.isFile()) {
|
||||
const gitContent = fs.readFileSync(gitPath, 'utf8');
|
||||
const match = gitContent.match(/^gitdir:\s+(.+)$/m);
|
||||
if (match && match[1]) {
|
||||
let worktreeGitDir = match[1].trim();
|
||||
if (!path.isAbsolute(worktreeGitDir)) {
|
||||
worktreeGitDir = path.resolve(workspacePath, worktreeGitDir);
|
||||
}
|
||||
const resolvedWorktreeGitDir = tryRealpath(worktreeGitDir);
|
||||
|
||||
// Security check: Verify the bidirectional link to prevent sandbox escape
|
||||
let isValid = false;
|
||||
try {
|
||||
const backlinkPath = path.join(resolvedWorktreeGitDir, 'gitdir');
|
||||
const backlink = fs.readFileSync(backlinkPath, 'utf8').trim();
|
||||
// The backlink must resolve to the workspace's .git file
|
||||
if (tryRealpath(backlink) === tryRealpath(gitPath)) {
|
||||
isValid = true;
|
||||
}
|
||||
} catch (_e) {
|
||||
// Fallback for submodules: check core.worktree in config
|
||||
try {
|
||||
const configPath = path.join(resolvedWorktreeGitDir, 'config');
|
||||
const config = fs.readFileSync(configPath, 'utf8');
|
||||
const match = config.match(/^\s*worktree\s*=\s*(.+)$/m);
|
||||
if (match && match[1]) {
|
||||
const worktreePath = path.resolve(
|
||||
resolvedWorktreeGitDir,
|
||||
match[1].trim(),
|
||||
);
|
||||
if (tryRealpath(worktreePath) === tryRealpath(workspacePath)) {
|
||||
isValid = true;
|
||||
}
|
||||
}
|
||||
} catch (_e2) {
|
||||
// Ignore
|
||||
}
|
||||
}
|
||||
|
||||
if (!isValid) {
|
||||
return {}; // Reject: valid worktrees/submodules must have a readable backlink
|
||||
}
|
||||
|
||||
const mainGitDir = tryRealpath(
|
||||
path.dirname(path.dirname(resolvedWorktreeGitDir)),
|
||||
);
|
||||
return {
|
||||
worktreeGitDir: resolvedWorktreeGitDir,
|
||||
mainGitDir: mainGitDir.endsWith('.git') ? mainGitDir : undefined,
|
||||
};
|
||||
}
|
||||
}
|
||||
} catch (_e) {
|
||||
// Ignore if .git doesn't exist, isn't readable, etc.
|
||||
}
|
||||
return {};
|
||||
}
|
||||
@@ -111,7 +111,7 @@ describe('WindowsSandboxManager', () => {
|
||||
};
|
||||
|
||||
await expect(planManager.prepareCommand(req)).rejects.toThrow(
|
||||
'Sandbox request rejected: Cannot override readonly/network restrictions in Plan mode.',
|
||||
'Sandbox request rejected: Cannot override readonly/network/filesystem restrictions in Plan mode.',
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
isStrictlyApproved,
|
||||
} from './commandSafety.js';
|
||||
import { type SandboxPolicyManager } from '../../policy/sandboxPolicyManager.js';
|
||||
import { verifySandboxOverrides } from '../utils/commandUtils.js';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
@@ -214,17 +215,7 @@ export class WindowsSandboxManager implements SandboxManager {
|
||||
const allowOverrides = this.options.modeConfig?.allowOverrides ?? true;
|
||||
|
||||
// Reject override attempts in plan mode
|
||||
if (!allowOverrides && req.policy?.additionalPermissions) {
|
||||
const perms = req.policy.additionalPermissions;
|
||||
if (
|
||||
perms.network ||
|
||||
(perms.fileSystem?.write && perms.fileSystem.write.length > 0)
|
||||
) {
|
||||
throw new Error(
|
||||
'Sandbox request rejected: Cannot override readonly/network restrictions in Plan mode.',
|
||||
);
|
||||
}
|
||||
}
|
||||
verifySandboxOverrides(allowOverrides, req.policy);
|
||||
|
||||
// Fetch persistent approvals for this command
|
||||
const commandName = await getCommandName(req.command, req.args);
|
||||
|
||||
@@ -10,7 +10,7 @@ import path from 'node:path';
|
||||
import {
|
||||
isKnownSafeCommand as isMacSafeCommand,
|
||||
isDangerousCommand as isMacDangerousCommand,
|
||||
} from '../sandbox/macos/commandSafety.js';
|
||||
} from '../sandbox/utils/commandSafety.js';
|
||||
import {
|
||||
isKnownSafeCommand as isWindowsSafeCommand,
|
||||
isDangerousCommand as isWindowsDangerousCommand,
|
||||
|
||||
@@ -42,7 +42,11 @@ export function createSandboxManager(
|
||||
policyManager,
|
||||
});
|
||||
} else if (os.platform() === 'linux') {
|
||||
return new LinuxSandboxManager({ workspace });
|
||||
return new LinuxSandboxManager({
|
||||
workspace,
|
||||
modeConfig,
|
||||
policyManager,
|
||||
});
|
||||
} else if (os.platform() === 'darwin') {
|
||||
return new MacOsSandboxManager({
|
||||
workspace,
|
||||
|
||||
Executable
+117
@@ -0,0 +1,117 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Gemini API Reliability Harvester
|
||||
# -------------------------------
|
||||
# This script gathers data about 500 API errors encountered during evaluation runs
|
||||
# (eval.yml) from GitHub Actions. It is used to analyze developer friction caused
|
||||
# by transient API failures.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/harvest_api_reliability.sh [SINCE] [LIMIT] [BRANCH]
|
||||
#
|
||||
# Examples:
|
||||
# ./scripts/harvest_api_reliability.sh # Last 7 days, all branches
|
||||
# ./scripts/harvest_api_reliability.sh 14d 500 # Last 14 days, limit 500
|
||||
# ./scripts/harvest_api_reliability.sh 2026-03-01 100 my-branch # Specific date and branch
|
||||
#
|
||||
# Prerequisites:
|
||||
# - GitHub CLI (gh) installed and authenticated (`gh auth login`)
|
||||
# - jq installed
|
||||
|
||||
# Arguments & Defaults
|
||||
if [[ -n "$1" && $1 =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then
|
||||
SINCE="$1"
|
||||
elif [[ -n "$1" && $1 =~ ^([0-9]+)d$ ]]; then
|
||||
DAYS="${BASH_REMATCH[1]}"
|
||||
if [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
SINCE=$(date -u -v-"${DAYS}"d +%Y-%m-%d)
|
||||
else
|
||||
SINCE=$(date -u -d "${DAYS} days ago" +%Y-%m-%d)
|
||||
fi
|
||||
else
|
||||
# Default to 7 days ago in YYYY-MM-DD format (UTC)
|
||||
if [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
SINCE=$(date -u -v-7d +%Y-%m-%d)
|
||||
else
|
||||
SINCE=$(date -u -d "7 days ago" +%Y-%m-%d)
|
||||
fi
|
||||
fi
|
||||
|
||||
LIMIT=${2:-300}
|
||||
BRANCH=${3:-""}
|
||||
WORKFLOWS=("Testing: E2E (Chained)" "Evals: Nightly")
|
||||
DEST_DIR=$(mktemp -d -t gemini-reliability-XXXXXX)
|
||||
MERGED_FILE="api-reliability-summary.jsonl"
|
||||
|
||||
# Ensure cleanup on exit
|
||||
trap 'rm -rf "$DEST_DIR"' EXIT
|
||||
|
||||
if ! command -v gh &> /dev/null; then
|
||||
echo "❌ Error: GitHub CLI (gh) is not installed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v jq &> /dev/null; then
|
||||
echo "❌ Error: jq is not installed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Clean start
|
||||
rm -f "$MERGED_FILE"
|
||||
|
||||
# gh run list --created expects a date (YYYY-MM-DD) or a range
|
||||
CREATED_QUERY=">=$SINCE"
|
||||
|
||||
for WORKFLOW in "${WORKFLOWS[@]}"; do
|
||||
echo "🔍 Fetching runs for '$WORKFLOW' created since $SINCE (max $LIMIT runs, branch: ${BRANCH:-all})..."
|
||||
|
||||
# Construct arguments for gh run list
|
||||
GH_ARGS=("--workflow" "$WORKFLOW" "--created" "$CREATED_QUERY" "--limit" "$LIMIT" "--json" "databaseId" "--jq" ".[].databaseId")
|
||||
if [ -n "$BRANCH" ]; then
|
||||
GH_ARGS+=("--branch" "$BRANCH")
|
||||
fi
|
||||
|
||||
RUN_IDS=$(gh run list "${GH_ARGS[@]}")
|
||||
exit_code=$?
|
||||
|
||||
if [ $exit_code -ne 0 ]; then
|
||||
echo "❌ Failed to fetch runs for '$WORKFLOW' (exit code: $exit_code). Please check 'gh auth status' and permissions." >&2
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ -z "$RUN_IDS" ]; then
|
||||
echo "📭 No runs found for workflow '$WORKFLOW' since $SINCE."
|
||||
continue
|
||||
fi
|
||||
|
||||
for ID in $RUN_IDS; do
|
||||
# Download artifacts named 'eval-logs-*'
|
||||
# Silencing output because many older runs won't have artifacts
|
||||
gh run download "$ID" -p "eval-logs-*" -D "$DEST_DIR/$ID" &>/dev/null || continue
|
||||
|
||||
# Append to master log
|
||||
# Use find to locate api-reliability.jsonl in any subdirectory of $DEST_DIR/$ID
|
||||
find "$DEST_DIR/$ID" -type f -name "api-reliability.jsonl" -exec cat {} + >> "$MERGED_FILE" 2>/dev/null
|
||||
done
|
||||
done
|
||||
|
||||
if [ ! -f "$MERGED_FILE" ]; then
|
||||
echo "📭 No reliability data found in the retrieved logs."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo -e "\n✅ Harvest Complete! Data merged into: $MERGED_FILE"
|
||||
echo "------------------------------------------------"
|
||||
echo "📊 Gemini API Reliability Summary (Since $SINCE)"
|
||||
echo "------------------------------------------------"
|
||||
|
||||
cat "$MERGED_FILE" | jq -s '
|
||||
group_by(.model) | map({
|
||||
model: .[0].model,
|
||||
"500s": (map(select(.errorCode == "500")) | length),
|
||||
"503s": (map(select(.errorCode == "503")) | length),
|
||||
retries: (map(select(.status == "RETRY")) | length),
|
||||
skips: (map(select(.status == "SKIP")) | length)
|
||||
})'
|
||||
|
||||
echo -e "\n💡 Total events captured: $(wc -l < "$MERGED_FILE")"
|
||||
Reference in New Issue
Block a user