Add support for output-format stream-jsonflag for headless mode (#10883)

This commit is contained in:
anj-s
2025-10-15 13:55:37 -07:00
committed by GitHub
parent 7bed302f21
commit 47f693173a
12 changed files with 957 additions and 20 deletions
+7
View File
@@ -228,6 +228,13 @@ the `--output-format json` flag to get structured output:
gemini -p "Explain the architecture of this codebase" --output-format json
```
For real-time event streaming (useful for monitoring long-running operations),
use `--output-format stream-json` to get newline-delimited JSON events:
```bash
gemini -p "Run tests and deploy" --output-format stream-json
```
### Quick Examples
#### Start a new project
+73 -10
View File
@@ -15,6 +15,13 @@ scripting, automation, CI/CD pipelines, and building AI-powered tools.
- [JSON Output](#json-output)
- [Response Schema](#response-schema)
- [Example Usage](#example-usage)
- [Streaming JSON Output](#streaming-json-output)
- [When to Use Streaming JSON](#when-to-use-streaming-json)
- [Event Types](#event-types)
- [Basic Usage](#basic-usage)
- [Example Output](#example-output)
- [Processing Stream Events](#processing-stream-events)
- [Real-World Examples](#real-world-examples)
- [File Redirection](#file-redirection)
- [Configuration Options](#configuration-options)
- [Examples](#examples)
@@ -211,6 +218,62 @@ Response:
}
```
### Streaming JSON Output
Returns real-time events as newline-delimited JSON (JSONL). Each significant
action (initialization, messages, tool calls, results) emits immediately as it
occurs. This format is ideal for monitoring long-running operations, building
UIs with live progress, and creating automation pipelines that react to events.
#### When to Use Streaming JSON
Use `--output-format stream-json` when you need:
- **Real-time progress monitoring** - See tool calls and responses as they
happen
- **Event-driven automation** - React to specific events (e.g., tool failures)
- **Live UI updates** - Build interfaces showing AI agent activity in real-time
- **Detailed execution logs** - Capture complete interaction history with
timestamps
- **Pipeline integration** - Stream events to logging/monitoring systems
#### Event Types
The streaming format emits 6 event types:
1. **`init`** - Session starts (includes session_id, model)
2. **`message`** - User prompts and assistant responses
3. **`tool_use`** - Tool call requests with parameters
4. **`tool_result`** - Tool execution results (success/error)
5. **`error`** - Non-fatal errors and warnings
6. **`result`** - Final session outcome with aggregated stats
#### Basic Usage
```bash
# Stream events to console
gemini --output-format stream-json --prompt "What is 2+2?"
# Save event stream to file
gemini --output-format stream-json --prompt "Analyze this code" > events.jsonl
# Parse with jq
gemini --output-format stream-json --prompt "List files" | jq -r '.type'
```
#### Example Output
Each line is a complete JSON event:
```jsonl
{"type":"init","timestamp":"2025-10-10T12:00:00.000Z","session_id":"abc123","model":"gemini-2.0-flash-exp"}
{"type":"message","role":"user","content":"List files in current directory","timestamp":"2025-10-10T12:00:01.000Z"}
{"type":"tool_use","tool_name":"Bash","tool_id":"bash-123","parameters":{"command":"ls -la"},"timestamp":"2025-10-10T12:00:02.000Z"}
{"type":"tool_result","tool_id":"bash-123","status":"success","output":"file1.txt\nfile2.txt","timestamp":"2025-10-10T12:00:03.000Z"}
{"type":"message","role":"assistant","content":"Here are the files...","delta":true,"timestamp":"2025-10-10T12:00:04.000Z"}
{"type":"result","status":"success","stats":{"total_tokens":250,"input_tokens":50,"output_tokens":200,"duration_ms":3000,"tool_calls":1},"timestamp":"2025-10-10T12:00:05.000Z"}
```
### File Redirection
Save output to files or pipe to other commands:
@@ -233,16 +296,16 @@ gemini -p "List programming languages" | grep -i "python"
Key command-line options for headless usage:
| Option | Description | Example |
| ----------------------- | ---------------------------------- | -------------------------------------------------- |
| `--prompt`, `-p` | Run in headless mode | `gemini -p "query"` |
| `--output-format` | Specify output format (text, json) | `gemini -p "query" --output-format json` |
| `--model`, `-m` | Specify the Gemini model | `gemini -p "query" -m gemini-2.5-flash` |
| `--debug`, `-d` | Enable debug mode | `gemini -p "query" --debug` |
| `--all-files`, `-a` | Include all files in context | `gemini -p "query" --all-files` |
| `--include-directories` | Include additional directories | `gemini -p "query" --include-directories src,docs` |
| `--yolo`, `-y` | Auto-approve all actions | `gemini -p "query" --yolo` |
| `--approval-mode` | Set approval mode | `gemini -p "query" --approval-mode auto_edit` |
| Option | Description | Example |
| ----------------------- | ----------------------------------------------- | -------------------------------------------------- |
| `--prompt`, `-p` | Run in headless mode | `gemini -p "query"` |
| `--output-format` | Specify output format (text, json, stream-json) | `gemini -p "query" --output-format stream-json` |
| `--model`, `-m` | Specify the Gemini model | `gemini -p "query" -m gemini-2.5-flash` |
| `--debug`, `-d` | Enable debug mode | `gemini -p "query" --debug` |
| `--all-files`, `-a` | Include all files in context | `gemini -p "query" --all-files` |
| `--include-directories` | Include additional directories | `gemini -p "query" --include-directories src,docs` |
| `--yolo`, `-y` | Auto-approve all actions | `gemini -p "query" --yolo` |
| `--approval-mode` | Set approval mode | `gemini -p "query" --approval-mode auto_edit` |
For complete details on all available configuration options, settings files, and
environment variables, see the
+1 -1
View File
@@ -227,7 +227,7 @@ for Gemini CLI:
- `file_filtering_respect_git_ignore` (boolean)
- `debug_mode` (boolean)
- `mcp_servers` (string)
- `output_format` (string: "text" or "json")
- `output_format` (string: "text", "json", or "stream-json")
- `gemini_cli.user_prompt`: This event occurs when a user submits a prompt.
- **Attributes**:
+3 -2
View File
@@ -128,7 +128,7 @@ their corresponding top-level category object in your `settings.json` file.
- **`output.format`** (string):
- **Description:** The format of the CLI output.
- **Default:** `"text"`
- **Values:** `"text"`, `"json"`
- **Values:** `"text"`, `"json"`, `"stream-json"`
#### `ui`
@@ -718,8 +718,9 @@ for that specific session.
- **Values:**
- `text`: (Default) The standard human-readable output.
- `json`: A machine-readable JSON output.
- `stream-json`: A streaming JSON output that emits real-time events.
- **Note:** For structured output and scripting, use the
`--output-format json` flag.
`--output-format json` or `--output-format stream-json` flag.
- **`--sandbox`** (**`-s`**):
- Enables sandbox mode for this session.
- **`--sandbox-image`**:
+16
View File
@@ -3432,6 +3432,22 @@ describe('Output format', () => {
expect(config.getOutputFormat()).toBe(OutputFormat.JSON);
});
it('should accept stream-json as a valid output format', async () => {
process.argv = ['node', 'script.js', '--output-format', 'stream-json'];
const argv = await parseArguments({} as Settings);
const config = await loadCliConfig(
{},
[],
new ExtensionEnablementManager(
ExtensionStorage.getUserExtensionsDir(),
argv.extensions,
),
'test-session',
argv,
);
expect(config.getOutputFormat()).toBe(OutputFormat.STREAM_JSON);
});
it('should error on invalid --output-format argument', async () => {
process.argv = ['node', 'script.js', '--output-format', 'yaml'];
const mockExit = vi.spyOn(process, 'exit').mockImplementation(() => {
+1 -1
View File
@@ -286,7 +286,7 @@ export async function parseArguments(settings: Settings): Promise<CliArgs> {
alias: 'o',
type: 'string',
description: 'The format of the CLI output.',
choices: ['text', 'json'],
choices: ['text', 'json', 'stream-json'],
})
.deprecateOption(
'show-memory-usage',
+94 -2
View File
@@ -20,6 +20,8 @@ import {
promptIdContext,
OutputFormat,
JsonFormatter,
StreamJsonFormatter,
JsonStreamEventType,
uiTelemetryService,
} from '@google/gemini-cli-core';
@@ -47,6 +49,12 @@ export async function runNonInteractive(
debugMode: config.getDebugMode(),
});
const startTime = Date.now();
const streamFormatter =
config.getOutputFormat() === OutputFormat.STREAM_JSON
? new StreamJsonFormatter()
: null;
try {
consolePatcher.patch();
// Handle EPIPE errors when the output is piped to a command that closes early.
@@ -59,6 +67,16 @@ export async function runNonInteractive(
const geminiClient = config.getGeminiClient();
// Emit init event for streaming JSON
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.INIT,
timestamp: new Date().toISOString(),
session_id: config.getSessionId(),
model: config.getModel(),
});
}
const abortController = new AbortController();
let query: Part[] | undefined;
@@ -98,6 +116,16 @@ export async function runNonInteractive(
query = processedQuery as Part[];
}
// Emit user message event for streaming JSON
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.MESSAGE,
timestamp: new Date().toISOString(),
role: 'user',
content: input,
});
}
let currentMessages: Content[] = [{ role: 'user', parts: query }];
let turnCount = 0;
@@ -124,13 +152,48 @@ export async function runNonInteractive(
}
if (event.type === GeminiEventType.Content) {
if (config.getOutputFormat() === OutputFormat.JSON) {
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.MESSAGE,
timestamp: new Date().toISOString(),
role: 'assistant',
content: event.value,
delta: true,
});
} else if (config.getOutputFormat() === OutputFormat.JSON) {
responseText += event.value;
} else {
process.stdout.write(event.value);
}
} else if (event.type === GeminiEventType.ToolCallRequest) {
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.TOOL_USE,
timestamp: new Date().toISOString(),
tool_name: event.value.name,
tool_id: event.value.callId,
parameters: event.value.args,
});
}
toolCallRequests.push(event.value);
} else if (event.type === GeminiEventType.LoopDetected) {
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.ERROR,
timestamp: new Date().toISOString(),
severity: 'warning',
message: 'Loop detected, stopping execution',
});
}
} else if (event.type === GeminiEventType.MaxSessionTurns) {
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.ERROR,
timestamp: new Date().toISOString(),
severity: 'error',
message: 'Maximum session turns exceeded',
});
}
}
}
@@ -148,6 +211,25 @@ export async function runNonInteractive(
completedToolCalls.push(completedToolCall);
if (streamFormatter) {
streamFormatter.emitEvent({
type: JsonStreamEventType.TOOL_RESULT,
timestamp: new Date().toISOString(),
tool_id: requestInfo.callId,
status: toolResponse.error ? 'error' : 'success',
output:
typeof toolResponse.resultDisplay === 'string'
? toolResponse.resultDisplay
: undefined,
error: toolResponse.error
? {
type: toolResponse.errorType || 'TOOL_EXECUTION_ERROR',
message: toolResponse.error.message,
}
: undefined,
});
}
if (toolResponse.error) {
handleToolError(
requestInfo.name,
@@ -180,7 +262,17 @@ export async function runNonInteractive(
currentMessages = [{ role: 'user', parts: toolResponseParts }];
} else {
if (config.getOutputFormat() === OutputFormat.JSON) {
// Emit final result event for streaming JSON
if (streamFormatter) {
const metrics = uiTelemetryService.getMetrics();
const durationMs = Date.now() - startTime;
streamFormatter.emitEvent({
type: JsonStreamEventType.RESULT,
timestamp: new Date().toISOString(),
status: 'success',
stats: streamFormatter.convertToStreamStats(metrics, durationMs),
});
} else if (config.getOutputFormat() === OutputFormat.JSON) {
const formatter = new JsonFormatter();
const stats = uiTelemetryService.getMetrics();
process.stdout.write(formatter.format(responseText, stats));
+66 -4
View File
@@ -8,6 +8,9 @@ import type { Config } from '@google/gemini-cli-core';
import {
OutputFormat,
JsonFormatter,
StreamJsonFormatter,
JsonStreamEventType,
uiTelemetryService,
parseAndFormatApiError,
FatalTurnLimitedError,
FatalCancellationError,
@@ -58,6 +61,7 @@ function getNumericExitCode(errorCode: string | number): number {
/**
* Handles errors consistently for both JSON and text output formats.
* In JSON mode, outputs formatted JSON error and exits.
* In streaming JSON mode, emits a result event with error status.
* In text mode, outputs error message and re-throws.
*/
export function handleError(
@@ -70,7 +74,24 @@ export function handleError(
config.getContentGeneratorConfig()?.authType,
);
if (config.getOutputFormat() === OutputFormat.JSON) {
if (config.getOutputFormat() === OutputFormat.STREAM_JSON) {
const streamFormatter = new StreamJsonFormatter();
const errorCode = customErrorCode ?? extractErrorCode(error);
const metrics = uiTelemetryService.getMetrics();
streamFormatter.emitEvent({
type: JsonStreamEventType.RESULT,
timestamp: new Date().toISOString(),
status: 'error',
error: {
type: error instanceof Error ? error.constructor.name : 'Error',
message: errorMessage,
},
stats: streamFormatter.convertToStreamStats(metrics, 0),
});
process.exit(getNumericExitCode(errorCode));
} else if (config.getOutputFormat() === OutputFormat.JSON) {
const formatter = new JsonFormatter();
const errorCode = customErrorCode ?? extractErrorCode(error);
@@ -110,7 +131,20 @@ export function handleToolError(
if (isFatal) {
const toolExecutionError = new FatalToolExecutionError(errorMessage);
if (config.getOutputFormat() === OutputFormat.JSON) {
if (config.getOutputFormat() === OutputFormat.STREAM_JSON) {
const streamFormatter = new StreamJsonFormatter();
const metrics = uiTelemetryService.getMetrics();
streamFormatter.emitEvent({
type: JsonStreamEventType.RESULT,
timestamp: new Date().toISOString(),
status: 'error',
error: {
type: errorType ?? 'FatalToolExecutionError',
message: toolExecutionError.message,
},
stats: streamFormatter.convertToStreamStats(metrics, 0),
});
} else if (config.getOutputFormat() === OutputFormat.JSON) {
const formatter = new JsonFormatter();
const formattedError = formatter.formatError(
toolExecutionError,
@@ -133,7 +167,21 @@ export function handleToolError(
export function handleCancellationError(config: Config): never {
const cancellationError = new FatalCancellationError('Operation cancelled.');
if (config.getOutputFormat() === OutputFormat.JSON) {
if (config.getOutputFormat() === OutputFormat.STREAM_JSON) {
const streamFormatter = new StreamJsonFormatter();
const metrics = uiTelemetryService.getMetrics();
streamFormatter.emitEvent({
type: JsonStreamEventType.RESULT,
timestamp: new Date().toISOString(),
status: 'error',
error: {
type: 'FatalCancellationError',
message: cancellationError.message,
},
stats: streamFormatter.convertToStreamStats(metrics, 0),
});
process.exit(cancellationError.exitCode);
} else if (config.getOutputFormat() === OutputFormat.JSON) {
const formatter = new JsonFormatter();
const formattedError = formatter.formatError(
cancellationError,
@@ -156,7 +204,21 @@ export function handleMaxTurnsExceededError(config: Config): never {
'Reached max session turns for this session. Increase the number of turns by specifying maxSessionTurns in settings.json.',
);
if (config.getOutputFormat() === OutputFormat.JSON) {
if (config.getOutputFormat() === OutputFormat.STREAM_JSON) {
const streamFormatter = new StreamJsonFormatter();
const metrics = uiTelemetryService.getMetrics();
streamFormatter.emitEvent({
type: JsonStreamEventType.RESULT,
timestamp: new Date().toISOString(),
status: 'error',
error: {
type: 'FatalTurnLimitedError',
message: maxTurnsError.message,
},
stats: streamFormatter.convertToStreamStats(metrics, 0),
});
process.exit(maxTurnsError.exitCode);
} else if (config.getOutputFormat() === OutputFormat.JSON) {
const formatter = new JsonFormatter();
const formattedError = formatter.formatError(
maxTurnsError,
+1
View File
@@ -8,6 +8,7 @@
export * from './config/config.js';
export * from './output/types.js';
export * from './output/json-formatter.js';
export * from './output/stream-json-formatter.js';
export * from './policy/types.js';
export * from './policy/policy-engine.js';
@@ -0,0 +1,554 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { StreamJsonFormatter } from './stream-json-formatter.js';
import { JsonStreamEventType } from './types.js';
import type {
InitEvent,
MessageEvent,
ToolUseEvent,
ToolResultEvent,
ErrorEvent,
ResultEvent,
} from './types.js';
import type { SessionMetrics } from '../telemetry/uiTelemetry.js';
import { ToolCallDecision } from '../telemetry/tool-call-decision.js';
describe('StreamJsonFormatter', () => {
let formatter: StreamJsonFormatter;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let stdoutWriteSpy: any;
beforeEach(() => {
formatter = new StreamJsonFormatter();
stdoutWriteSpy = vi
.spyOn(process.stdout, 'write')
.mockImplementation(() => true);
});
afterEach(() => {
stdoutWriteSpy.mockRestore();
});
describe('formatEvent', () => {
it('should format init event as JSONL', () => {
const event: InitEvent = {
type: JsonStreamEventType.INIT,
timestamp: '2025-10-10T12:00:00.000Z',
session_id: 'test-session-123',
model: 'gemini-2.0-flash-exp',
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format user message event', () => {
const event: MessageEvent = {
type: JsonStreamEventType.MESSAGE,
timestamp: '2025-10-10T12:00:00.000Z',
role: 'user',
content: 'What is 2+2?',
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format assistant message event with delta', () => {
const event: MessageEvent = {
type: JsonStreamEventType.MESSAGE,
timestamp: '2025-10-10T12:00:00.000Z',
role: 'assistant',
content: '4',
delta: true,
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
const parsed = JSON.parse(result.trim());
expect(parsed.delta).toBe(true);
});
it('should format tool_use event', () => {
const event: ToolUseEvent = {
type: JsonStreamEventType.TOOL_USE,
timestamp: '2025-10-10T12:00:00.000Z',
tool_name: 'Read',
tool_id: 'read-123',
parameters: { file_path: '/path/to/file.txt' },
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format tool_result event (success)', () => {
const event: ToolResultEvent = {
type: JsonStreamEventType.TOOL_RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
tool_id: 'read-123',
status: 'success',
output: 'File contents here',
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format tool_result event (error)', () => {
const event: ToolResultEvent = {
type: JsonStreamEventType.TOOL_RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
tool_id: 'read-123',
status: 'error',
error: {
type: 'FILE_NOT_FOUND',
message: 'File not found',
},
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format error event', () => {
const event: ErrorEvent = {
type: JsonStreamEventType.ERROR,
timestamp: '2025-10-10T12:00:00.000Z',
severity: 'warning',
message: 'Loop detected, stopping execution',
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format result event with success status', () => {
const event: ResultEvent = {
type: JsonStreamEventType.RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
status: 'success',
stats: {
total_tokens: 100,
input_tokens: 50,
output_tokens: 50,
duration_ms: 1200,
tool_calls: 2,
},
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should format result event with error status', () => {
const event: ResultEvent = {
type: JsonStreamEventType.RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
status: 'error',
error: {
type: 'MaxSessionTurnsError',
message: 'Maximum session turns exceeded',
},
stats: {
total_tokens: 100,
input_tokens: 50,
output_tokens: 50,
duration_ms: 1200,
tool_calls: 0,
},
};
const result = formatter.formatEvent(event);
expect(result).toBe(JSON.stringify(event) + '\n');
expect(JSON.parse(result.trim())).toEqual(event);
});
it('should produce minified JSON without pretty-printing', () => {
const event: MessageEvent = {
type: JsonStreamEventType.MESSAGE,
timestamp: '2025-10-10T12:00:00.000Z',
role: 'user',
content: 'Test',
};
const result = formatter.formatEvent(event);
// Should not contain multiple spaces or newlines (except trailing)
expect(result).not.toContain(' ');
expect(result.split('\n').length).toBe(2); // JSON + trailing newline
});
});
describe('emitEvent', () => {
it('should write formatted event to stdout', () => {
const event: InitEvent = {
type: JsonStreamEventType.INIT,
timestamp: '2025-10-10T12:00:00.000Z',
session_id: 'test-session',
model: 'gemini-2.0-flash-exp',
};
formatter.emitEvent(event);
expect(stdoutWriteSpy).toHaveBeenCalledTimes(1);
expect(stdoutWriteSpy).toHaveBeenCalledWith(JSON.stringify(event) + '\n');
});
it('should emit multiple events sequentially', () => {
const event1: InitEvent = {
type: JsonStreamEventType.INIT,
timestamp: '2025-10-10T12:00:00.000Z',
session_id: 'test-session',
model: 'gemini-2.0-flash-exp',
};
const event2: MessageEvent = {
type: JsonStreamEventType.MESSAGE,
timestamp: '2025-10-10T12:00:01.000Z',
role: 'user',
content: 'Hello',
};
formatter.emitEvent(event1);
formatter.emitEvent(event2);
expect(stdoutWriteSpy).toHaveBeenCalledTimes(2);
expect(stdoutWriteSpy).toHaveBeenNthCalledWith(
1,
JSON.stringify(event1) + '\n',
);
expect(stdoutWriteSpy).toHaveBeenNthCalledWith(
2,
JSON.stringify(event2) + '\n',
);
});
});
describe('convertToStreamStats', () => {
it('should aggregate token counts from single model', () => {
const metrics: SessionMetrics = {
models: {
'gemini-2.0-flash': {
api: {
totalRequests: 1,
totalErrors: 0,
totalLatencyMs: 1000,
},
tokens: {
prompt: 50,
candidates: 30,
total: 80,
cached: 0,
thoughts: 0,
tool: 0,
},
},
},
tools: {
totalCalls: 2,
totalSuccess: 2,
totalFail: 0,
totalDurationMs: 500,
totalDecisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 2,
},
byName: {},
},
files: {
totalLinesAdded: 0,
totalLinesRemoved: 0,
},
};
const result = formatter.convertToStreamStats(metrics, 1200);
expect(result).toEqual({
total_tokens: 80,
input_tokens: 50,
output_tokens: 30,
duration_ms: 1200,
tool_calls: 2,
});
});
it('should aggregate token counts from multiple models', () => {
const metrics: SessionMetrics = {
models: {
'gemini-2.0-flash': {
api: {
totalRequests: 1,
totalErrors: 0,
totalLatencyMs: 1000,
},
tokens: {
prompt: 50,
candidates: 30,
total: 80,
cached: 0,
thoughts: 0,
tool: 0,
},
},
'gemini-1.5-pro': {
api: {
totalRequests: 1,
totalErrors: 0,
totalLatencyMs: 2000,
},
tokens: {
prompt: 100,
candidates: 70,
total: 170,
cached: 0,
thoughts: 0,
tool: 0,
},
},
},
tools: {
totalCalls: 5,
totalSuccess: 5,
totalFail: 0,
totalDurationMs: 1000,
totalDecisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 5,
},
byName: {},
},
files: {
totalLinesAdded: 0,
totalLinesRemoved: 0,
},
};
const result = formatter.convertToStreamStats(metrics, 3000);
expect(result).toEqual({
total_tokens: 250, // 80 + 170
input_tokens: 150, // 50 + 100
output_tokens: 100, // 30 + 70
duration_ms: 3000,
tool_calls: 5,
});
});
it('should handle empty metrics', () => {
const metrics: SessionMetrics = {
models: {},
tools: {
totalCalls: 0,
totalSuccess: 0,
totalFail: 0,
totalDurationMs: 0,
totalDecisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 0,
},
byName: {},
},
files: {
totalLinesAdded: 0,
totalLinesRemoved: 0,
},
};
const result = formatter.convertToStreamStats(metrics, 100);
expect(result).toEqual({
total_tokens: 0,
input_tokens: 0,
output_tokens: 0,
duration_ms: 100,
tool_calls: 0,
});
});
it('should use session-level tool calls count', () => {
const metrics: SessionMetrics = {
models: {},
tools: {
totalCalls: 3,
totalSuccess: 2,
totalFail: 1,
totalDurationMs: 500,
totalDecisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 3,
},
byName: {
Read: {
count: 2,
success: 2,
fail: 0,
durationMs: 300,
decisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 2,
},
},
Glob: {
count: 1,
success: 0,
fail: 1,
durationMs: 200,
decisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 1,
},
},
},
},
files: {
totalLinesAdded: 0,
totalLinesRemoved: 0,
},
};
const result = formatter.convertToStreamStats(metrics, 1000);
expect(result.tool_calls).toBe(3);
});
it('should pass through duration unchanged', () => {
const metrics: SessionMetrics = {
models: {},
tools: {
totalCalls: 0,
totalSuccess: 0,
totalFail: 0,
totalDurationMs: 0,
totalDecisions: {
[ToolCallDecision.ACCEPT]: 0,
[ToolCallDecision.REJECT]: 0,
[ToolCallDecision.MODIFY]: 0,
[ToolCallDecision.AUTO_ACCEPT]: 0,
},
byName: {},
},
files: {
totalLinesAdded: 0,
totalLinesRemoved: 0,
},
};
const result = formatter.convertToStreamStats(metrics, 5000);
expect(result.duration_ms).toBe(5000);
});
});
describe('JSON validity', () => {
it('should produce valid JSON for all event types', () => {
const events = [
{
type: JsonStreamEventType.INIT,
timestamp: '2025-10-10T12:00:00.000Z',
session_id: 'test',
model: 'gemini-2.0-flash',
} as InitEvent,
{
type: JsonStreamEventType.MESSAGE,
timestamp: '2025-10-10T12:00:00.000Z',
role: 'user',
content: 'Test',
} as MessageEvent,
{
type: JsonStreamEventType.TOOL_USE,
timestamp: '2025-10-10T12:00:00.000Z',
tool_name: 'Read',
tool_id: 'read-1',
parameters: {},
} as ToolUseEvent,
{
type: JsonStreamEventType.TOOL_RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
tool_id: 'read-1',
status: 'success',
} as ToolResultEvent,
{
type: JsonStreamEventType.ERROR,
timestamp: '2025-10-10T12:00:00.000Z',
severity: 'error',
message: 'Test error',
} as ErrorEvent,
{
type: JsonStreamEventType.RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
status: 'success',
stats: {
total_tokens: 0,
input_tokens: 0,
output_tokens: 0,
duration_ms: 0,
tool_calls: 0,
},
} as ResultEvent,
];
events.forEach((event) => {
const formatted = formatter.formatEvent(event);
expect(() => JSON.parse(formatted)).not.toThrow();
});
});
it('should preserve field types', () => {
const event: ResultEvent = {
type: JsonStreamEventType.RESULT,
timestamp: '2025-10-10T12:00:00.000Z',
status: 'success',
stats: {
total_tokens: 100,
input_tokens: 50,
output_tokens: 50,
duration_ms: 1200,
tool_calls: 2,
},
};
const formatted = formatter.formatEvent(event);
const parsed = JSON.parse(formatted.trim());
expect(typeof parsed.stats.total_tokens).toBe('number');
expect(typeof parsed.stats.input_tokens).toBe('number');
expect(typeof parsed.stats.output_tokens).toBe('number');
expect(typeof parsed.stats.duration_ms).toBe('number');
expect(typeof parsed.stats.tool_calls).toBe('number');
});
});
});
@@ -0,0 +1,62 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import type { JsonStreamEvent, StreamStats } from './types.js';
import type { SessionMetrics } from '../telemetry/uiTelemetry.js';
/**
* Formatter for streaming JSON output.
* Emits newline-delimited JSON (JSONL) events to stdout in real-time.
*/
export class StreamJsonFormatter {
/**
* Formats a single event as a JSON string with newline (JSONL format).
* @param event - The stream event to format
* @returns JSON string with trailing newline
*/
formatEvent(event: JsonStreamEvent): string {
return JSON.stringify(event) + '\n';
}
/**
* Emits an event directly to stdout in JSONL format.
* @param event - The stream event to emit
*/
emitEvent(event: JsonStreamEvent): void {
process.stdout.write(this.formatEvent(event));
}
/**
* Converts SessionMetrics to simplified StreamStats format.
* Aggregates token counts across all models.
* @param metrics - The session metrics from telemetry
* @param durationMs - The session duration in milliseconds
* @returns Simplified stats for streaming output
*/
convertToStreamStats(
metrics: SessionMetrics,
durationMs: number,
): StreamStats {
let totalTokens = 0;
let inputTokens = 0;
let outputTokens = 0;
// Aggregate token counts across all models
for (const modelMetrics of Object.values(metrics.models)) {
totalTokens += modelMetrics.tokens.total;
inputTokens += modelMetrics.tokens.prompt;
outputTokens += modelMetrics.tokens.candidates;
}
return {
total_tokens: totalTokens,
input_tokens: inputTokens,
output_tokens: outputTokens,
duration_ms: durationMs,
tool_calls: metrics.tools.totalCalls,
};
}
}
+79
View File
@@ -9,6 +9,7 @@ import type { SessionMetrics } from '../telemetry/uiTelemetry.js';
export enum OutputFormat {
TEXT = 'text',
JSON = 'json',
STREAM_JSON = 'stream-json',
}
export interface JsonError {
@@ -22,3 +23,81 @@ export interface JsonOutput {
stats?: SessionMetrics;
error?: JsonError;
}
// Streaming JSON event types
export enum JsonStreamEventType {
INIT = 'init',
MESSAGE = 'message',
TOOL_USE = 'tool_use',
TOOL_RESULT = 'tool_result',
ERROR = 'error',
RESULT = 'result',
}
export interface BaseJsonStreamEvent {
type: JsonStreamEventType;
timestamp: string;
}
export interface InitEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.INIT;
session_id: string;
model: string;
}
export interface MessageEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.MESSAGE;
role: 'user' | 'assistant';
content: string;
delta?: boolean;
}
export interface ToolUseEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.TOOL_USE;
tool_name: string;
tool_id: string;
parameters: Record<string, unknown>;
}
export interface ToolResultEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.TOOL_RESULT;
tool_id: string;
status: 'success' | 'error';
output?: string;
error?: {
type: string;
message: string;
};
}
export interface ErrorEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.ERROR;
severity: 'warning' | 'error';
message: string;
}
export interface StreamStats {
total_tokens: number;
input_tokens: number;
output_tokens: number;
duration_ms: number;
tool_calls: number;
}
export interface ResultEvent extends BaseJsonStreamEvent {
type: JsonStreamEventType.RESULT;
status: 'success' | 'error';
error?: {
type: string;
message: string;
};
stats?: StreamStats;
}
export type JsonStreamEvent =
| InitEvent
| MessageEvent
| ToolUseEvent
| ToolResultEvent
| ErrorEvent
| ResultEvent;