diff --git a/README.md b/README.md index 432bf187ae..1a975738b0 100644 --- a/README.md +++ b/README.md @@ -228,6 +228,13 @@ the `--output-format json` flag to get structured output: gemini -p "Explain the architecture of this codebase" --output-format json ``` +For real-time event streaming (useful for monitoring long-running operations), +use `--output-format stream-json` to get newline-delimited JSON events: + +```bash +gemini -p "Run tests and deploy" --output-format stream-json +``` + ### Quick Examples #### Start a new project diff --git a/docs/cli/headless.md b/docs/cli/headless.md index 31e05c34dd..a0c081341c 100644 --- a/docs/cli/headless.md +++ b/docs/cli/headless.md @@ -15,6 +15,13 @@ scripting, automation, CI/CD pipelines, and building AI-powered tools. - [JSON Output](#json-output) - [Response Schema](#response-schema) - [Example Usage](#example-usage) + - [Streaming JSON Output](#streaming-json-output) + - [When to Use Streaming JSON](#when-to-use-streaming-json) + - [Event Types](#event-types) + - [Basic Usage](#basic-usage) + - [Example Output](#example-output) + - [Processing Stream Events](#processing-stream-events) + - [Real-World Examples](#real-world-examples) - [File Redirection](#file-redirection) - [Configuration Options](#configuration-options) - [Examples](#examples) @@ -211,6 +218,62 @@ Response: } ``` +### Streaming JSON Output + +Returns real-time events as newline-delimited JSON (JSONL). Each significant +action (initialization, messages, tool calls, results) emits immediately as it +occurs. This format is ideal for monitoring long-running operations, building +UIs with live progress, and creating automation pipelines that react to events. + +#### When to Use Streaming JSON + +Use `--output-format stream-json` when you need: + +- **Real-time progress monitoring** - See tool calls and responses as they + happen +- **Event-driven automation** - React to specific events (e.g., tool failures) +- **Live UI updates** - Build interfaces showing AI agent activity in real-time +- **Detailed execution logs** - Capture complete interaction history with + timestamps +- **Pipeline integration** - Stream events to logging/monitoring systems + +#### Event Types + +The streaming format emits 6 event types: + +1. **`init`** - Session starts (includes session_id, model) +2. **`message`** - User prompts and assistant responses +3. **`tool_use`** - Tool call requests with parameters +4. **`tool_result`** - Tool execution results (success/error) +5. **`error`** - Non-fatal errors and warnings +6. **`result`** - Final session outcome with aggregated stats + +#### Basic Usage + +```bash +# Stream events to console +gemini --output-format stream-json --prompt "What is 2+2?" + +# Save event stream to file +gemini --output-format stream-json --prompt "Analyze this code" > events.jsonl + +# Parse with jq +gemini --output-format stream-json --prompt "List files" | jq -r '.type' +``` + +#### Example Output + +Each line is a complete JSON event: + +```jsonl +{"type":"init","timestamp":"2025-10-10T12:00:00.000Z","session_id":"abc123","model":"gemini-2.0-flash-exp"} +{"type":"message","role":"user","content":"List files in current directory","timestamp":"2025-10-10T12:00:01.000Z"} +{"type":"tool_use","tool_name":"Bash","tool_id":"bash-123","parameters":{"command":"ls -la"},"timestamp":"2025-10-10T12:00:02.000Z"} +{"type":"tool_result","tool_id":"bash-123","status":"success","output":"file1.txt\nfile2.txt","timestamp":"2025-10-10T12:00:03.000Z"} +{"type":"message","role":"assistant","content":"Here are the files...","delta":true,"timestamp":"2025-10-10T12:00:04.000Z"} +{"type":"result","status":"success","stats":{"total_tokens":250,"input_tokens":50,"output_tokens":200,"duration_ms":3000,"tool_calls":1},"timestamp":"2025-10-10T12:00:05.000Z"} +``` + ### File Redirection Save output to files or pipe to other commands: @@ -233,16 +296,16 @@ gemini -p "List programming languages" | grep -i "python" Key command-line options for headless usage: -| Option | Description | Example | -| ----------------------- | ---------------------------------- | -------------------------------------------------- | -| `--prompt`, `-p` | Run in headless mode | `gemini -p "query"` | -| `--output-format` | Specify output format (text, json) | `gemini -p "query" --output-format json` | -| `--model`, `-m` | Specify the Gemini model | `gemini -p "query" -m gemini-2.5-flash` | -| `--debug`, `-d` | Enable debug mode | `gemini -p "query" --debug` | -| `--all-files`, `-a` | Include all files in context | `gemini -p "query" --all-files` | -| `--include-directories` | Include additional directories | `gemini -p "query" --include-directories src,docs` | -| `--yolo`, `-y` | Auto-approve all actions | `gemini -p "query" --yolo` | -| `--approval-mode` | Set approval mode | `gemini -p "query" --approval-mode auto_edit` | +| Option | Description | Example | +| ----------------------- | ----------------------------------------------- | -------------------------------------------------- | +| `--prompt`, `-p` | Run in headless mode | `gemini -p "query"` | +| `--output-format` | Specify output format (text, json, stream-json) | `gemini -p "query" --output-format stream-json` | +| `--model`, `-m` | Specify the Gemini model | `gemini -p "query" -m gemini-2.5-flash` | +| `--debug`, `-d` | Enable debug mode | `gemini -p "query" --debug` | +| `--all-files`, `-a` | Include all files in context | `gemini -p "query" --all-files` | +| `--include-directories` | Include additional directories | `gemini -p "query" --include-directories src,docs` | +| `--yolo`, `-y` | Auto-approve all actions | `gemini -p "query" --yolo` | +| `--approval-mode` | Set approval mode | `gemini -p "query" --approval-mode auto_edit` | For complete details on all available configuration options, settings files, and environment variables, see the diff --git a/docs/cli/telemetry.md b/docs/cli/telemetry.md index 954fe4d90e..a0c1773fa1 100644 --- a/docs/cli/telemetry.md +++ b/docs/cli/telemetry.md @@ -227,7 +227,7 @@ for Gemini CLI: - `file_filtering_respect_git_ignore` (boolean) - `debug_mode` (boolean) - `mcp_servers` (string) - - `output_format` (string: "text" or "json") + - `output_format` (string: "text", "json", or "stream-json") - `gemini_cli.user_prompt`: This event occurs when a user submits a prompt. - **Attributes**: diff --git a/docs/get-started/configuration.md b/docs/get-started/configuration.md index 8e519f71f7..8b59f271f5 100644 --- a/docs/get-started/configuration.md +++ b/docs/get-started/configuration.md @@ -128,7 +128,7 @@ their corresponding top-level category object in your `settings.json` file. - **`output.format`** (string): - **Description:** The format of the CLI output. - **Default:** `"text"` - - **Values:** `"text"`, `"json"` + - **Values:** `"text"`, `"json"`, `"stream-json"` #### `ui` @@ -718,8 +718,9 @@ for that specific session. - **Values:** - `text`: (Default) The standard human-readable output. - `json`: A machine-readable JSON output. + - `stream-json`: A streaming JSON output that emits real-time events. - **Note:** For structured output and scripting, use the - `--output-format json` flag. + `--output-format json` or `--output-format stream-json` flag. - **`--sandbox`** (**`-s`**): - Enables sandbox mode for this session. - **`--sandbox-image`**: diff --git a/packages/cli/src/config/config.test.ts b/packages/cli/src/config/config.test.ts index 7b1e888e87..ebe40726a7 100644 --- a/packages/cli/src/config/config.test.ts +++ b/packages/cli/src/config/config.test.ts @@ -3432,6 +3432,22 @@ describe('Output format', () => { expect(config.getOutputFormat()).toBe(OutputFormat.JSON); }); + it('should accept stream-json as a valid output format', async () => { + process.argv = ['node', 'script.js', '--output-format', 'stream-json']; + const argv = await parseArguments({} as Settings); + const config = await loadCliConfig( + {}, + [], + new ExtensionEnablementManager( + ExtensionStorage.getUserExtensionsDir(), + argv.extensions, + ), + 'test-session', + argv, + ); + expect(config.getOutputFormat()).toBe(OutputFormat.STREAM_JSON); + }); + it('should error on invalid --output-format argument', async () => { process.argv = ['node', 'script.js', '--output-format', 'yaml']; const mockExit = vi.spyOn(process, 'exit').mockImplementation(() => { diff --git a/packages/cli/src/config/config.ts b/packages/cli/src/config/config.ts index 35c6a88aa7..13c9a4def2 100755 --- a/packages/cli/src/config/config.ts +++ b/packages/cli/src/config/config.ts @@ -286,7 +286,7 @@ export async function parseArguments(settings: Settings): Promise { alias: 'o', type: 'string', description: 'The format of the CLI output.', - choices: ['text', 'json'], + choices: ['text', 'json', 'stream-json'], }) .deprecateOption( 'show-memory-usage', diff --git a/packages/cli/src/nonInteractiveCli.ts b/packages/cli/src/nonInteractiveCli.ts index 54f9eb909e..4d854139e2 100644 --- a/packages/cli/src/nonInteractiveCli.ts +++ b/packages/cli/src/nonInteractiveCli.ts @@ -20,6 +20,8 @@ import { promptIdContext, OutputFormat, JsonFormatter, + StreamJsonFormatter, + JsonStreamEventType, uiTelemetryService, } from '@google/gemini-cli-core'; @@ -47,6 +49,12 @@ export async function runNonInteractive( debugMode: config.getDebugMode(), }); + const startTime = Date.now(); + const streamFormatter = + config.getOutputFormat() === OutputFormat.STREAM_JSON + ? new StreamJsonFormatter() + : null; + try { consolePatcher.patch(); // Handle EPIPE errors when the output is piped to a command that closes early. @@ -59,6 +67,16 @@ export async function runNonInteractive( const geminiClient = config.getGeminiClient(); + // Emit init event for streaming JSON + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.INIT, + timestamp: new Date().toISOString(), + session_id: config.getSessionId(), + model: config.getModel(), + }); + } + const abortController = new AbortController(); let query: Part[] | undefined; @@ -98,6 +116,16 @@ export async function runNonInteractive( query = processedQuery as Part[]; } + // Emit user message event for streaming JSON + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.MESSAGE, + timestamp: new Date().toISOString(), + role: 'user', + content: input, + }); + } + let currentMessages: Content[] = [{ role: 'user', parts: query }]; let turnCount = 0; @@ -124,13 +152,48 @@ export async function runNonInteractive( } if (event.type === GeminiEventType.Content) { - if (config.getOutputFormat() === OutputFormat.JSON) { + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.MESSAGE, + timestamp: new Date().toISOString(), + role: 'assistant', + content: event.value, + delta: true, + }); + } else if (config.getOutputFormat() === OutputFormat.JSON) { responseText += event.value; } else { process.stdout.write(event.value); } } else if (event.type === GeminiEventType.ToolCallRequest) { + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.TOOL_USE, + timestamp: new Date().toISOString(), + tool_name: event.value.name, + tool_id: event.value.callId, + parameters: event.value.args, + }); + } toolCallRequests.push(event.value); + } else if (event.type === GeminiEventType.LoopDetected) { + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.ERROR, + timestamp: new Date().toISOString(), + severity: 'warning', + message: 'Loop detected, stopping execution', + }); + } + } else if (event.type === GeminiEventType.MaxSessionTurns) { + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.ERROR, + timestamp: new Date().toISOString(), + severity: 'error', + message: 'Maximum session turns exceeded', + }); + } } } @@ -148,6 +211,25 @@ export async function runNonInteractive( completedToolCalls.push(completedToolCall); + if (streamFormatter) { + streamFormatter.emitEvent({ + type: JsonStreamEventType.TOOL_RESULT, + timestamp: new Date().toISOString(), + tool_id: requestInfo.callId, + status: toolResponse.error ? 'error' : 'success', + output: + typeof toolResponse.resultDisplay === 'string' + ? toolResponse.resultDisplay + : undefined, + error: toolResponse.error + ? { + type: toolResponse.errorType || 'TOOL_EXECUTION_ERROR', + message: toolResponse.error.message, + } + : undefined, + }); + } + if (toolResponse.error) { handleToolError( requestInfo.name, @@ -180,7 +262,17 @@ export async function runNonInteractive( currentMessages = [{ role: 'user', parts: toolResponseParts }]; } else { - if (config.getOutputFormat() === OutputFormat.JSON) { + // Emit final result event for streaming JSON + if (streamFormatter) { + const metrics = uiTelemetryService.getMetrics(); + const durationMs = Date.now() - startTime; + streamFormatter.emitEvent({ + type: JsonStreamEventType.RESULT, + timestamp: new Date().toISOString(), + status: 'success', + stats: streamFormatter.convertToStreamStats(metrics, durationMs), + }); + } else if (config.getOutputFormat() === OutputFormat.JSON) { const formatter = new JsonFormatter(); const stats = uiTelemetryService.getMetrics(); process.stdout.write(formatter.format(responseText, stats)); diff --git a/packages/cli/src/utils/errors.ts b/packages/cli/src/utils/errors.ts index ba018434eb..9323f31b48 100644 --- a/packages/cli/src/utils/errors.ts +++ b/packages/cli/src/utils/errors.ts @@ -8,6 +8,9 @@ import type { Config } from '@google/gemini-cli-core'; import { OutputFormat, JsonFormatter, + StreamJsonFormatter, + JsonStreamEventType, + uiTelemetryService, parseAndFormatApiError, FatalTurnLimitedError, FatalCancellationError, @@ -58,6 +61,7 @@ function getNumericExitCode(errorCode: string | number): number { /** * Handles errors consistently for both JSON and text output formats. * In JSON mode, outputs formatted JSON error and exits. + * In streaming JSON mode, emits a result event with error status. * In text mode, outputs error message and re-throws. */ export function handleError( @@ -70,7 +74,24 @@ export function handleError( config.getContentGeneratorConfig()?.authType, ); - if (config.getOutputFormat() === OutputFormat.JSON) { + if (config.getOutputFormat() === OutputFormat.STREAM_JSON) { + const streamFormatter = new StreamJsonFormatter(); + const errorCode = customErrorCode ?? extractErrorCode(error); + const metrics = uiTelemetryService.getMetrics(); + + streamFormatter.emitEvent({ + type: JsonStreamEventType.RESULT, + timestamp: new Date().toISOString(), + status: 'error', + error: { + type: error instanceof Error ? error.constructor.name : 'Error', + message: errorMessage, + }, + stats: streamFormatter.convertToStreamStats(metrics, 0), + }); + + process.exit(getNumericExitCode(errorCode)); + } else if (config.getOutputFormat() === OutputFormat.JSON) { const formatter = new JsonFormatter(); const errorCode = customErrorCode ?? extractErrorCode(error); @@ -110,7 +131,20 @@ export function handleToolError( if (isFatal) { const toolExecutionError = new FatalToolExecutionError(errorMessage); - if (config.getOutputFormat() === OutputFormat.JSON) { + if (config.getOutputFormat() === OutputFormat.STREAM_JSON) { + const streamFormatter = new StreamJsonFormatter(); + const metrics = uiTelemetryService.getMetrics(); + streamFormatter.emitEvent({ + type: JsonStreamEventType.RESULT, + timestamp: new Date().toISOString(), + status: 'error', + error: { + type: errorType ?? 'FatalToolExecutionError', + message: toolExecutionError.message, + }, + stats: streamFormatter.convertToStreamStats(metrics, 0), + }); + } else if (config.getOutputFormat() === OutputFormat.JSON) { const formatter = new JsonFormatter(); const formattedError = formatter.formatError( toolExecutionError, @@ -133,7 +167,21 @@ export function handleToolError( export function handleCancellationError(config: Config): never { const cancellationError = new FatalCancellationError('Operation cancelled.'); - if (config.getOutputFormat() === OutputFormat.JSON) { + if (config.getOutputFormat() === OutputFormat.STREAM_JSON) { + const streamFormatter = new StreamJsonFormatter(); + const metrics = uiTelemetryService.getMetrics(); + streamFormatter.emitEvent({ + type: JsonStreamEventType.RESULT, + timestamp: new Date().toISOString(), + status: 'error', + error: { + type: 'FatalCancellationError', + message: cancellationError.message, + }, + stats: streamFormatter.convertToStreamStats(metrics, 0), + }); + process.exit(cancellationError.exitCode); + } else if (config.getOutputFormat() === OutputFormat.JSON) { const formatter = new JsonFormatter(); const formattedError = formatter.formatError( cancellationError, @@ -156,7 +204,21 @@ export function handleMaxTurnsExceededError(config: Config): never { 'Reached max session turns for this session. Increase the number of turns by specifying maxSessionTurns in settings.json.', ); - if (config.getOutputFormat() === OutputFormat.JSON) { + if (config.getOutputFormat() === OutputFormat.STREAM_JSON) { + const streamFormatter = new StreamJsonFormatter(); + const metrics = uiTelemetryService.getMetrics(); + streamFormatter.emitEvent({ + type: JsonStreamEventType.RESULT, + timestamp: new Date().toISOString(), + status: 'error', + error: { + type: 'FatalTurnLimitedError', + message: maxTurnsError.message, + }, + stats: streamFormatter.convertToStreamStats(metrics, 0), + }); + process.exit(maxTurnsError.exitCode); + } else if (config.getOutputFormat() === OutputFormat.JSON) { const formatter = new JsonFormatter(); const formattedError = formatter.formatError( maxTurnsError, diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 8d48ecd9ec..6abbace7f8 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -8,6 +8,7 @@ export * from './config/config.js'; export * from './output/types.js'; export * from './output/json-formatter.js'; +export * from './output/stream-json-formatter.js'; export * from './policy/types.js'; export * from './policy/policy-engine.js'; diff --git a/packages/core/src/output/stream-json-formatter.test.ts b/packages/core/src/output/stream-json-formatter.test.ts new file mode 100644 index 0000000000..6e9d3c792b --- /dev/null +++ b/packages/core/src/output/stream-json-formatter.test.ts @@ -0,0 +1,554 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { StreamJsonFormatter } from './stream-json-formatter.js'; +import { JsonStreamEventType } from './types.js'; +import type { + InitEvent, + MessageEvent, + ToolUseEvent, + ToolResultEvent, + ErrorEvent, + ResultEvent, +} from './types.js'; +import type { SessionMetrics } from '../telemetry/uiTelemetry.js'; +import { ToolCallDecision } from '../telemetry/tool-call-decision.js'; + +describe('StreamJsonFormatter', () => { + let formatter: StreamJsonFormatter; + // eslint-disable-next-line @typescript-eslint/no-explicit-any + let stdoutWriteSpy: any; + + beforeEach(() => { + formatter = new StreamJsonFormatter(); + stdoutWriteSpy = vi + .spyOn(process.stdout, 'write') + .mockImplementation(() => true); + }); + + afterEach(() => { + stdoutWriteSpy.mockRestore(); + }); + + describe('formatEvent', () => { + it('should format init event as JSONL', () => { + const event: InitEvent = { + type: JsonStreamEventType.INIT, + timestamp: '2025-10-10T12:00:00.000Z', + session_id: 'test-session-123', + model: 'gemini-2.0-flash-exp', + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format user message event', () => { + const event: MessageEvent = { + type: JsonStreamEventType.MESSAGE, + timestamp: '2025-10-10T12:00:00.000Z', + role: 'user', + content: 'What is 2+2?', + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format assistant message event with delta', () => { + const event: MessageEvent = { + type: JsonStreamEventType.MESSAGE, + timestamp: '2025-10-10T12:00:00.000Z', + role: 'assistant', + content: '4', + delta: true, + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + const parsed = JSON.parse(result.trim()); + expect(parsed.delta).toBe(true); + }); + + it('should format tool_use event', () => { + const event: ToolUseEvent = { + type: JsonStreamEventType.TOOL_USE, + timestamp: '2025-10-10T12:00:00.000Z', + tool_name: 'Read', + tool_id: 'read-123', + parameters: { file_path: '/path/to/file.txt' }, + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format tool_result event (success)', () => { + const event: ToolResultEvent = { + type: JsonStreamEventType.TOOL_RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + tool_id: 'read-123', + status: 'success', + output: 'File contents here', + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format tool_result event (error)', () => { + const event: ToolResultEvent = { + type: JsonStreamEventType.TOOL_RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + tool_id: 'read-123', + status: 'error', + error: { + type: 'FILE_NOT_FOUND', + message: 'File not found', + }, + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format error event', () => { + const event: ErrorEvent = { + type: JsonStreamEventType.ERROR, + timestamp: '2025-10-10T12:00:00.000Z', + severity: 'warning', + message: 'Loop detected, stopping execution', + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format result event with success status', () => { + const event: ResultEvent = { + type: JsonStreamEventType.RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + status: 'success', + stats: { + total_tokens: 100, + input_tokens: 50, + output_tokens: 50, + duration_ms: 1200, + tool_calls: 2, + }, + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should format result event with error status', () => { + const event: ResultEvent = { + type: JsonStreamEventType.RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + status: 'error', + error: { + type: 'MaxSessionTurnsError', + message: 'Maximum session turns exceeded', + }, + stats: { + total_tokens: 100, + input_tokens: 50, + output_tokens: 50, + duration_ms: 1200, + tool_calls: 0, + }, + }; + + const result = formatter.formatEvent(event); + + expect(result).toBe(JSON.stringify(event) + '\n'); + expect(JSON.parse(result.trim())).toEqual(event); + }); + + it('should produce minified JSON without pretty-printing', () => { + const event: MessageEvent = { + type: JsonStreamEventType.MESSAGE, + timestamp: '2025-10-10T12:00:00.000Z', + role: 'user', + content: 'Test', + }; + + const result = formatter.formatEvent(event); + + // Should not contain multiple spaces or newlines (except trailing) + expect(result).not.toContain(' '); + expect(result.split('\n').length).toBe(2); // JSON + trailing newline + }); + }); + + describe('emitEvent', () => { + it('should write formatted event to stdout', () => { + const event: InitEvent = { + type: JsonStreamEventType.INIT, + timestamp: '2025-10-10T12:00:00.000Z', + session_id: 'test-session', + model: 'gemini-2.0-flash-exp', + }; + + formatter.emitEvent(event); + + expect(stdoutWriteSpy).toHaveBeenCalledTimes(1); + expect(stdoutWriteSpy).toHaveBeenCalledWith(JSON.stringify(event) + '\n'); + }); + + it('should emit multiple events sequentially', () => { + const event1: InitEvent = { + type: JsonStreamEventType.INIT, + timestamp: '2025-10-10T12:00:00.000Z', + session_id: 'test-session', + model: 'gemini-2.0-flash-exp', + }; + + const event2: MessageEvent = { + type: JsonStreamEventType.MESSAGE, + timestamp: '2025-10-10T12:00:01.000Z', + role: 'user', + content: 'Hello', + }; + + formatter.emitEvent(event1); + formatter.emitEvent(event2); + + expect(stdoutWriteSpy).toHaveBeenCalledTimes(2); + expect(stdoutWriteSpy).toHaveBeenNthCalledWith( + 1, + JSON.stringify(event1) + '\n', + ); + expect(stdoutWriteSpy).toHaveBeenNthCalledWith( + 2, + JSON.stringify(event2) + '\n', + ); + }); + }); + + describe('convertToStreamStats', () => { + it('should aggregate token counts from single model', () => { + const metrics: SessionMetrics = { + models: { + 'gemini-2.0-flash': { + api: { + totalRequests: 1, + totalErrors: 0, + totalLatencyMs: 1000, + }, + tokens: { + prompt: 50, + candidates: 30, + total: 80, + cached: 0, + thoughts: 0, + tool: 0, + }, + }, + }, + tools: { + totalCalls: 2, + totalSuccess: 2, + totalFail: 0, + totalDurationMs: 500, + totalDecisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 2, + }, + byName: {}, + }, + files: { + totalLinesAdded: 0, + totalLinesRemoved: 0, + }, + }; + + const result = formatter.convertToStreamStats(metrics, 1200); + + expect(result).toEqual({ + total_tokens: 80, + input_tokens: 50, + output_tokens: 30, + duration_ms: 1200, + tool_calls: 2, + }); + }); + + it('should aggregate token counts from multiple models', () => { + const metrics: SessionMetrics = { + models: { + 'gemini-2.0-flash': { + api: { + totalRequests: 1, + totalErrors: 0, + totalLatencyMs: 1000, + }, + tokens: { + prompt: 50, + candidates: 30, + total: 80, + cached: 0, + thoughts: 0, + tool: 0, + }, + }, + 'gemini-1.5-pro': { + api: { + totalRequests: 1, + totalErrors: 0, + totalLatencyMs: 2000, + }, + tokens: { + prompt: 100, + candidates: 70, + total: 170, + cached: 0, + thoughts: 0, + tool: 0, + }, + }, + }, + tools: { + totalCalls: 5, + totalSuccess: 5, + totalFail: 0, + totalDurationMs: 1000, + totalDecisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 5, + }, + byName: {}, + }, + files: { + totalLinesAdded: 0, + totalLinesRemoved: 0, + }, + }; + + const result = formatter.convertToStreamStats(metrics, 3000); + + expect(result).toEqual({ + total_tokens: 250, // 80 + 170 + input_tokens: 150, // 50 + 100 + output_tokens: 100, // 30 + 70 + duration_ms: 3000, + tool_calls: 5, + }); + }); + + it('should handle empty metrics', () => { + const metrics: SessionMetrics = { + models: {}, + tools: { + totalCalls: 0, + totalSuccess: 0, + totalFail: 0, + totalDurationMs: 0, + totalDecisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 0, + }, + byName: {}, + }, + files: { + totalLinesAdded: 0, + totalLinesRemoved: 0, + }, + }; + + const result = formatter.convertToStreamStats(metrics, 100); + + expect(result).toEqual({ + total_tokens: 0, + input_tokens: 0, + output_tokens: 0, + duration_ms: 100, + tool_calls: 0, + }); + }); + + it('should use session-level tool calls count', () => { + const metrics: SessionMetrics = { + models: {}, + tools: { + totalCalls: 3, + totalSuccess: 2, + totalFail: 1, + totalDurationMs: 500, + totalDecisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 3, + }, + byName: { + Read: { + count: 2, + success: 2, + fail: 0, + durationMs: 300, + decisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 2, + }, + }, + Glob: { + count: 1, + success: 0, + fail: 1, + durationMs: 200, + decisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 1, + }, + }, + }, + }, + files: { + totalLinesAdded: 0, + totalLinesRemoved: 0, + }, + }; + + const result = formatter.convertToStreamStats(metrics, 1000); + + expect(result.tool_calls).toBe(3); + }); + + it('should pass through duration unchanged', () => { + const metrics: SessionMetrics = { + models: {}, + tools: { + totalCalls: 0, + totalSuccess: 0, + totalFail: 0, + totalDurationMs: 0, + totalDecisions: { + [ToolCallDecision.ACCEPT]: 0, + [ToolCallDecision.REJECT]: 0, + [ToolCallDecision.MODIFY]: 0, + [ToolCallDecision.AUTO_ACCEPT]: 0, + }, + byName: {}, + }, + files: { + totalLinesAdded: 0, + totalLinesRemoved: 0, + }, + }; + + const result = formatter.convertToStreamStats(metrics, 5000); + + expect(result.duration_ms).toBe(5000); + }); + }); + + describe('JSON validity', () => { + it('should produce valid JSON for all event types', () => { + const events = [ + { + type: JsonStreamEventType.INIT, + timestamp: '2025-10-10T12:00:00.000Z', + session_id: 'test', + model: 'gemini-2.0-flash', + } as InitEvent, + { + type: JsonStreamEventType.MESSAGE, + timestamp: '2025-10-10T12:00:00.000Z', + role: 'user', + content: 'Test', + } as MessageEvent, + { + type: JsonStreamEventType.TOOL_USE, + timestamp: '2025-10-10T12:00:00.000Z', + tool_name: 'Read', + tool_id: 'read-1', + parameters: {}, + } as ToolUseEvent, + { + type: JsonStreamEventType.TOOL_RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + tool_id: 'read-1', + status: 'success', + } as ToolResultEvent, + { + type: JsonStreamEventType.ERROR, + timestamp: '2025-10-10T12:00:00.000Z', + severity: 'error', + message: 'Test error', + } as ErrorEvent, + { + type: JsonStreamEventType.RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + status: 'success', + stats: { + total_tokens: 0, + input_tokens: 0, + output_tokens: 0, + duration_ms: 0, + tool_calls: 0, + }, + } as ResultEvent, + ]; + + events.forEach((event) => { + const formatted = formatter.formatEvent(event); + expect(() => JSON.parse(formatted)).not.toThrow(); + }); + }); + + it('should preserve field types', () => { + const event: ResultEvent = { + type: JsonStreamEventType.RESULT, + timestamp: '2025-10-10T12:00:00.000Z', + status: 'success', + stats: { + total_tokens: 100, + input_tokens: 50, + output_tokens: 50, + duration_ms: 1200, + tool_calls: 2, + }, + }; + + const formatted = formatter.formatEvent(event); + const parsed = JSON.parse(formatted.trim()); + + expect(typeof parsed.stats.total_tokens).toBe('number'); + expect(typeof parsed.stats.input_tokens).toBe('number'); + expect(typeof parsed.stats.output_tokens).toBe('number'); + expect(typeof parsed.stats.duration_ms).toBe('number'); + expect(typeof parsed.stats.tool_calls).toBe('number'); + }); + }); +}); diff --git a/packages/core/src/output/stream-json-formatter.ts b/packages/core/src/output/stream-json-formatter.ts new file mode 100644 index 0000000000..b42177b7ff --- /dev/null +++ b/packages/core/src/output/stream-json-formatter.ts @@ -0,0 +1,62 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import type { JsonStreamEvent, StreamStats } from './types.js'; +import type { SessionMetrics } from '../telemetry/uiTelemetry.js'; + +/** + * Formatter for streaming JSON output. + * Emits newline-delimited JSON (JSONL) events to stdout in real-time. + */ +export class StreamJsonFormatter { + /** + * Formats a single event as a JSON string with newline (JSONL format). + * @param event - The stream event to format + * @returns JSON string with trailing newline + */ + formatEvent(event: JsonStreamEvent): string { + return JSON.stringify(event) + '\n'; + } + + /** + * Emits an event directly to stdout in JSONL format. + * @param event - The stream event to emit + */ + emitEvent(event: JsonStreamEvent): void { + process.stdout.write(this.formatEvent(event)); + } + + /** + * Converts SessionMetrics to simplified StreamStats format. + * Aggregates token counts across all models. + * @param metrics - The session metrics from telemetry + * @param durationMs - The session duration in milliseconds + * @returns Simplified stats for streaming output + */ + convertToStreamStats( + metrics: SessionMetrics, + durationMs: number, + ): StreamStats { + let totalTokens = 0; + let inputTokens = 0; + let outputTokens = 0; + + // Aggregate token counts across all models + for (const modelMetrics of Object.values(metrics.models)) { + totalTokens += modelMetrics.tokens.total; + inputTokens += modelMetrics.tokens.prompt; + outputTokens += modelMetrics.tokens.candidates; + } + + return { + total_tokens: totalTokens, + input_tokens: inputTokens, + output_tokens: outputTokens, + duration_ms: durationMs, + tool_calls: metrics.tools.totalCalls, + }; + } +} diff --git a/packages/core/src/output/types.ts b/packages/core/src/output/types.ts index 08477d21ed..0448354255 100644 --- a/packages/core/src/output/types.ts +++ b/packages/core/src/output/types.ts @@ -9,6 +9,7 @@ import type { SessionMetrics } from '../telemetry/uiTelemetry.js'; export enum OutputFormat { TEXT = 'text', JSON = 'json', + STREAM_JSON = 'stream-json', } export interface JsonError { @@ -22,3 +23,81 @@ export interface JsonOutput { stats?: SessionMetrics; error?: JsonError; } + +// Streaming JSON event types +export enum JsonStreamEventType { + INIT = 'init', + MESSAGE = 'message', + TOOL_USE = 'tool_use', + TOOL_RESULT = 'tool_result', + ERROR = 'error', + RESULT = 'result', +} + +export interface BaseJsonStreamEvent { + type: JsonStreamEventType; + timestamp: string; +} + +export interface InitEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.INIT; + session_id: string; + model: string; +} + +export interface MessageEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.MESSAGE; + role: 'user' | 'assistant'; + content: string; + delta?: boolean; +} + +export interface ToolUseEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.TOOL_USE; + tool_name: string; + tool_id: string; + parameters: Record; +} + +export interface ToolResultEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.TOOL_RESULT; + tool_id: string; + status: 'success' | 'error'; + output?: string; + error?: { + type: string; + message: string; + }; +} + +export interface ErrorEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.ERROR; + severity: 'warning' | 'error'; + message: string; +} + +export interface StreamStats { + total_tokens: number; + input_tokens: number; + output_tokens: number; + duration_ms: number; + tool_calls: number; +} + +export interface ResultEvent extends BaseJsonStreamEvent { + type: JsonStreamEventType.RESULT; + status: 'success' | 'error'; + error?: { + type: string; + message: string; + }; + stats?: StreamStats; +} + +export type JsonStreamEvent = + | InitEvent + | MessageEvent + | ToolUseEvent + | ToolResultEvent + | ErrorEvent + | ResultEvent;