Compare commits

..

21 Commits

Author SHA1 Message Date
Coco Sheng f6aa8f70f4 docs(triage): explicitly forbid skipping vagueness checks for complex bugs 2026-04-09 15:04:32 -04:00
Coco Sheng 21e16eb9f7 docs(triage): explicitly exclude complex background AI integrations from help-wanted 2026-04-09 14:42:31 -04:00
Coco Sheng a18c15fa2d docs(triage): explicitly check UI hotkeys during existing feature checks 2026-04-09 14:41:07 -04:00
Coco Sheng e542b81670 docs(triage): explicitly require triage summary for non-help-wanted issues in closed PR re-evaluation 2026-04-09 14:21:35 -04:00
Coco Sheng cda023088f docs(triage): explicitly check for newer duplicates with active PRs 2026-04-09 13:50:12 -04:00
Coco Sheng 7df8d11133 docs(triage): explicitly require explanation for non-help-wanted issues 2026-04-09 13:44:30 -04:00
Coco Sheng a2f75357f8 docs(triage): update triage rules based on maintainer feedback 2026-04-09 13:08:06 -04:00
Coco Sheng 9263ce45fa refactor(skills): move github-issue-triage to workspace-specific skills 2026-04-08 16:12:07 -04:00
Coco Sheng 71154cc4bc chore(skills): remove packaged .skill file from source 2026-04-08 16:10:16 -04:00
Coco Sheng 3c7203f79e docs(skills): add key model and diff/edit functionalities to maintainer-only criteria 2026-04-08 16:06:17 -04:00
Coco Sheng a7ee67c177 docs(skills): add check for existing features before reopening PRs 2026-04-08 16:00:50 -04:00
Coco Sheng 2884958ae1 docs(skills): add token and billing logic to maintainer-only criteria 2026-04-08 15:36:25 -04:00
Coco Sheng bbde72c237 docs(skills): add key model functionalities to maintainer-only criteria 2026-04-08 15:29:34 -04:00
Coco Sheng 786336f9f2 docs(skills): add PR comment step when discarding flawed PRs 2026-04-08 15:07:54 -04:00
Coco Sheng d499979f04 docs(skills): add PR approach evaluation to triage rules to prevent reopening flawed PRs 2026-04-08 15:05:56 -04:00
Coco Sheng 4d3e63b270 docs(skills): add core internal data structure and IPC changes to maintainer-only criteria 2026-04-08 14:37:37 -04:00
Coco Sheng c075a1c6a8 docs(skills): add key UI/UX changes to maintainer-only criteria 2026-04-08 14:20:07 -04:00
Coco Sheng a9192497ed docs(skills): require explanation comment when adding maintainer-only or help-wanted labels 2026-04-08 14:18:01 -04:00
Coco Sheng 2475f75dca docs(skills): clarify help-wanted criteria in github-issue-triage 2026-04-08 14:06:10 -04:00
Coco Sheng 5eb6339025 feat(skills): update issue triage closed PR rule 2026-04-08 14:05:03 -04:00
Coco Sheng affec04593 feat(skills): add github-issue-triage skill 2026-04-08 13:55:06 -04:00
23 changed files with 525 additions and 1182 deletions
@@ -0,0 +1,39 @@
---
name: github-issue-triage
description: Analyzes and cleans up GitHub issues. DO NOT trigger this skill automatically. ONLY use when the user explicitly mentions "github-issue-triage", or explicitly asks to "triage issues", "clean up old issues", or "triage this issue".
---
# GitHub Issue Triage
This skill provides workflows for finding, analyzing, and triaging GitHub issues to maintain a clean and actionable backlog.
## Phase 1: Discovery (Optional)
If the user asks you to "triage issues" or "clean up old issues" without providing a specific issue URL, you must first find candidate issues.
Run the following script to get a list of issues:
`node scripts/find_issues.cjs <owner/repo>` (e.g., `node scripts/find_issues.cjs google-gemini/gemini-cli`)
You may optionally pass a custom search string and limit.
`node scripts/find_issues.cjs <owner/repo> "<search_string>" <limit>`
Pick the first issue from the list to triage and proceed to Phase 2. If the user provided a specific issue URL, start at Phase 2 directly.
## Phase 2: Analysis
For the target issue, you must run the analysis script to gather metadata and determine staleness/inactivity heuristics.
Run:
`node scripts/analyze_issue.cjs <issue_url> "<optional_comma_separated_maintainers>"`
Read the JSON output carefully.
- If `is_stale` is `true`, the issue has already been marked as stale and should be closed according to the rules in Phase 3.
- Take note of `inactive_over_30_days`, `inactive_over_60_days`, `is_epic`, and other boolean flags.
## Phase 3: Triage Execution
After analyzing the issue and receiving the JSON output, you MUST consult the detailed triage rules to determine the next steps.
Read the rules in [references/triage_rules.md](references/triage_rules.md) and execute the appropriate steps. You must follow the steps sequentially.
If a step instructs you to **STOP EXECUTION**, you must conclude your work on this issue and not proceed to subsequent steps. If you are triaging a batch of issues, you may move on to the next issue in the list.
@@ -0,0 +1,162 @@
# Issue Triage Rules
**Important Note on CLI Commands**: When posting comments via the `gh` CLI that contain newlines (e.g. `\n`), you MUST use bash command substitution with `echo -e` so the newlines are rendered correctly. For example: `gh issue comment <issue_url> --body "$(echo -e "### Triage Summary\n\n<your summary>")"`.
When executing triage on an issue, you must evaluate the following steps sequentially using the data provided from `scripts/analyze_issue.cjs` and the issue comments (`gh issue view <issue_url> --json comments`).
## Categorization Guide: Help-wanted
When categorizing an issue, determine if it is a good candidate for community contributions. Only use the **Help-wanted** label for these types of issues. Everything else remains unassigned to a specific whitelist label. Examples of **Help-wanted** issues include:
* Small, well-defined features.
* Easy-to-fix bugs (where the root cause might be identified but the fix isn't trivially "simple" enough to just patch immediately).
* Tasks that are clearly scoped and ready for external help.
* Issues that DO NOT require deep architectural knowledge, significant maintainer review time, modifications to core sensitive business logic (telemetry, security, billing), or sweeping UI/UX changes.
Conversely, do **NOT** use the `help wanted` label for issues such as:
* Easily reproducible bugs with a simple identified fix.
* Epics or roadmap initiatives.
* Changes to core architecture, sensitive security fixes, or internal tasks.
* Issues requiring deep investigation.
* Changes to key UI/UX that affect all users.
* Modifications to core internal data structures and IPC mechanisms.
* Changes to token/billing logic.
* Changes to key model and diff/edit functionalities.
* Features or changes that touch multiple parts of the codebase and would require significant reviewer time from maintainers.
* Changes touching key workflows (like the core stream or execution workflows) that affect all users and require careful architectural consideration.
* Changes touching sensitive business logic, telemetry, or API usage statistics/tracking.
* Proposals affecting community guidelines, contributor frameworks, or project governance that require significant maintainer discretion and alignment.
* Complex new subsystems, such as background LLM integration tasks, AI-driven automation, or new agentic behaviors that require significant design validation.
## Step 1: Resolution Check
**CRITICAL MISTAKE PREVENTION**: You MUST thoroughly read every single comment. Look explicitly for phrases like "appears to fix", "fixes this", "might be fixed", "resolved by", "no longer an issue", or links to other PRs/issues that suggest a resolution. Do not skim. If a user (even a non-maintainer) suggests a fix or PR exists, and the original reporter has not contradicted them, you MUST treat the issue as resolved.
Read ALL the comments from the issue carefully, and review the JSON output for `cross_references`.
Check if ANY of the following conditions are met:
1. Is there a cross-referenced PR in `cross_references` where `is_pr` is `true` and `is_merged` is `true`? If so, use `gh pr view <pr_url> --json title,body` to verify that the PR actually addresses and resolves the issue's request. If it does not, treat condition 1 as NOT met.
2. Is it fixed, resolved, no longer reproducible, or functioning properly in the comments? (e.g., someone mentions it `might be fixed`, `should be fixed`, or `no longer an issue`) AND the reporter has not replied afterward to contradict this?
3. Is there a workaround provided in the comments?
4. Does someone state the issue is actually unrelated to this repository/project, or is an external problem (like a terminal emulator bug)?
- If condition 1 is met: Execute `gh issue close <issue_url> --comment "Closing because this issue was referenced by a merged pull request. Feel free to reopen if the problem persists or if the PR did not fully resolve this." --reason "completed"` and **STOP EXECUTION**.
- If condition 2 or 3 is met: Execute `gh issue close <issue_url> --comment "Closing because the comments indicate this issue might be fixed, has a workaround, is no longer an issue, or is resolved. Feel free to reopen if the problem persists." --reason "completed"` and **STOP EXECUTION**.
- If condition 4 is met: Execute `gh issue close <issue_url> --comment "Closing because the comments indicate this issue is unrelated to this project or is an external problem." --reason "not planned"` and **STOP EXECUTION**.
- If NONE of these conditions are met: Proceed to Step 1.1.
## Step 1.1: Existing Feature Check
**CRITICAL MISTAKE PREVENTION**: If the issue describes a feature request or enhancement (regardless of whether the JSON `is_feature_request` flag is true or false), you **MUST explicitly search the codebase** to verify if it is already implemented. You cannot skip this step for feature requests.
1. Use the `grep_search` tool to look for relevant keywords related to the feature in files like `schemas/settings.schema.json`, `packages/cli/src/config/config.ts`, command definitions, or UI components.
- **Hotkeys & UI Actions**: If the user asks for a way to expand text, pause output, copy text, or perform a UI action, explicitly check `packages/cli/src/ui/key/keyBindings.ts` and the `Command` enum. Many interactive features (e.g., `Ctrl+O` for expanding truncated tool confirmations or output) already exist natively.
2. If you verify that the requested functionality (e.g., a setting, flag, hotkey, or command) already exists natively:
- Execute `gh issue close <issue_url> --comment "This feature is actually already implemented! <Provide a brief explanation of how to use the feature, such as the command to run, the setting to change, or the hotkey to press>.\n\nI'm going to close this issue since the functionality already exists natively. Let us know if you run into any other issues!" --reason "completed"`
- **STOP EXECUTION**.
3. If the feature does NOT exist, proceed to Step 1.2.
## Step 1.2: Closed PR Re-evaluation
If there is a cross-referenced PR in `cross_references` where `is_pr` is `true` and `is_merged` is `false` (and its state is `closed` or it has an automated closure comment):
1. Use `gh pr view <pr_url> --json author,comments,state,title,body` to analyze the PR.
2. Check the comments to see if it was closed by an automated bot (e.g., `gemini-cli` bot closing it automatically due to missing labels like 'help wanted' after 14 days).
3. Analyze the PR's title, body, and comments to determine if it implements a valid and useful feature/fix and is worth resuming.
4. **Existing Feature Check**: Check if the requested feature is actually already implemented natively in the codebase (e.g., an existing hotkey, setting, or command). Search the codebase using `grep_search` if unsure.
- If the feature is ALREADY IMPLEMENTED:
a. Do NOT reopen the PR.
b. Execute `gh issue close <issue_url> --comment "This feature is actually already implemented! <Provide a brief explanation of how to use the feature>.\n\nI'm going to close this issue since the functionality already exists natively. Let us know if you run into any other issues!" --reason "completed"`
c. Comment on the PR: `gh pr comment <pr_url> --body "@<author_username>, thank you for your contribution! We've reviewed this again and decided to keep it closed. The core feature requested in the parent issue is actually already implemented natively in the CLI. We appreciate the effort, but this specific PR is no longer needed!"`
d. **STOP EXECUTION**.
5. Critically evaluate the PR's approach for correctness and safety. Ensure it does not introduce breaking changes, make false assumptions (e.g., making an optional configuration mandatory for all users), or negatively impact other workflows.
- If the PR's approach is flawed or introduces breaking changes:
a. Do NOT reopen the PR.
b. Determine if the issue itself should be **Help-wanted** (using the Categorization Guide).
c. If **Help-wanted**, run `gh issue edit <issue_url> --remove-label "status/need-triage" --add-label "help wanted"`. If not, just remove `status/need-triage`.
d. Execute `gh issue comment <issue_url> --body "### Triage Summary\n\nWhile this issue is valid, the proposed fix in PR <pr_url> introduces potential breaking changes or relies on incorrect assumptions (e.g., <brief explanation of the flaw>). Therefore, we will not reopen that PR, but we still welcome a different approach to fix this issue!"`
e. Comment on the PR: `gh pr comment <pr_url> --body "@<author_username>, thank you for your contribution! We've reviewed the approach in this PR and decided to keep it closed because <brief explanation of the flaw>. We still welcome improvements here using a different approach! Feel free to open a new PR if you're interested."`
f. **STOP EXECUTION**.
- If it is worth resuming AND was closed by a bot:
a. Determine if the issue should be **Help-wanted** (using the Categorization Guide).
b. If it should NOT be **Help-wanted**:
- Execute `gh issue edit <issue_url> --remove-label "status/need-triage"`
- Execute `gh issue comment <issue_url> --body "$(echo -e "### Triage Summary\n\n<brief explanation of why this issue requires maintainer attention and is not suitable for community contribution>")"`
- **STOP EXECUTION**. (Do not reopen the PR).
c. If it should be **Help-wanted**:
- Reopen the PR: `gh pr reopen <pr_url>`
- Assign the PR to the author: `gh pr edit <pr_url> --add-assignee <author_username>`
- Assign the issue to the author: `gh issue edit <issue_url> --add-assignee <author_username>`
- Add the help wanted label to the issue to prevent the bot from closing the PR again: `gh issue edit <issue_url> --add-label "help wanted"`
- Comment on the issue: `gh issue comment <issue_url> --body "### Triage Summary\n\n<brief explanation of why this is a help-wanted task>\n\n@<author_username>, apologies! It looks like your PR <pr_url> was incorrectly closed by our bot. I have reopened it and assigned this issue to you. Would you like to continue working on it?"`
- Comment on the PR: `gh pr comment <pr_url> --body "@<author_username>, apologies for the bot closing this PR! We have reopened it. Please sync your branch to the latest \`main\` and we will have someone review it shortly."`
- Execute `gh issue edit <issue_url> --remove-label "status/need-triage"`
- **STOP EXECUTION**.
- If NOT met: Proceed to Step 1.5.
## Step 1.5: Pending Response Check
Read ALL the comments from the issue carefully.
1. Check if the most recent comments include a request for more information, clarification, or reproduction steps directed at the reporter from any other user (maintainer or community member).
2. Check if the reporter has NOT replied to that request.
3. Check if that request was made over 14 days ago. (You can check the date of the comment vs today's date).
4. **CRITICAL**: Before closing, verify if OTHER users have chimed in after the request to provide the necessary context, answer the question on behalf of the reporter, or confirm the bug's existence. If they have, the issue is NO LONGER pending response and you must proceed to Step 2.
- If conditions 1, 2, and 3 are met AND condition 4 is false: Execute `gh issue close <issue_url> --comment "Closing because more information was requested over 2 weeks ago and we haven't received a response. Feel free to reopen if you can provide the requested details." --reason "not planned"` and **STOP EXECUTION**.
- If NOT met: Proceed to Step 2.
## Step 2: Assignee and Inactivity Handling
Use the JSON output from `analyze_issue.cjs` to determine necessary actions.
**CRITICAL VERIFICATION**: Before proceeding, explicitly verify the exact boolean values of `inactive_over_60_days`, `inactive_over_30_days`, and `is_feature_request` in the JSON output. Do not guess these values based on the date. Note that `is_feature_request` might be `false` even for feature requests if the title/labels lack specific keywords; if the body clearly asks for a new feature/enhancement, treat it as a feature request regardless of the JSON flag.
1. **Assignee Check:** If an assignee is a contributor and hasn't made any updates on the issue for over 2 weeks, execute `gh issue edit <issue_url> --remove-assignee <username>` to remove them. (Do this before proceeding further).
2. **Inactivity Check:**
- If `inactive_over_60_days` is `true`:
a. Formulate a comment to the reporter (@<reporter_username>). Evaluate the issue description and whether it is an Epic (`is_epic`):
- Always mention that the issue is being closed or pinged because it has been inactive for over 60 days.
- IF IT IS AN EPIC: Ask the reporter if it is still in progress or complete, and if it is complete, ask them to close it.
- IF IT IS NOT AN EPIC:
- If it's a feature/enhancement request and the description is relatively vague, ask: 1) if it is still needed and 2) if they can provide more details on the feature request.
- If the issue was mentioned by another issue that is closed as completed or by a pull request that is merged/closed (check `cross_references`), mention this cross-reference (e.g., "I see this issue was mentioned by #123 which is closed as completed...") and ask if this means it is resolved. Do NOT mention cross-references that are still open or closed as "not planned".
- If it's a feature/enhancement request but well-described, just ask if it's still needed.
- If it's a bug, ask if they can reproduce it with the latest build and provide detailed reproduction steps.
- If the issue has assignees, append a ping to the assignees to check in.
- If it is NOT an Epic AND `is_tracked_by_epic` is `false`, append "Feel free to reopen this issue." to the comment.
b. Execute `gh issue edit <issue_url> --remove-label "status/need-triage"`. If it is NOT an Epic, also append `--add-label "status/needs-info"`.
c. Execute `gh issue comment <issue_url> --body "<your formulated comment>"`
d. If it is NOT an Epic AND `is_tracked_by_epic` is `false`, execute `gh issue close <issue_url> --reason "not planned"`.
- After executing these actions, **STOP EXECUTION**.
- If `inactive_over_30_days` is `true` AND it is a bug report (`is_feature_request` is `false`) AND it is NOT an Epic (`is_epic` is `false`):
a. Execute `gh issue edit <issue_url> --remove-label "status/need-triage" --add-label "status/needs-info"`.
b. Execute `gh issue comment <issue_url> --body "@<reporter_username>, this issue has been inactive for over a month. Could you please try reproducing it with the latest nightly build and let us know if it still occurs? If we don't hear back, we will close this issue on <deadline_date>."`
- After executing these actions, **STOP EXECUTION**.
- If neither condition is met, proceed to Step 3.
## Step 3: Vagueness Check
**CRITICAL MISTAKE PREVENTION**: You MUST NOT skip this step just because you recognize the problem domain as complex or requiring maintainer attention. No matter how obvious the problem domain might seem, a bug report without explicit, step-by-step reproduction instructions must be halted and marked as vague first.
Is the issue fundamentally missing context AND no one has asked for more information yet?
- **For bugs**: Explicit reproduction steps are **REQUIRED**. Even if the user provides logs, error traces, or screenshots, if they do not provide clear, step-by-step instructions on how to reproduce the bug, it MUST be considered vague.
- **For feature requests**: If it is just a vague statement without clear use cases or details, it is considered vague.
- If YES (it is vague): Execute `gh issue edit <issue_url> --remove-label "status/need-triage" --add-label "status/needs-info"`. Ask the reporter: `gh issue comment <issue_url> --body "@<reporter_username>, thank you for the report! Could you please provide more specific details (e.g., detailed reproduction steps, expected behavior, logs, and environment details)? Closing this as vague if no response is received in a week."` and **STOP EXECUTION**.
- If NO: Proceed to Step 4.
## Step 4: Reproduction & Code Validity
1. Review the issue comments. If a community member has already clearly identified the root cause of the bug or answered the feature request, DO NOT investigate the code. Proceed to Step 5.
2. Clone the target repository to a temporary directory (`git clone <repo_url> target-repo`).
3. Search the `target-repo/` codebase using `grep_search` and `read_file` ONLY. You are explicitly FORBIDDEN from writing new files, running tests, attempting to fix the code, OR attempting to reproduce the bug by executing code or shell commands. Your ONLY goal is to perform STATIC code analysis to determine if the logic for the bug still exists or if the reported behavior is actually intentional by design.
- If definitively NO LONGER VALID: Close it: `gh issue close <issue_url> --comment "Closing because I have verified this works correctly in the latest codebase. <brief explanation>"` and **STOP EXECUTION**.
- If INTENTIONAL BY DESIGN: Close it: `gh issue close <issue_url> --reason "not planned" --comment "Closing this issue as the reported behavior is intentional by design. <brief explanation of the design logic>"` and **STOP EXECUTION**.
- If still valid: Proceed to Step 5.
## Step 5: Duplicates
Search for duplicates using `gh issue list --search "<keywords>" --repo <owner/repo> --state all`.
- **CRITICAL**: Pay special attention to newer issues that might already have active pull requests or more detailed context. If a duplicate exists that already has an active PR or more maintainer engagement, close the *current* issue you are triaging in favor of the active one.
- If found: `gh issue close <issue_url> --reason "not planned" --comment "Closing as duplicate of #<duplicate_number>."` and **STOP EXECUTION**.
- If no duplicates: Proceed to Step 6.
## Step 6: Triage Summary
Review the issue comments to see if a community member has already identified the root cause.
- Determine if it should be **Help-wanted** (using the Categorization Guide) and explain why in your summary.
- If you categorized the issue as **Help-wanted**, run `gh issue edit <issue_url> --remove-label "status/need-triage" --add-label "help wanted"`.
- If it does not fit **Help-wanted**, you must still explain *why* it requires maintainer attention or why it's not a good fit for community contribution (e.g., "This issue touches core architecture and requires significant maintainer review time"). Simply run `gh issue edit <issue_url> --remove-label "status/need-triage"` to mark it triaged without a specific whitelist label.
- Action: `gh issue comment <issue_url> --body "### Triage Summary\n\n<your summary>"`
- **STOP EXECUTION**.
## Mandatory Final Step
Every issue that is triaged and remains OPEN MUST be assigned at least one of the following labels before you finish your triage process for that issue:
- `status/needs-info` (for inactive, vague, or issues requiring more details)
- `help wanted` (for well-defined, community-friendly issues)
- NONE of the above (If the issue is triaged, valid, but too complex for a community contribution, it should simply remain open without `status/need-triage`). If an issue reaches the end of triage and remains open, you must remove `status/need-triage`.
@@ -0,0 +1,161 @@
const { execSync } = require('child_process');
function runCommand(cmd) {
try {
return execSync(cmd, { encoding: 'utf-8' }).trim();
} catch (error) {
// Return empty or structured error so the agent can see it rather than failing the whole script
return JSON.stringify({ error: error.message });
}
}
async function analyzeIssue(issueLink, maintainersList) {
if (!issueLink) {
console.error(JSON.stringify({ error: "Issue link is required." }));
process.exit(1);
}
const parts = issueLink.split('/');
const issueNumberStr = parts[parts.length - 1];
const issueNumber = parseInt(issueNumberStr, 10);
const repoName = parts[parts.length - 3];
const repoOwner = parts[parts.length - 4];
const repo = `${repoOwner}/${repoName}`;
const maintainers = maintainersList ? maintainersList.split(',').map(m => m.trim()) : [];
const deadline = new Date();
deadline.setDate(deadline.getDate() + 14);
const result = {
issue_link: issueLink,
repo: repo,
issue_number: issueNumber,
is_tracked_by_epic: false,
is_stale: false,
inactive_over_30_days: false,
inactive_over_60_days: false,
has_assignees: false,
is_feature_request: false,
is_high_priority: false,
is_epic: false,
reporter: null,
assignees: [],
labels: [],
cross_references: [],
deadline_date: deadline.toISOString().split('T')[0]
};
try {
// 1. Fetch Issue Data
const issueDataRaw = runCommand(`gh issue view ${issueLink} --json title,body,author,comments,labels,assignees,updatedAt`);
if (issueDataRaw.startsWith('{"error"')) {
console.error(issueDataRaw);
process.exit(1);
}
const issue = JSON.parse(issueDataRaw);
result.reporter = issue.author.login;
result.assignees = issue.assignees.map(a => a.login);
result.has_assignees = result.assignees.length > 0;
result.labels = issue.labels.map(l => l.name);
result.is_feature_request = result.labels.some(l => {
const lower = l.toLowerCase();
return lower.includes('feature') || lower.includes('enhancement');
}) || issue.title.toLowerCase().includes('feature') || issue.title.toLowerCase().includes('proposal');
result.is_high_priority = result.labels.some(l => {
const lower = l.toLowerCase();
return lower.includes('priority/p0') || lower.includes('priority/p1');
});
// 2. Fetch Timeline for cross references
const timelineCmd = `gh api repos/${repo}/issues/${issueNumber}/timeline --jq '[.[] | select(.event == "cross-referenced" and .source.issue)] | map({issue: .source.issue.number, state: .source.issue.state, state_reason: .source.issue.state_reason, is_pr: (.source.issue.pull_request != null), is_merged: (.source.issue.pull_request.merged_at != null)})'`;
const timelineRaw = runCommand(timelineCmd);
if (!timelineRaw.startsWith('{"error"')) {
result.cross_references = JSON.parse(timelineRaw);
}
// 3. Check if Epic (has sub issues or title starts with [Epic])
const epicCmd = `gh api repos/${repo}/issues/${issueNumber} --jq '{is_epic: (.sub_issues_summary.total > 0)}'`;
const epicRaw = runCommand(epicCmd);
if (!epicRaw.startsWith('{"error"')) {
result.is_epic = JSON.parse(epicRaw).is_epic || issue.title.toLowerCase().startsWith('[epic]');
} else {
result.is_epic = issue.title.toLowerCase().startsWith('[epic]');
}
// 4. Check for Parent Issue via GraphQL
const query = `query($owner: String!, $repo: String!, $issueNumber: Int!) { repository(owner: $owner, name: $repo) { issue(number: $issueNumber) { trackedInIssues(first: 1) { totalCount } } } }`;
const gqlCmd = `gh api graphql -F owner=${repoOwner} -F repo=${repoName} -F issueNumber=${issueNumber} -f query='${query}' --jq '.data.repository.issue.trackedInIssues.totalCount'`;
const parentCountRaw = runCommand(gqlCmd);
if (!parentCountRaw.startsWith('{"error"')) {
const count = parseInt(parentCountRaw, 10);
result.is_tracked_by_epic = !isNaN(count) && count > 0;
}
// Staleness Logic
if (result.is_tracked_by_epic) {
result.is_stale = true;
} else {
// Find last maintainer comment
const maintainerComments = issue.comments.filter(c => {
if (c.author.login === result.reporter) return false;
const isMaintainer = maintainers.includes(c.author.login) ||
['OWNER', 'MEMBER', 'COLLABORATOR'].includes(c.authorAssociation);
if (maintainers.length > 0) return isMaintainer;
// Basic bot exclusion if no maintainers defined
return !c.author.login.includes('github-actions') && c.author.login !== 'app/github-actions';
});
if (maintainerComments.length > 0) {
const lastMaintainerComment = maintainerComments[maintainerComments.length - 1];
// Did reporter reply after maintainer?
const reporterReplied = issue.comments.some(c => {
return c.author.login === result.reporter &&
new Date(c.updatedAt) > new Date(lastMaintainerComment.updatedAt);
});
if (!reporterReplied) {
const daysAgo = (new Date() - new Date(lastMaintainerComment.updatedAt)) / (1000 * 60 * 60 * 24);
if (daysAgo > 7) {
result.is_stale = true;
}
}
}
}
// Age / Inactivity check
if (!result.is_stale) {
const lastUpdateDaysAgo = (new Date() - new Date(issue.updatedAt)) / (1000 * 60 * 60 * 24);
if (lastUpdateDaysAgo > 60) {
result.inactive_over_60_days = true;
result.inactive_over_30_days = true;
} else if (lastUpdateDaysAgo > 30) {
result.inactive_over_30_days = true;
}
}
console.log(JSON.stringify(result, null, 2));
} catch (error) {
console.error(JSON.stringify({ error: error.message }));
process.exit(1);
}
}
const args = process.argv.slice(2);
const issueLink = args[0];
const maintainers = args[1] || ""; // Comma separated list of maintainers
if (!issueLink) {
console.log("Usage: node analyze_issue.cjs <issue_link> [maintainers_csv]");
console.log("Example: node analyze_issue.cjs https://github.com/owner/repo/issues/123 'user1,user2'");
process.exit(1);
}
analyzeIssue(issueLink, maintainers);
@@ -0,0 +1,34 @@
const { execSync } = require('child_process');
function findIssues(repo, searchString = "label:area/core,area/extensions,area/site,area/non-interactive sort:updated-asc", limit = 10) {
if (!repo) {
console.error(JSON.stringify({ error: "Repository is required (e.g., owner/repo)" }));
process.exit(1);
}
try {
const cmd = `gh issue list --repo ${repo} --state open --search "${searchString}" --json url --limit ${limit}`;
const output = execSync(cmd, { encoding: 'utf-8' });
const issues = JSON.parse(output);
const urls = issues.map(issue => issue.url);
console.log(JSON.stringify({ issue_urls: urls }, null, 2));
} catch (error) {
console.error(JSON.stringify({ error: error.message }));
process.exit(1);
}
}
const args = process.argv.slice(2);
const repo = args[0];
const searchString = args[1];
const limitStr = args[2];
const limit = limitStr ? parseInt(limitStr, 10) : 10;
if (!repo) {
console.log("Usage: node find_issues.cjs <owner/repo> [search_string] [limit]");
console.log("Example: node find_issues.cjs google-gemini/gemini-cli 'label:area/core sort:updated-asc' 10");
process.exit(1);
}
findIssues(repo, searchString, limit);
-33
View File
@@ -1,33 +0,0 @@
name: 'Memory Tests: Nightly'
on:
schedule:
- cron: '0 2 * * *' # Runs at 2 AM every day
workflow_dispatch: # Allow manual trigger
permissions:
contents: 'read'
jobs:
memory-test:
name: 'Run Memory Usage Tests'
runs-on: 'gemini-cli-ubuntu-16-core'
if: "github.repository == 'google-gemini/gemini-cli'"
steps:
- name: 'Checkout'
uses: 'actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8' # ratchet:actions/checkout@v5
- name: 'Set up Node.js'
uses: 'actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020' # ratchet:actions/setup-node@v4
with:
node-version-file: '.nvmrc'
cache: 'npm'
- name: 'Install dependencies'
run: 'npm ci'
- name: 'Build project'
run: 'npm run build'
- name: 'Run Memory Tests'
run: 'npm run test:memory'
-2
View File
@@ -44,8 +44,6 @@ powerful tool for developers.
- **Test Commands:**
- **Unit (All):** `npm run test`
- **Integration (E2E):** `npm run test:e2e`
- **Memory (Nightly):** `npm run test:memory` (Runs memory regression tests
against baselines. Excluded from `preflight`, run nightly.)
- **Workspace-Specific:** `npm test -w <pkg> -- <path>` (Note: `<path>` must
be relative to the workspace root, e.g.,
`-w @google/gemini-cli-core -- src/routing/modelRouterService.test.ts`)
-40
View File
@@ -117,46 +117,6 @@ npm run test:integration:sandbox:docker
npm run test:integration:sandbox:podman
```
## Memory regression tests
Memory regression tests are designed to detect heap growth and leaks across key
CLI scenarios. They are located in the `memory-tests` directory.
These tests are distinct from standard integration tests because they measure
memory usage and compare it against committed baselines.
### Running memory tests
Memory tests are not run as part of the default `npm run test` or
`npm run test:e2e` commands. They are run nightly in CI but can be run manually:
```bash
npm run test:memory
```
### Updating baselines
If you intentionally change behavior that affects memory usage, you may need to
update the baselines. Set the `UPDATE_MEMORY_BASELINES` environment variable to
`true`:
```bash
UPDATE_MEMORY_BASELINES=true npm run test:memory
```
This will run the tests, take median snapshots, and overwrite
`memory-tests/baselines.json`. You should review the changes and commit the
updated baseline file.
### How it works
The harness (`MemoryTestHarness` in `packages/test-utils`):
- Forces garbage collection multiple times to reduce noise.
- Takes median snapshots to filter spikes.
- Compares against baselines with a 10% tolerance.
- Can analyze sustained leaks across 3 snapshots using `analyzeSnapshots()`.
## Diagnostics
The integration test runner provides several options for diagnostics to help
-30
View File
@@ -1,30 +0,0 @@
{
"version": 1,
"updatedAt": "2026-04-08T01:21:58.770Z",
"scenarios": {
"multi-turn-conversation": {
"heapUsedBytes": 120082704,
"heapTotalBytes": 177586176,
"rssBytes": 269172736,
"timestamp": "2026-04-08T01:21:57.127Z"
},
"multi-function-call-repo-search": {
"heapUsedBytes": 104644984,
"heapTotalBytes": 111575040,
"rssBytes": 204079104,
"timestamp": "2026-04-08T01:21:58.770Z"
},
"idle-session-startup": {
"heapUsedBytes": 119813672,
"heapTotalBytes": 177061888,
"rssBytes": 267943936,
"timestamp": "2026-04-08T01:21:53.855Z"
},
"simple-prompt-response": {
"heapUsedBytes": 119722064,
"heapTotalBytes": 177324032,
"rssBytes": 268812288,
"timestamp": "2026-04-08T01:21:55.491Z"
}
}
}
-71
View File
@@ -1,71 +0,0 @@
/**
* @license
* Copyright 2026 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { mkdir, readdir, rm } from 'node:fs/promises';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { canUseRipgrep } from '../packages/core/src/tools/ripGrep.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const rootDir = join(__dirname, '..');
const memoryTestsDir = join(rootDir, '.memory-tests');
let runDir = '';
export async function setup() {
runDir = join(memoryTestsDir, `${Date.now()}`);
await mkdir(runDir, { recursive: true });
// Set the home directory to the test run directory to avoid conflicts
// with the user's local config.
process.env['HOME'] = runDir;
if (process.platform === 'win32') {
process.env['USERPROFILE'] = runDir;
}
process.env['GEMINI_CONFIG_DIR'] = join(runDir, '.gemini');
// Download ripgrep to avoid race conditions
const available = await canUseRipgrep();
if (!available) {
throw new Error('Failed to download ripgrep binary');
}
// Clean up old test runs, keeping the latest few for debugging
try {
const testRuns = await readdir(memoryTestsDir);
if (testRuns.length > 3) {
const oldRuns = testRuns.sort().slice(0, testRuns.length - 3);
await Promise.all(
oldRuns.map((oldRun) =>
rm(join(memoryTestsDir, oldRun), {
recursive: true,
force: true,
}),
),
);
}
} catch (e) {
console.error('Error cleaning up old memory test runs:', e);
}
process.env['INTEGRATION_TEST_FILE_DIR'] = runDir;
process.env['GEMINI_CLI_INTEGRATION_TEST'] = 'true';
process.env['GEMINI_FORCE_FILE_STORAGE'] = 'true';
process.env['TELEMETRY_LOG_FILE'] = join(runDir, 'telemetry.log');
process.env['VERBOSE'] = process.env['VERBOSE'] ?? 'false';
console.log(`\nMemory test output directory: ${runDir}`);
}
export async function teardown() {
// Cleanup unless KEEP_OUTPUT is set
if (process.env['KEEP_OUTPUT'] !== 'true' && runDir) {
try {
await rm(runDir, { recursive: true, force: true });
} catch (e) {
console.warn('Failed to clean up memory test directory:', e);
}
}
}
-185
View File
@@ -1,185 +0,0 @@
/**
* @license
* Copyright 2026 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { describe, it, beforeAll, afterAll, afterEach } from 'vitest';
import { TestRig, MemoryTestHarness } from '@google/gemini-cli-test-utils';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const BASELINES_PATH = join(__dirname, 'baselines.json');
const UPDATE_BASELINES = process.env['UPDATE_MEMORY_BASELINES'] === 'true';
const TOLERANCE_PERCENT = 10;
// Fake API key for tests using fake responses
const TEST_ENV = { GEMINI_API_KEY: 'fake-memory-test-key' };
describe('Memory Usage Tests', () => {
let harness: MemoryTestHarness;
let rig: TestRig;
beforeAll(() => {
harness = new MemoryTestHarness({
baselinesPath: BASELINES_PATH,
defaultTolerancePercent: TOLERANCE_PERCENT,
gcCycles: 3,
gcDelayMs: 100,
sampleCount: 3,
});
});
afterEach(async () => {
await rig.cleanup();
});
afterAll(async () => {
// Generate the summary report after all tests
await harness.generateReport();
});
it('idle-session-startup: memory usage within baseline', async () => {
rig = new TestRig();
rig.setup('memory-idle-startup', {
fakeResponsesPath: join(__dirname, 'memory.idle-startup.responses'),
});
const result = await harness.runScenario(
'idle-session-startup',
async (recordSnapshot) => {
await rig.run({
args: ['hello'],
timeout: 120000,
env: TEST_ENV,
});
await recordSnapshot('after-startup');
},
);
if (UPDATE_BASELINES) {
harness.updateScenarioBaseline(result);
console.log(
`Updated baseline for idle-session-startup: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`,
);
} else {
harness.assertWithinBaseline(result);
}
});
it('simple-prompt-response: memory usage within baseline', async () => {
rig = new TestRig();
rig.setup('memory-simple-prompt', {
fakeResponsesPath: join(__dirname, 'memory.simple-prompt.responses'),
});
const result = await harness.runScenario(
'simple-prompt-response',
async (recordSnapshot) => {
await rig.run({
args: ['What is the capital of France?'],
timeout: 120000,
env: TEST_ENV,
});
await recordSnapshot('after-response');
},
);
if (UPDATE_BASELINES) {
harness.updateScenarioBaseline(result);
console.log(
`Updated baseline for simple-prompt-response: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`,
);
} else {
harness.assertWithinBaseline(result);
}
});
it('multi-turn-conversation: memory remains stable over turns', async () => {
rig = new TestRig();
rig.setup('memory-multi-turn', {
fakeResponsesPath: join(__dirname, 'memory.multi-turn.responses'),
});
const prompts = [
'Hello, what can you help me with?',
'Tell me about JavaScript',
'How is TypeScript different?',
'Can you write a simple TypeScript function?',
'What are some TypeScript best practices?',
];
const result = await harness.runScenario(
'multi-turn-conversation',
async (recordSnapshot) => {
// Run through all turns as a piped sequence
const stdinContent = prompts.join('\n');
await rig.run({
stdin: stdinContent,
timeout: 120000,
env: TEST_ENV,
});
// Take snapshots after the conversation completes
await recordSnapshot('after-all-turns');
},
);
if (UPDATE_BASELINES) {
harness.updateScenarioBaseline(result);
console.log(
`Updated baseline for multi-turn-conversation: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`,
);
} else {
harness.assertWithinBaseline(result);
}
});
it('multi-function-call-repo-search: memory after tool use', async () => {
rig = new TestRig();
rig.setup('memory-multi-func-call', {
fakeResponsesPath: join(
__dirname,
'memory.multi-function-call.responses',
),
});
// Create directories first, then files in the workspace so the tools have targets
rig.mkdir('packages/core/src/telemetry');
rig.createFile(
'packages/core/src/telemetry/memory-monitor.ts',
'export class MemoryMonitor { constructor() {} }',
);
rig.createFile(
'packages/core/src/telemetry/metrics.ts',
'export function recordMemoryUsage() {}',
);
const result = await harness.runScenario(
'multi-function-call-repo-search',
async (recordSnapshot) => {
await rig.run({
args: [
'Search this repository for MemoryMonitor and tell me what it does',
],
timeout: 120000,
env: TEST_ENV,
});
await recordSnapshot('after-tool-calls');
},
);
if (UPDATE_BASELINES) {
harness.updateScenarioBaseline(result);
console.log(
`Updated baseline for multi-function-call-repo-search: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`,
);
} else {
harness.assertWithinBaseline(result);
}
});
});
@@ -1,2 +0,0 @@
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help. What would you like to work on?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":12,"totalTokenCount":17,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]}
@@ -1,4 +0,0 @@
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll search for MemoryMonitor in the repository and analyze what it does."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":30,"candidatesTokenCount":15,"totalTokenCount":45,"promptTokensDetails":[{"modality":"TEXT","tokenCount":30}]}}]}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"grep_search","args":{"pattern":"MemoryMonitor","path":".","include_pattern":"*.ts"}}},{"functionCall":{"name":"list_directory","args":{"path":"packages/core/src/telemetry"}}},{"functionCall":{"name":"read_file","args":{"file_path":"packages/core/src/telemetry/memory-monitor.ts"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":30,"candidatesTokenCount":80,"totalTokenCount":110,"promptTokensDetails":[{"modality":"TEXT","tokenCount":30}]}}]}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I found the memory monitoring code. Here's a summary:\n\nThe `MemoryMonitor` class in `packages/core/src/telemetry/memory-monitor.ts` provides:\n\n1. **Continuous monitoring** via `start()`/`stop()` with configurable intervals\n2. **V8 heap snapshots** using `v8.getHeapStatistics()` and `process.memoryUsage()`\n3. **High-water mark tracking** to detect significant memory growth\n4. **Rate-limited recording** to avoid metric flood\n5. **Activity detection** — only records when user is active\n\nThe class uses a singleton pattern via `initializeMemoryMonitor()` for global access."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":500,"candidatesTokenCount":120,"totalTokenCount":620,"promptTokensDetails":[{"modality":"TEXT","tokenCount":500}]}}]}
-10
View File
@@ -1,10 +0,0 @@
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help you with your coding tasks. What would you like to work on today?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":18,"totalTokenCount":23,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]}
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"JavaScript is a high-level, interpreted programming language. It was originally designed for adding interactivity to web pages."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":25,"candidatesTokenCount":60,"totalTokenCount":85,"promptTokensDetails":[{"modality":"TEXT","tokenCount":25}]}}]}
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"TypeScript is a typed superset of JavaScript developed by Microsoft. The main differences from JavaScript are static typing and better tooling."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":45,"candidatesTokenCount":80,"totalTokenCount":125,"promptTokensDetails":[{"modality":"TEXT","tokenCount":45}]}}]}
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Here is a simple TypeScript function:\n\nfunction greet(name: string): string { return `Hello, ${name}!`; }"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":60,"candidatesTokenCount":55,"totalTokenCount":115,"promptTokensDetails":[{"modality":"TEXT","tokenCount":60}]}}]}
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Here are 5 key TypeScript best practices: Enable strict mode, prefer interfaces, use union types, leverage type inference, and use readonly."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":75,"candidatesTokenCount":70,"totalTokenCount":145,"promptTokensDetails":[{"modality":"TEXT","tokenCount":75}]}}]}
@@ -1,2 +0,0 @@
{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}}
{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The capital of France is Paris. It has been the capital since the 10th century and is known for iconic landmarks like the Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral. Paris is also the most populous city in France, with a metropolitan area population of over 12 million people."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":7,"candidatesTokenCount":55,"totalTokenCount":62,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7}]}}]}
-12
View File
@@ -1,12 +0,0 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"noEmit": true,
"allowJs": true
},
"include": ["**/*.ts"],
"references": [
{ "path": "../packages/core" },
{ "path": "../packages/test-utils" }
]
}
-28
View File
@@ -1,28 +0,0 @@
/**
* @license
* Copyright 2026 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
testTimeout: 600000, // 10 minutes — memory profiling is slow
globalSetup: './globalSetup.ts',
reporters: ['default'],
include: ['**/*.test.ts'],
retry: 0, // No retries for memory tests — noise is handled by tolerance
fileParallelism: false, // Must run serially to avoid memory interference
pool: 'forks', // Use forks pool for --expose-gc support
poolOptions: {
forks: {
singleFork: true, // Single process for accurate per-test memory readings
execArgv: ['--expose-gc'], // Enable global.gc() for forced GC
},
},
env: {
GEMINI_TEST_TYPE: 'memory',
},
},
});
+3 -38
View File
@@ -446,8 +446,7 @@
"version": "2.11.0",
"resolved": "https://registry.npmjs.org/@bufbuild/protobuf/-/protobuf-2.11.0.tgz",
"integrity": "sha512-sBXGT13cpmPR5BMgHE6UEEfEaShh5Ror6rfN3yEK5si7QVrtZg8LEPQb0VVhiLRUslD2yLnXtnRzG035J/mZXQ==",
"license": "(Apache-2.0 AND BSD-3-Clause)",
"peer": true
"license": "(Apache-2.0 AND BSD-3-Clause)"
},
"node_modules/@bundled-es-modules/cookie": {
"version": "2.0.1",
@@ -1450,7 +1449,6 @@
"resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.13.4.tgz",
"integrity": "sha512-GsFaMXCkMqkKIvwCQjCrwH+GHbPKBjhwo/8ZuUkWHqbI73Kky9I+pQltrlT0+MWpedCoosda53lgjYfyEPgxBg==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@grpc/proto-loader": "^0.7.13",
"@js-sdsl/ordered-map": "^4.4.2"
@@ -2157,7 +2155,6 @@
"integrity": "sha512-t54CUOsFMappY1Jbzb7fetWeO0n6K0k/4+/ZpkS+3Joz8I4VcvY9OiEBFRYISqaI2fq5sCiPtAjRDOzVYG8m+Q==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@octokit/auth-token": "^6.0.0",
"@octokit/graphql": "^9.0.2",
@@ -2338,7 +2335,6 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz",
"integrity": "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==",
"license": "Apache-2.0",
"peer": true,
"engines": {
"node": ">=8.0.0"
}
@@ -2388,7 +2384,6 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-2.5.0.tgz",
"integrity": "sha512-ka4H8OM6+DlUhSAZpONu0cPBtPPTQKxbxVzC4CzVx5+K4JnroJVBtDzLAMx4/3CDTJXRvVFhpFjtl4SaiTNoyQ==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@opentelemetry/semantic-conventions": "^1.29.0"
},
@@ -2763,7 +2758,6 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-2.5.0.tgz",
"integrity": "sha512-F8W52ApePshpoSrfsSk1H2yJn9aKjCrbpQF1M9Qii0GHzbfVeFUB+rc3X4aggyZD8x9Gu3Slua+s6krmq6Dt8g==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@opentelemetry/core": "2.5.0",
"@opentelemetry/semantic-conventions": "^1.29.0"
@@ -2797,7 +2791,6 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/sdk-metrics/-/sdk-metrics-2.5.0.tgz",
"integrity": "sha512-BeJLtU+f5Gf905cJX9vXFQorAr6TAfK3SPvTFqP+scfIpDQEJfRaGJWta7sJgP+m4dNtBf9y3yvBKVAZZtJQVA==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@opentelemetry/core": "2.5.0",
"@opentelemetry/resources": "2.5.0"
@@ -2852,7 +2845,6 @@
"resolved": "https://registry.npmjs.org/@opentelemetry/sdk-trace-base/-/sdk-trace-base-2.5.0.tgz",
"integrity": "sha512-VzRf8LzotASEyNDUxTdaJ9IRJ1/h692WyArDBInf5puLCjxbICD6XkHgpuudis56EndyS7LYFmtTMny6UABNdQ==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@opentelemetry/core": "2.5.0",
"@opentelemetry/resources": "2.5.0",
@@ -4089,7 +4081,6 @@
"integrity": "sha512-6mDvHUFSjyT2B2yeNx2nUgMxh9LtOWvkhIU3uePn2I2oyNymUAX1NIsdgviM4CH+JSrp2D2hsMvJOkxY+0wNRA==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"csstype": "^3.0.2"
}
@@ -4364,7 +4355,6 @@
"integrity": "sha512-6sMvZePQrnZH2/cJkwRpkT7DxoAWh+g6+GFRK6bV3YQo7ogi3SX5rgF6099r5Q53Ma5qeT7LGmOmuIutF4t3lA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@typescript-eslint/scope-manager": "8.35.0",
"@typescript-eslint/types": "8.35.0",
@@ -5238,7 +5228,6 @@
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
"integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
"license": "MIT",
"peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -5580,12 +5569,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/asciichart": {
"version": "1.5.25",
"resolved": "https://registry.npmjs.org/asciichart/-/asciichart-1.5.25.tgz",
"integrity": "sha512-PNxzXIPPOtWq8T7bgzBtk9cI2lgS4SJZthUHEiQ1aoIc3lNzGfUvIvo9LiAnq26TACo9t1/4qP6KTGAUbzX9Xg==",
"license": "MIT"
},
"node_modules/assertion-error": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz",
@@ -7379,8 +7362,7 @@
"version": "0.0.1581282",
"resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.1581282.tgz",
"integrity": "sha512-nv7iKtNZQshSW2hKzYNr46nM/Cfh5SEvE2oV0/SEGgc9XupIY5ggf84Cz8eJIkBce7S3bmTAauFD6aysMpnqsQ==",
"license": "BSD-3-Clause",
"peer": true
"license": "BSD-3-Clause"
},
"node_modules/dezalgo": {
"version": "1.0.4",
@@ -7964,7 +7946,6 @@
"integrity": "sha512-GsGizj2Y1rCWDu6XoEekL3RLilp0voSePurjZIkxL3wlm5o5EC9VpgaP7lrCvjnkuLvzFBQWB3vWB3K5KQTveQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@eslint-community/eslint-utils": "^4.2.0",
"@eslint-community/regexpp": "^4.12.1",
@@ -8482,7 +8463,6 @@
"resolved": "https://registry.npmjs.org/express/-/express-5.2.1.tgz",
"integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==",
"license": "MIT",
"peer": true,
"dependencies": {
"accepts": "^2.0.0",
"body-parser": "^2.2.1",
@@ -9795,7 +9775,6 @@
"resolved": "https://registry.npmjs.org/hono/-/hono-4.12.7.tgz",
"integrity": "sha512-jq9l1DM0zVIvsm3lv9Nw9nlJnMNPOcAtsbsgiUhWcFzPE99Gvo6yRTlszSLLYacMeQ6quHD6hMfId8crVHvexw==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=16.9.0"
}
@@ -10074,7 +10053,6 @@
"resolved": "https://registry.npmjs.org/@jrichman/ink/-/ink-6.6.7.tgz",
"integrity": "sha512-bDzQLpLzK/dn9Ur/Ku88ZZR9totVcMGrGYAgPHidsAAbe9NKztU1fggj/iu0wRp5g1kBeALb3cfagFGdDxAU1w==",
"license": "MIT",
"peer": true,
"dependencies": {
"ansi-escapes": "^7.0.0",
"ansi-styles": "^6.2.3",
@@ -13848,7 +13826,6 @@
"resolved": "https://registry.npmjs.org/react/-/react-19.2.4.tgz",
"integrity": "sha512-9nfp2hYpCwOjAN+8TZFGhtWEwgvWHXqESH8qT89AT/lWklpLON22Lc8pEtnpsZz7VmawabSU0gCjnj8aC0euHQ==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -13859,7 +13836,6 @@
"integrity": "sha512-ePrwPfxAnB+7hgnEr8vpKxL9cmnp7F322t8oqcPshbIQQhDKgFDW4tjhF2wjVbdXF9O/nyuy3sQWd9JGpiLPvA==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"shell-quote": "^1.6.1",
"ws": "^7"
@@ -16009,7 +15985,6 @@
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -16232,8 +16207,7 @@
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
"license": "0BSD",
"peer": true
"license": "0BSD"
},
"node_modules/tsx": {
"version": "4.20.3",
@@ -16241,7 +16215,6 @@
"integrity": "sha512-qjbnuR9Tr+FJOMBqJCW5ehvIo/buZq7vH7qD7JziU98h6l3qGy0a/yPFjwO+y0/T7GFpNgNAvEcPPVfyT8rrPQ==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"esbuild": "~0.25.0",
"get-tsconfig": "^4.7.5"
@@ -16407,7 +16380,6 @@
"integrity": "sha512-p1diW6TqL9L07nNxvRMM7hMMw4c5XOo/1ibL4aAIGmSAt9slTE1Xgw5KWuof2uTOvCg9BY7ZRi+GaF+7sfgPeQ==",
"devOptional": true,
"license": "Apache-2.0",
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -16630,7 +16602,6 @@
"resolved": "https://registry.npmjs.org/vite/-/vite-7.2.2.tgz",
"integrity": "sha512-BxAKBWmIbrDgrokdGZH1IgkIk/5mMHDreLDmCJ0qpyJaAteP8NvMhkwr/ZCQNqNH97bw/dANTE9PDzqwJghfMQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"esbuild": "^0.25.0",
"fdir": "^6.5.0",
@@ -16744,7 +16715,6 @@
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -16757,7 +16727,6 @@
"resolved": "https://registry.npmjs.org/vitest/-/vitest-3.2.4.tgz",
"integrity": "sha512-LUCP5ev3GURDysTWiP47wRRUpLKMOfPh+yKTx3kVIEiu5KOMeqzpnYNsKyOoVrULivR8tLcks4+lga33Whn90A==",
"license": "MIT",
"peer": true,
"dependencies": {
"@types/chai": "^5.2.2",
"@vitest/expect": "3.2.4",
@@ -17405,7 +17374,6 @@
"resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz",
"integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
@@ -17849,7 +17817,6 @@
"resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.14.3.tgz",
"integrity": "sha512-Iq8QQQ/7X3Sac15oB6p0FmUg/klxQvXLeileoqrTRGJYLV+/9tubbr9ipz0GKHjmXVsgFPo/+W+2cA8eNcR+XA==",
"license": "Apache-2.0",
"peer": true,
"dependencies": {
"@grpc/proto-loader": "^0.8.0",
"@js-sdsl/ordered-map": "^4.4.2"
@@ -17953,7 +17920,6 @@
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=12"
},
@@ -18013,7 +17979,6 @@
"dependencies": {
"@google/gemini-cli-core": "file:../core",
"@lydell/node-pty": "1.1.0",
"asciichart": "^1.5.25",
"strip-ansi": "^7.1.2",
"vitest": "^3.2.4"
},
-2
View File
@@ -51,8 +51,6 @@
"test:integration:all": "npm run test:integration:sandbox:none && npm run test:integration:sandbox:docker && npm run test:integration:sandbox:podman",
"test:integration:flaky": "cross-env RUN_FLAKY_INTEGRATION=1 npm run test:integration:sandbox:none",
"test:integration:sandbox:none": "cross-env GEMINI_SANDBOX=false vitest run --root ./integration-tests",
"test:memory": "vitest run --root ./memory-tests",
"test:memory:update-baselines": "cross-env UPDATE_MEMORY_BASELINES=true vitest run --root ./memory-tests",
"test:integration:sandbox:docker": "cross-env GEMINI_SANDBOX=docker npm run build:sandbox && cross-env GEMINI_SANDBOX=docker vitest run --root ./integration-tests",
"test:integration:sandbox:podman": "cross-env GEMINI_SANDBOX=podman vitest run --root ./integration-tests",
"lint": "cross-env NODE_OPTIONS=\"--max-old-space-size=8192\" eslint . --cache --max-warnings 0",
+126 -161
View File
@@ -108,8 +108,6 @@ export async function resolveExecutable(
let bashLanguage: Language | null = null;
let treeSitterInitialization: Promise<void> | null = null;
let treeSitterInitializationError: Error | null = null;
let parserState: 'uninitialized' | 'initializing' | 'initialized' | 'error' =
'uninitialized';
class ShellParserInitializationError extends Error {
constructor(cause: Error) {
@@ -165,37 +163,16 @@ async function loadBashLanguage(): Promise<void> {
}
export async function initializeShellParsers(): Promise<void> {
if (parserState === 'uninitialized') {
parserState = 'initializing';
let timerId: NodeJS.Timeout | undefined;
const timeoutPromise = new Promise<void>((_, reject) => {
timerId = setTimeout(
() => reject(new Error('Tree-sitter initialization timed out')),
30000,
);
if (!treeSitterInitialization) {
treeSitterInitialization = loadBashLanguage().catch((error) => {
treeSitterInitialization = null;
// Log the error but don't throw, allowing the application to fall back to safe defaults (ASK_USER)
// or regex checks where appropriate.
debugLogger.debug('Failed to initialize shell parsers:', error);
});
treeSitterInitialization = Promise.race([
loadBashLanguage(),
timeoutPromise,
])
.finally(() => {
if (timerId) clearTimeout(timerId);
})
.then(() => {
parserState = 'initialized';
})
.catch((error) => {
parserState = 'error';
treeSitterInitialization = null;
// Log the error but don't throw, allowing the application to fall back to safe defaults (ASK_USER)
// or regex checks where appropriate.
debugLogger.debug('Failed to initialize shell parsers:', error);
});
}
if (treeSitterInitialization) {
await treeSitterInitialization;
}
await treeSitterInitialization;
}
export interface ParsedCommandDetail {
@@ -870,40 +847,34 @@ export const spawnAsync = async (
const { program: finalCommand, args: finalArgs, env: finalEnv } = prepared;
try {
return await new Promise((resolve, reject) => {
const child = spawn(finalCommand, finalArgs, {
...options,
env: finalEnv,
});
let stdout = '';
let stderr = '';
child.stdout.on('data', (data) => {
stdout += data.toString();
});
child.stderr.on('data', (data) => {
stderr += data.toString();
});
child.on('close', (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(
new Error(`Command failed with exit code ${code}:\n${stderr}`),
);
}
});
child.on('error', (err) => {
reject(err);
});
return new Promise((resolve, reject) => {
const child = spawn(finalCommand, finalArgs, {
...options,
env: finalEnv,
});
} finally {
prepared.cleanup?.();
}
let stdout = '';
let stderr = '';
child.stdout.on('data', (data) => {
stdout += data.toString();
});
child.stderr.on('data', (data) => {
stderr += data.toString();
});
child.on('close', (code) => {
if (code === 0) {
resolve({ stdout, stderr });
} else {
reject(new Error(`Command failed with exit code ${code}:\n${stderr}`));
}
});
child.on('error', (err) => {
reject(err);
});
});
};
/**
@@ -931,115 +902,109 @@ export async function* execStreaming(
env: options?.env ?? process.env,
});
const { program: finalCommand, args: finalArgs, env: finalEnv } = prepared;
const child = spawn(finalCommand, finalArgs, {
...options,
env: finalEnv,
// ensure we don't open a window on windows if possible/relevant
windowsHide: true,
});
const rl = readline.createInterface({
input: child.stdout,
terminal: false,
});
const errorChunks: Buffer[] = [];
let stderrTotalBytes = 0;
const MAX_STDERR_BYTES = 20 * 1024; // 20KB limit
child.stderr.on('data', (chunk) => {
if (stderrTotalBytes < MAX_STDERR_BYTES) {
errorChunks.push(chunk);
stderrTotalBytes += chunk.length;
}
});
let error: Error | null = null;
child.on('error', (err) => {
error = err;
});
const onAbort = () => {
// If manually aborted by signal, we kill immediately.
if (!child.killed) child.kill();
};
if (options?.signal?.aborted) {
onAbort();
} else {
options?.signal?.addEventListener('abort', onAbort);
}
let finished = false;
try {
const { program: finalCommand, args: finalArgs, env: finalEnv } = prepared;
for await (const line of rl) {
if (options?.signal?.aborted) break;
yield line;
}
finished = true;
} finally {
rl.close();
options?.signal?.removeEventListener('abort', onAbort);
const child = spawn(finalCommand, finalArgs, {
...options,
env: finalEnv,
// ensure we don't open a window on windows if possible/relevant
windowsHide: true,
});
const rl = readline.createInterface({
input: child.stdout,
terminal: false,
});
const errorChunks: Buffer[] = [];
let stderrTotalBytes = 0;
const MAX_STDERR_BYTES = 20 * 1024; // 20KB limit
child.stderr.on('data', (chunk) => {
if (stderrTotalBytes < MAX_STDERR_BYTES) {
errorChunks.push(chunk);
stderrTotalBytes += chunk.length;
// Ensure process is killed when the generator is closed (consumer breaks loop)
let killedByGenerator = false;
if (!finished && child.exitCode === null && !child.killed) {
try {
child.kill();
} catch {
// ignore error if process is already dead
}
});
let error: Error | null = null;
child.on('error', (err) => {
error = err;
});
const onAbort = () => {
// If manually aborted by signal, we kill immediately.
if (!child.killed) child.kill();
};
if (options?.signal?.aborted) {
onAbort();
} else {
options?.signal?.addEventListener('abort', onAbort);
killedByGenerator = true;
}
let finished = false;
try {
for await (const line of rl) {
if (options?.signal?.aborted) break;
yield line;
}
finished = true;
} finally {
rl.close();
options?.signal?.removeEventListener('abort', onAbort);
// Ensure process is killed when the generator is closed (consumer breaks loop)
let killedByGenerator = false;
if (!finished && child.exitCode === null && !child.killed) {
try {
child.kill();
} catch {
// ignore error if process is already dead
}
killedByGenerator = true;
// Ensure we wait for the process to exit to check codes
await new Promise<void>((resolve, reject) => {
// If an error occurred before we got here (e.g. spawn failure), reject immediately.
if (error) {
reject(error);
return;
}
// Ensure we wait for the process to exit to check codes
await new Promise<void>((resolve, reject) => {
// If an error occurred before we got here (e.g. spawn failure), reject immediately.
if (error) {
reject(error);
function checkExit(code: number | null) {
// If we aborted or killed it manually, we treat it as success (stop waiting)
if (options?.signal?.aborted || killedByGenerator) {
resolve();
return;
}
function checkExit(code: number | null) {
// If we aborted or killed it manually, we treat it as success (stop waiting)
if (options?.signal?.aborted || killedByGenerator) {
resolve();
return;
}
const allowed = options?.allowedExitCodes ?? [0];
if (code !== null && allowed.includes(code)) {
resolve();
} else {
// If we have an accumulated error or explicit error event
if (error) reject(error);
else {
const stderr = Buffer.concat(errorChunks).toString('utf8');
const truncatedMsg =
stderrTotalBytes >= MAX_STDERR_BYTES ? '...[truncated]' : '';
reject(
new Error(
`Process exited with code ${code}: ${stderr}${truncatedMsg}`,
),
);
}
}
}
if (child.exitCode !== null) {
checkExit(child.exitCode);
const allowed = options?.allowedExitCodes ?? [0];
if (code !== null && allowed.includes(code)) {
resolve();
} else {
child.on('close', (code) => checkExit(code));
child.on('error', (err) => {
reject(err);
});
// If we have an accumulated error or explicit error event
if (error) reject(error);
else {
const stderr = Buffer.concat(errorChunks).toString('utf8');
const truncatedMsg =
stderrTotalBytes >= MAX_STDERR_BYTES ? '...[truncated]' : '';
reject(
new Error(
`Process exited with code ${code}: ${stderr}${truncatedMsg}`,
),
);
}
}
});
}
} finally {
prepared.cleanup?.();
}
if (child.exitCode !== null) {
checkExit(child.exitCode);
} else {
child.on('close', (code) => checkExit(code));
child.on('error', (err) => reject(err));
}
});
}
}
-1
View File
@@ -12,7 +12,6 @@
"dependencies": {
"@google/gemini-cli-core": "file:../core",
"@lydell/node-pty": "1.1.0",
"asciichart": "^1.5.25",
"strip-ansi": "^7.1.2",
"vitest": "^3.2.4"
},
-2
View File
@@ -6,8 +6,6 @@
export * from './file-system-test-helpers.js';
export * from './fixtures/agents.js';
export * from './memory-baselines.js';
export * from './memory-test-harness.js';
export * from './mock-utils.js';
export * from './test-mcp-server.js';
export * from './test-rig.js';
@@ -1,76 +0,0 @@
/**
* @license
* Copyright 2026 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { readFileSync, writeFileSync, existsSync } from 'node:fs';
/**
* Baseline entry for a single memory test scenario.
*/
export interface MemoryBaseline {
heapUsedBytes: number;
heapTotalBytes: number;
rssBytes: number;
timestamp: string;
}
/**
* Top-level structure of the baselines JSON file.
*/
export interface MemoryBaselineFile {
version: number;
updatedAt: string;
scenarios: Record<string, MemoryBaseline>;
}
/**
* Load baselines from a JSON file.
* Returns an empty baseline file if the file does not exist yet.
*/
export function loadBaselines(path: string): MemoryBaselineFile {
if (!existsSync(path)) {
return {
version: 1,
updatedAt: new Date().toISOString(),
scenarios: {},
};
}
const content = readFileSync(path, 'utf-8');
return JSON.parse(content) as MemoryBaselineFile;
}
/**
* Save baselines to a JSON file.
*/
export function saveBaselines(
path: string,
baselines: MemoryBaselineFile,
): void {
baselines.updatedAt = new Date().toISOString();
writeFileSync(path, JSON.stringify(baselines, null, 2) + '\n');
}
/**
* Update (or create) a single scenario baseline in the file.
*/
export function updateBaseline(
path: string,
scenarioName: string,
measured: {
heapUsedBytes: number;
heapTotalBytes: number;
rssBytes: number;
},
): void {
const baselines = loadBaselines(path);
baselines.scenarios[scenarioName] = {
heapUsedBytes: measured.heapUsedBytes,
heapTotalBytes: measured.heapTotalBytes,
rssBytes: measured.rssBytes,
timestamp: new Date().toISOString(),
};
saveBaselines(path, baselines);
}
@@ -1,483 +0,0 @@
/**
* @license
* Copyright 2026 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import v8 from 'node:v8';
import { setTimeout as sleep } from 'node:timers/promises';
import { loadBaselines, updateBaseline } from './memory-baselines.js';
import type { MemoryBaseline, MemoryBaselineFile } from './memory-baselines.js';
/** Configuration for asciichart plot function. */
interface PlotConfig {
height?: number;
format?: (x: number) => string;
}
/** Type for the asciichart plot function. */
type PlotFn = (series: number[], config?: PlotConfig) => string;
/**
* A single memory snapshot at a point in time.
*/
export interface MemorySnapshot {
timestamp: number;
label: string;
heapUsed: number;
heapTotal: number;
rss: number;
external: number;
arrayBuffers: number;
heapSizeLimit: number;
heapSpaces: any[];
}
/**
* Result from running a memory test scenario.
*/
export interface MemoryTestResult {
scenarioName: string;
snapshots: MemorySnapshot[];
peakHeapUsed: number;
peakRss: number;
finalHeapUsed: number;
finalRss: number;
baseline: MemoryBaseline | undefined;
withinTolerance: boolean;
deltaPercent: number;
}
/**
* Options for the MemoryTestHarness.
*/
export interface MemoryTestHarnessOptions {
/** Path to the baselines JSON file */
baselinesPath: string;
/** Default tolerance percentage (0-100). Default: 10 */
defaultTolerancePercent?: number;
/** Number of GC cycles to run before each snapshot. Default: 3 */
gcCycles?: number;
/** Delay in ms between GC cycles. Default: 100 */
gcDelayMs?: number;
/** Number of samples to take for median calculation. Default: 3 */
sampleCount?: number;
/** Pause in ms between samples. Default: 50 */
samplePauseMs?: number;
}
/**
* MemoryTestHarness provides infrastructure for running memory usage tests.
*
* It handles:
* - Forcing V8 garbage collection to reduce noise
* - Taking V8 heap snapshots for accurate memory measurement
* - Comparing against baselines with configurable tolerance
* - Generating ASCII chart reports of memory trends
*/
export class MemoryTestHarness {
private baselines: MemoryBaselineFile;
private readonly baselinesPath: string;
private readonly defaultTolerancePercent: number;
private readonly gcCycles: number;
private readonly gcDelayMs: number;
private readonly sampleCount: number;
private readonly samplePauseMs: number;
private allResults: MemoryTestResult[] = [];
constructor(options: MemoryTestHarnessOptions) {
this.baselinesPath = options.baselinesPath;
this.defaultTolerancePercent = options.defaultTolerancePercent ?? 10;
this.gcCycles = options.gcCycles ?? 3;
this.gcDelayMs = options.gcDelayMs ?? 100;
this.sampleCount = options.sampleCount ?? 3;
this.samplePauseMs = options.samplePauseMs ?? 50;
this.baselines = loadBaselines(this.baselinesPath);
}
/**
* Force garbage collection multiple times and take a V8 heap snapshot.
* Forces GC multiple times with delays to allow weak references and
* FinalizationRegistry callbacks to run, reducing measurement noise.
*/
async takeSnapshot(label: string = 'snapshot'): Promise<MemorySnapshot> {
await this.forceGC();
const memUsage = process.memoryUsage();
const heapStats = v8.getHeapStatistics();
return {
timestamp: Date.now(),
label,
heapUsed: memUsage.heapUsed,
heapTotal: memUsage.heapTotal,
rss: memUsage.rss,
external: memUsage.external,
arrayBuffers: memUsage.arrayBuffers,
heapSizeLimit: heapStats.heap_size_limit,
heapSpaces: v8.getHeapSpaceStatistics(),
};
}
/**
* Take multiple snapshot samples and return the median to reduce noise.
*/
async takeMedianSnapshot(
label: string = 'median',
count?: number,
): Promise<MemorySnapshot> {
const samples: MemorySnapshot[] = [];
const numSamples = count ?? this.sampleCount;
for (let i = 0; i < numSamples; i++) {
samples.push(await this.takeSnapshot(`${label}_sample_${i}`));
if (i < numSamples - 1) {
await sleep(this.samplePauseMs);
}
}
// Sort by heapUsed and take the median
samples.sort((a, b) => a.heapUsed - b.heapUsed);
const medianIdx = Math.floor(samples.length / 2);
const median = samples[medianIdx]!;
return {
...median,
label,
timestamp: Date.now(),
};
}
/**
* Run a memory test scenario.
*
* Takes before/after snapshots around the scenario function, collects
* intermediate snapshots if the scenario provides them, and compares
* the result against the stored baseline.
*
* @param name - Scenario name (must match baseline key)
* @param fn - Async function that executes the scenario. Receives a
* `recordSnapshot` callback for recording intermediate snapshots.
* @param tolerancePercent - Override default tolerance for this scenario
*/
async runScenario(
name: string,
fn: (
recordSnapshot: (label: string) => Promise<MemorySnapshot>,
) => Promise<void>,
tolerancePercent?: number,
): Promise<MemoryTestResult> {
const tolerance = tolerancePercent ?? this.defaultTolerancePercent;
const snapshots: MemorySnapshot[] = [];
// Record a callback for intermediate snapshots
const recordSnapshot = async (label: string): Promise<MemorySnapshot> => {
const snap = await this.takeMedianSnapshot(label);
snapshots.push(snap);
return snap;
};
// Before snapshot
const beforeSnap = await this.takeMedianSnapshot('before');
snapshots.push(beforeSnap);
// Run the scenario
await fn(recordSnapshot);
// After snapshot (median of multiple samples)
const afterSnap = await this.takeMedianSnapshot('after');
snapshots.push(afterSnap);
// Calculate peak values
const peakHeapUsed = Math.max(...snapshots.map((s) => s.heapUsed));
const peakRss = Math.max(...snapshots.map((s) => s.rss));
// Get baseline
const baseline = this.baselines.scenarios[name];
// Determine if within tolerance
let deltaPercent = 0;
let withinTolerance = true;
if (baseline) {
deltaPercent =
((afterSnap.heapUsed - baseline.heapUsedBytes) /
baseline.heapUsedBytes) *
100;
withinTolerance = deltaPercent <= tolerance;
}
const result: MemoryTestResult = {
scenarioName: name,
snapshots,
peakHeapUsed,
peakRss,
finalHeapUsed: afterSnap.heapUsed,
finalRss: afterSnap.rss,
baseline,
withinTolerance,
deltaPercent,
};
this.allResults.push(result);
return result;
}
/**
* Assert that a scenario result is within the baseline tolerance.
* Throws an assertion error with details if it exceeds the threshold.
*/
assertWithinBaseline(
result: MemoryTestResult,
tolerancePercent?: number,
): void {
const tolerance = tolerancePercent ?? this.defaultTolerancePercent;
if (!result.baseline) {
console.warn(
`⚠ No baseline found for "${result.scenarioName}". ` +
`Run with UPDATE_MEMORY_BASELINES=true to create one. ` +
`Measured: ${formatMB(result.finalHeapUsed)} heap used.`,
);
return; // Don't fail if no baseline exists yet
}
const deltaPercent =
((result.finalHeapUsed - result.baseline.heapUsedBytes) /
result.baseline.heapUsedBytes) *
100;
if (deltaPercent > tolerance) {
throw new Error(
`Memory regression detected for "${result.scenarioName}"!\n` +
` Measured: ${formatMB(result.finalHeapUsed)} heap used\n` +
` Baseline: ${formatMB(result.baseline.heapUsedBytes)} heap used\n` +
` Delta: ${deltaPercent.toFixed(1)}% (tolerance: ${tolerance}%)\n` +
` Peak heap: ${formatMB(result.peakHeapUsed)}\n` +
` Peak RSS: ${formatMB(result.peakRss)}`,
);
}
}
/**
* Update the baseline for a scenario with the current measured values.
*/
updateScenarioBaseline(result: MemoryTestResult): void {
updateBaseline(this.baselinesPath, result.scenarioName, {
heapUsedBytes: result.finalHeapUsed,
heapTotalBytes:
result.snapshots[result.snapshots.length - 1]?.heapTotal ?? 0,
rssBytes: result.finalRss,
});
// Reload baselines after update
this.baselines = loadBaselines(this.baselinesPath);
}
/**
* Analyze snapshots to detect sustained leaks across 3 snapshots.
* A leak is flagged if growth is observed in both phases for any heap space.
*/
analyzeSnapshots(
snapshots: MemorySnapshot[],
thresholdBytes: number = 1024 * 1024, // 1 MB
): { leaked: boolean; message: string } {
if (snapshots.length < 3) {
return { leaked: false, message: 'Not enough snapshots to analyze' };
}
const snap1 = snapshots[snapshots.length - 3];
const snap2 = snapshots[snapshots.length - 2];
const snap3 = snapshots[snapshots.length - 1];
if (!snap1 || !snap2 || !snap3) {
return { leaked: false, message: 'Missing snapshots' };
}
const spaceNames = new Set<string>();
snap1.heapSpaces.forEach((s: any) => spaceNames.add(s.space_name));
snap2.heapSpaces.forEach((s: any) => spaceNames.add(s.space_name));
snap3.heapSpaces.forEach((s: any) => spaceNames.add(s.space_name));
let hasSustainedGrowth = false;
const growthDetails: string[] = [];
for (const name of spaceNames) {
const size1 =
snap1.heapSpaces.find((s: any) => s.space_name === name)
?.space_used_size ?? 0;
const size2 =
snap2.heapSpaces.find((s: any) => s.space_name === name)
?.space_used_size ?? 0;
const size3 =
snap3.heapSpaces.find((s: any) => s.space_name === name)
?.space_used_size ?? 0;
const growth1 = size2 - size1;
const growth2 = size3 - size2;
if (growth1 > thresholdBytes && growth2 > thresholdBytes) {
hasSustainedGrowth = true;
growthDetails.push(
`${name}: sustained growth (${formatMB(growth1)} -> ${formatMB(growth2)})`,
);
}
}
let message = '';
if (hasSustainedGrowth) {
message =
`Memory bloat detected in heap spaces:\n ` +
growthDetails.join('\n ');
} else {
message = `No sustained growth detected in any heap space above threshold.`;
}
return { leaked: hasSustainedGrowth, message };
}
/**
* Assert that memory returns to a baseline level after a peak.
* Useful for verifying that large tool outputs are not retained.
*/
assertMemoryReturnsToBaseline(
snapshots: MemorySnapshot[],
tolerancePercent: number = 10,
): void {
if (snapshots.length < 3) {
throw new Error('Need at least 3 snapshots to check return to baseline');
}
const baseline = snapshots[0]; // Assume first is baseline
const peak = snapshots.reduce(
(max, s) => (s.heapUsed > max.heapUsed ? s : max),
snapshots[0],
);
const final = snapshots[snapshots.length - 1];
if (!baseline || !peak || !final) {
throw new Error('Missing snapshots for return to baseline check');
}
const tolerance = baseline.heapUsed * (tolerancePercent / 100);
const delta = final.heapUsed - baseline.heapUsed;
if (delta > tolerance) {
throw new Error(
`Memory did not return to baseline!\n` +
` Baseline: ${formatMB(baseline.heapUsed)}\n` +
` Peak: ${formatMB(peak.heapUsed)}\n` +
` Final: ${formatMB(final.heapUsed)}\n` +
` Delta: ${formatMB(delta)} (tolerance: ${formatMB(tolerance)})`,
);
}
}
/**
* Generate a report with ASCII charts and summary table.
* Uses the `asciichart` library for terminal visualization.
*/
async generateReport(results?: MemoryTestResult[]): Promise<string> {
const resultsToReport = results ?? this.allResults;
const lines: string[] = [];
lines.push('');
lines.push('═══════════════════════════════════════════════════');
lines.push(' MEMORY USAGE TEST REPORT');
lines.push('═══════════════════════════════════════════════════');
lines.push('');
for (const result of resultsToReport) {
const measured = formatMB(result.finalHeapUsed);
const baseline = result.baseline
? formatMB(result.baseline.heapUsedBytes)
: 'N/A';
const delta = result.baseline
? `${result.deltaPercent >= 0 ? '+' : ''}${result.deltaPercent.toFixed(1)}%`
: 'N/A';
const status = !result.baseline
? 'NEW'
: result.withinTolerance
? '✅'
: '❌';
lines.push(
`${result.scenarioName}: ${measured} (Baseline: ${baseline}, Delta: ${delta}) ${status}`,
);
}
lines.push('');
// Generate ASCII chart for each scenario with multiple snapshots
try {
// @ts-expect-error - asciichart may not have types
const asciichart = (await import('asciichart')) as {
default?: { plot?: PlotFn };
plot?: PlotFn;
};
const plot: PlotFn | undefined =
asciichart.default?.plot ?? asciichart.plot;
for (const result of resultsToReport) {
if (result.snapshots.length > 2) {
lines.push(`📈 Memory trend: ${result.scenarioName}`);
lines.push('─'.repeat(60));
const heapDataMB = result.snapshots.map(
(s) => s.heapUsed / (1024 * 1024),
);
if (plot) {
const chart = plot(heapDataMB, {
height: 10,
format: (x: number) => `${x.toFixed(1)} MB`.padStart(10),
});
lines.push(chart);
}
// Label the x-axis with snapshot labels
const labels = result.snapshots.map((s) => s.label);
lines.push(' ' + labels.join(' → '));
lines.push('');
}
}
} catch {
lines.push(
'(asciichart not available — install with: npm install --save-dev asciichart)',
);
lines.push('');
}
lines.push('═══════════════════════════════════════════════════');
lines.push('');
const report = lines.join('\n');
console.log(report);
return report;
}
/**
* Force V8 garbage collection.
* Runs multiple GC cycles with delays to allow weak references
* and FinalizationRegistry callbacks to run.
*/
private async forceGC(): Promise<void> {
if (typeof globalThis.gc !== 'function') {
throw new Error(
'global.gc() not available. Run with --expose-gc for accurate measurements.',
);
}
for (let i = 0; i < this.gcCycles; i++) {
globalThis.gc();
if (i < this.gcCycles - 1) {
await sleep(this.gcDelayMs);
}
}
}
}
/**
* Format bytes as a human-readable MB string.
*/
function formatMB(bytes: number): string {
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
}