Improve mitigations agents.

This commit is contained in:
Christian Gunderman
2026-04-22 10:42:31 -07:00
parent 80ffb7dafd
commit 5e24d6285e
3 changed files with 42 additions and 21 deletions
+23 -8
View File
@@ -2,7 +2,7 @@ import { Command } from 'commander';
import fs from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { execSync } from 'node:child_process';
import { execSync, spawn } from 'node:child_process';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
@@ -11,6 +11,7 @@ async function main() {
program
.option('--investigate', 'Run investigation phase', false)
.option('--update-processes', 'Update processes based on learnings', false)
.option('--create-pr', 'Create a PR when updating processes', false)
.option('--commit', 'Run processes and commit changes', false)
.parse(process.argv);
@@ -86,9 +87,10 @@ async function runPhase(phaseDir: string, env: Record<string, string>, options:
}
const instructionsPath = path.join(phasePath, promptFile);
const instructionsContent = await fs.readFile(instructionsPath, 'utf8');
const envString = Object.entries(env).map(([k, v]) => `${k}=${v}`).join('\n');
const userPrompt = `Execution Context:\n${envString}\n\nPlease proceed with the ${phaseDir} tasks as defined in your instructions. Always output CSV files as requested.`;
const userPrompt = `Execution Context:\n${envString}\n\n${instructionsContent}\n\nPlease proceed with the ${phaseDir} tasks as defined in your instructions. Always output CSV files as requested.`;
console.log(`Running agent with prompt: ${promptFile}`);
@@ -98,14 +100,27 @@ async function runPhase(phaseDir: string, env: Record<string, string>, options:
try {
// Run GCLI non-interactively with --yolo to bypass policies
const cliPath = path.join(rootDir, 'packages', 'cli');
const command = `node ${cliPath} --prompt "${userPrompt}" --instructions-path "${instructionsPath}" --yolo`;
execSync(command, {
stdio: 'inherit',
cwd: rootDir,
env: { ...process.env, ...env }
await new Promise<void>((resolve, reject) => {
const child = spawn('node', [cliPath, '--prompt', userPrompt, '--yolo', '--model', 'gemini-3-flash-preview'], {
stdio: 'inherit',
cwd: rootDir,
env: { ...process.env, ...env }
});
child.on('close', (code) => {
if (code === 0) {
resolve();
} else {
reject(new Error(`Exit code ${code}`));
}
});
child.on('error', (err) => {
reject(err);
});
});
} catch (err) {
} catch (err: any) {
console.error(`Error in phase ${phaseDir}:`, err.message);
}
@@ -9,10 +9,11 @@ of the data and identify any opportunities for improvement.
2. Run existing scripts in `investigations/scripts/` to gather more data.
3. If necessary, create NEW investigation scripts in `investigations/scripts/`
to dig deeper (e.g., check issue labels, age, or assignees).
4. Maintain a table of all available investigation scripts in
4. **Hypothesis Testing**: For each metric not meeting goals:
- **Develop Competing Hypotheses**: Brainstorm multiple potential root causes (e.g., "Latency is due to slow reviews" vs. "Latency is due to slow author responses").
- **Gather Evidence**: Use or create scripts to collect data that supports or refutes EACH hypothesis (e.g., check timestamp of last review vs. last commit).
- **Select Root Cause**: Identify the hypothesis most strongly supported by the data.
5. **Output Actionable Data**: Write specific targets for optimization to CSV files (e.g., `reviewer_bottlenecks.csv`, `author_stale_prs.csv`). These files MUST contain identifiers and the specific reason (evidence) for targeting.
6. Maintain a table of all available investigation scripts in
`investigations/INVESTIGATIONS.md`.
5. Write any gathered data to corresponding CSV files. This data will be passed
along to and consumed by the Processes Agent
(`processes/PROCESSES-AGENT.md`).
6. Document your findings in `investigations/INVESTIGATIONS.md`, noting if
metrics are improving or worsening.
7. Document your hypotheses, the data gathered for each, and your final conclusion in `investigations/INVESTIGATIONS.md`.
+12 -7
View File
@@ -2,10 +2,15 @@
Your task is to optimize repository metrics based on investigations and current state.
1. Analyze `metrics-before.csv`, `investigations/INVESTIGATIONS.md`, and historical data in `history/`.
2. Propose improvements to existing processes or create NEW ones in `processes/scripts/` based on whether current processes are effectively improving metrics.
3. If `UPDATE_PROCESSES=true`, submit a PR with changes to `tools/optimizer/` only using the `gh` CLI.
4. Run all active processes documented in `processes/PROCESSES.md`.
5. If `COMMIT=true`, apply changes directly to the repository (e.g., triage issues, close stale PRs) using the `gh` CLI.
6. Regardless of `COMMIT` value, always generate `[concept]-after.csv` (e.g., `issues-after.csv`) in the project root simulating the final state of the targeted items. Use `[concept]-before.csv` as a baseline.
7. If any tool fails (e.g., policy denial), report the error and do not claim success for that specific optimization.
1. Analyze `metrics-before.csv`, `investigations/INVESTIGATIONS.md`, and any actionable target files (e.g., `reviewer_bottlenecks.csv`) produced by the Investigations Agent.
2. **Targeted Mitigation**: Ensure your proposed improvements or new scripts in `processes/scripts/` directly address the *confirmed* root cause.
3. **Professional Communication & Empathy**: All automated communications MUST be friendly, professional, and appreciative.
- **Always thank the contributor** for their work or for reporting an issue.
- **Tone**: Maintain a helpful and collaborative tone. Avoid blunt or dismissive language (e.g., "low signal", "non-actionable").
- **Clarification First**: NEVER close an item without first politely requesting the specific missing details and allowing at least 7 days for a response.
- **Example**: "Thank you for your contribution to Gemini CLI! We are interested in resolving this, but we need a bit more information to take action. Could you please provide [specific details]? If we don't hear back in 7 days, we'll close this for now, but you're welcome to reopen it once the info is available."
4. **Safety & Idempotency**: Ensure scripts are safe to run multiple times. They should check for existing states (e.g., "Has a reminder already been sent in the last 24h?") before acting.
5. If `UPDATE_PROCESSES=true`, apply changes to `tools/optimizer/` locally. If `CREATE_PR=true` is also provided, submit a PR with these changes using the `gh` CLI. Otherwise, DO NOT submit a PR.
6. Run all active processes documented in `processes/PROCESSES.md`.
7. **Mandatory Simulation**: Regardless of `COMMIT` value, always generate `[concept]-after.csv` (e.g., `issues-after.csv`) in the project root simulating the final state. Use `[concept]-before.csv` as a baseline.
8. If any tool fails (e.g., policy denial), report the error and do not claim success for that specific optimization.