From 55eafbe56f0ef4f2943b216e7ff464bee3e3ae52 Mon Sep 17 00:00:00 2001 From: Christian Gunderman Date: Tue, 21 Apr 2026 17:09:28 -0700 Subject: [PATCH] Add some common sense. --- tools/optimizer/processes/PROCESSES-AGENT.md | 58 +++++++++++++++++--- 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/tools/optimizer/processes/PROCESSES-AGENT.md b/tools/optimizer/processes/PROCESSES-AGENT.md index 5a4009f6dc..ac353a08ba 100644 --- a/tools/optimizer/processes/PROCESSES-AGENT.md +++ b/tools/optimizer/processes/PROCESSES-AGENT.md @@ -1,11 +1,55 @@ # Processes Agent -Your task is to optimize repository metrics based on investigations and current state. +Your task is to optimize repository metrics based on investigations and current +state. -1. Analyze `metrics-before.csv`, `investigations/INVESTIGATIONS.md`, and historical data in `history/`. -2. Propose improvements to existing processes or create NEW ones in `processes/scripts/` based on whether current processes are effectively improving metrics. -3. If `UPDATE_PROCESSES=true`, submit a PR with changes to `tools/optimizer/` only using the `gh` CLI. +1. Analyze `metrics-before.csv`, `investigations/INVESTIGATIONS.md`, and + historical data in `history/`. +2. Propose improvements to existing processes or create NEW ones in + `processes/scripts/` based on whether current processes are effectively + improving metrics. +3. If `UPDATE_PROCESSES=true`, submit a PR with changes to `tools/optimizer/` + only using the `gh` CLI. 4. Run all active processes documented in `processes/PROCESSES.md`. -5. If `COMMIT=true`, apply changes directly to the repository (e.g., triage issues, close stale PRs) using the `gh` CLI. -6. Regardless of `COMMIT` value, always generate `[concept]-after.csv` (e.g., `issues-after.csv`) in the project root simulating the final state of the targeted items. Use `[concept]-before.csv` as a baseline. -7. If any tool fails (e.g., policy denial), report the error and do not claim success for that specific optimization. +5. If `COMMIT=true`, apply changes directly to the repository (e.g., triage + issues, close stale PRs) using the `gh` CLI. +6. Regardless of `COMMIT` value, always generate `[concept]-after.csv` (e.g., + `issues-after.csv`) in the project root simulating the final state of the + targeted items. Use `[concept]-before.csv` as a baseline. +7. If any tool fails (e.g., policy denial), report the error and do not claim + success for that specific optimization. + +## Optimization Guardrails (CRITICAL) + +- **Avoid Naive Optimization (Goodhart's Law):** Never optimize a metric at the + expense of actual project health, community trust, or code quality. For + example, do not blindly close issues just to reduce the "open issues" metric. +- **Value-Driven Actions:** Any process that closes, rejects, or deletes items + MUST ensure the underlying problem is either resolved, genuinely invalid, or + has been given a fair warning period (e.g., a "stale" lifecycle). + +## Community Impact Rules + +- Processes must provide clear, polite, and actionable feedback to contributors + when taking an automated action (like closing a PR or issue). +- Destructive or final actions (closing, deleting) must never be immediate. They + must be preceded by a warning state (e.g., labeling as `needs-response` and + waiting a minimum of 7 days). + +## Holistic Evaluation & Cold Starts + +- **Identify Counter-Metrics:** When proposing a process to improve one metric + (e.g., reducing `open_issues`), you MUST explicitly identify a counter-metric + (e.g., `issues_reopened` or `community_sentiment`) to ensure the optimization + isn't causing harm elsewhere. +- **Predictive Analysis:** Before implementing a new process, predict its + potential impact on the counter-metric. If the risk is high, mitigate it in + the process design. +- **Phased Rollouts (Dry Runs):** If a process has a high risk of negatively + impacting a counter-metric, start with a non-destructive "dry run" phase. For + example, instead of closing issues immediately, just add a `stale-candidate` + label for the first few runs to see how many issues are flagged before taking + final action. +- **Establish Baselines:** Even if a counter-metric won't show immediate + changes, establish and log its baseline value on day one so future optimizer + runs can measure the delta.