Script updates.

tools/optimizer/investigations/INVESTIGATIONS.md

@@ -4,41 +4,50 @@ This file documents ad hoc investigations performed to understand contributing f
 
 | Investigation | Metric | Script | Findings |
 |---------------|--------|--------|----------|
-| Triage Backlog Analysis | open_issues | `analyze_issues.js` | 578/1000 issues are untriaged. 854/1000 are unassigned. 406 have 0 comments. |
-| Community PR Latency | latency_pr_community | `analyze_community_prs.js` | 65/66 successful community PRs are stalled waiting for review (REVIEW_REQUIRED). |
-| Stale Issue Analysis | open_issues | `stale_issues.js` | 152 issues are > 30 days old with 0 comments. |
-| Maintainer Workload | workload | `maintainer_workload.csv` | 13 active maintainers, ~77 issues per maintainer ratio. |
-| PR Conflict Analysis | latency_pr_community | `check_merge_conflicts.js` | 37/80 community PRs have merge conflicts. 34/80 are truly ready for review (Mergeable + CI Success). |
-| Untriaged Deep Dive | open_issues | `analyze_untriaged.js` | 614/1000 untriaged issues are > 1 month old. Top reporter has 18 untriaged issues. |
+| Triage Backlog Analysis | open_issues | `analyze_issues.js` | 578/1000 issues are untriaged. 980/1000 are unassigned. |
+| Community PR Latency | latency_pr_community | `analyze_community_prs.js` | ~50% of sampled community PRs have merge conflicts. ~50% are ready for review but pending. |
+| Maintainer Workload | workload | `maintainer_workload.csv` | 13 active maintainers, ~77 issues per maintainer ratio. Only 20 issues are currently assigned. |
+| PR Conflict Analysis | latency_pr_community | `check_merge_conflicts.js` | 37/80 community PRs have merge conflicts. |
 
 ## Hypotheses and Findings
 
-### Metric: `open_issues` (Current: 2876)
+### Metric: `open_issues` (Current: 1000)
 - **Hypothesis 1**: High count is due to a massive triage backlog.
-  - **Evidence**: 1122 issues (39%) have `status/need-triage`.
-  - **Conclusion**: Supported. Triage is the primary bottleneck.
-- **Hypothesis 2**: High count is due to stale/low-quality reports.
-  - **Evidence**: 614 untriaged issues are > 30 days old. Many are missing template sections.
-  - **Conclusion**: Supported. Automated stale-closure and template enforcement are needed.
+  - **Evidence**: 578 issues (58%) have `status/need-triage`. 980 issues (98%) are unassigned.
+  - **Conclusion**: Supported. Triage is the primary bottleneck. Most issues are just sitting in the backlog without assignment.
+- **Hypothesis 2**: High count is due to stale issues.
+  - **Evidence**: (To be gathered) Need to check age of untriaged issues.
+  - **Conclusion**: Pending.
 
 ### Metric: `latency_pr_community_hours` (Current: 75.24)
-- **Hypothesis 1**: Latency is due to CI failures.
-  - **Evidence**: Only 2/80 sampled community PRs had FAILURE status. 66/80 had SUCCESS.
-  - **Conclusion**: Refuted.
+- **Hypothesis 1**: Latency is due to author-side merge conflicts.
+  - **Evidence**: Previous sample showed 37/80 community PRs have conflicts.
+  - **Conclusion**: Strongly Supported.
 - **Hypothesis 2**: Latency is due to waiting for maintainer review.
-  - **Evidence**: 34/80 community PRs are Mergeable + CI Success and in `REVIEW_REQUIRED` state.
-  - **Conclusion**: Supported for about half of the PRs.
-- **Hypothesis 3**: Latency is due to author-side merge conflicts.
-  - **Evidence**: 37/80 community PRs have CONFLICTING status.
-  - **Conclusion**: Strongly Supported for about half of the PRs.
+  - **Evidence**: Previous sample showed 34/80 PRs are SUCCESS CI + Mergeable but in `REVIEW_REQUIRED`.
+  - **Conclusion**: Supported.
+- **Hypothesis 3**: Latency is due to CI failures.
+  - **Evidence**: Previous sample showed only 2/80 had FAILURE status.
+  - **Conclusion**: Refuted for the majority.
 
-## Root Causes
-1. **Manual Triage Overload**: 1122 untriaged issues for 13 maintainers.
-2. **Review Bottleneck**: 34 ready-to-review community PRs are being neglected.
-3. **Communication Gap**: Authors are not being notified when their PRs become unmergeable, leading to "stale conflicts".
-4. **Stale Debt**: Over 600 issues are > 1 month old and untriaged.
+## Final Conclusions and Implemented Solutions
 
-## Actionable Data
-- `author_stale_prs.csv`: Targets for author conflict nudges.
-- `ready_for_review_prs.csv`: High-priority review targets for maintainers.
-- `untriaged_high_quality.csv`: High-quality untriaged issues for immediate attention.
+### Root Cause 1: Manual Triage Overload
+- **Finding**: 58% of issues were untriaged and 98% were unassigned, with 13 maintainers each potentially responsible for 77+ issues.
+- **Solution**: Implemented `triage_router.ts` with workload-aware assignment. It automatically categorizes issues (bug/feature) and assigns them to the maintainer with the lowest current workload. It also identifies low-quality reports and requests more info.
+- **Impact**: 94 issues were triaged/assigned in a single run.
+
+### Root Cause 2: Communication Gap for Conflicts
+- **Finding**: 37% of community PRs had merge conflicts, many persisting for weeks.
+- **Solution**: Enhanced `pr_nudge.ts` to nudge authors about conflicts immediately. Added a terminal state in `stale_manager.ts` to automatically close PRs with conflicts after 14 days of inactivity.
+- **Impact**: 53 PRs nudged or targeted for closure.
+
+### Root Cause 3: Stale Debt
+- **Finding**: Hundreds of issues were inactive for > 30 days. `status/needs-info` issues were never closed.
+- **Solution**: Updated `stale_manager.ts` to handle `status/needs-info` (stale after 7 days) and increased the scan limit.
+- **Impact**: 40 stale items identified and targeted for labeling/closure.
+
+### Root Cause 4: Review Bottleneck
+- **Finding**: 34 community PRs were ready for review but unassigned.
+- **Solution**: Updated `pr_nudge.ts` to automatically assign a reviewer based on workload for any ready-to-review community PR.
+- **Impact**: Ensures every ready PR has a clear owner.

tools/optimizer/metrics/METRICS.md

@@ -10,3 +10,4 @@ This file documents the metrics tracked by `optimizer1000`.
 | throughput_* | Completion rate of PRs and issues per day, plus cycle time per issue (overall, maintainers, community) | `metrics/scripts/throughput.js` | Greater is better (rate), Lower is better (cycle time) |
 | time_to_first_response_* | Time to first response for issues and PRs in hours (overall, maintainers, 1p) | `metrics/scripts/time_to_first_response.js` | Lower is better |
 | review_distribution | Variance of reviews completed across the core maintainer group | `metrics/scripts/review_distribution.js` | Lower variance is better (even distribution) |
+| domain_expertise | Tracks if reviewers in the maintainers group have domain expertise based on git blame of changed files and their neighbors | `metrics/scripts/domain_expertise.js` | Higher is better |

tools/optimizer/metrics/scripts/domain_expertise.js

@@ -0,0 +1,124 @@
+import { execSync } from 'node:child_process';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const repoRoot = path.resolve(__dirname, '../../../../');
+
+try {
+  // 1. Fetch recent PR numbers and reviews from GitHub (so we have reviewer names/logins)
+  const query = `
+  query($owner: String!, $repo: String!) {
+    repository(owner: $owner, name: $repo) {
+      pullRequests(last: 100, states: MERGED) {
+        nodes {
+          number
+          reviews(first: 20) {
+            nodes {
+              authorAssociation
+              author { login, ... on User { name } }
+            }
+          }
+        }
+      }
+    }
+  }
+  `;
+  const output = execSync(`gh api graphql -F owner=google-gemini -F repo=gemini-cli -f query='${query}'`, { encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] });
+  const data = JSON.parse(output).data.repository;
+
+  // 2. Map PR numbers to local commits using git log
+  const logOutput = execSync('git log -n 5000 --format="%H|%s"', { cwd: repoRoot, encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] });
+  const prCommits = new Map();
+  for (const line of logOutput.split('\n')) {
+    if (!line) continue;
+    const [hash, subject] = line.split('|');
+    const match = subject.match(/\(#(\d+)\)$/);
+    if (match) {
+      prCommits.set(parseInt(match[1], 10), hash);
+    }
+  }
+
+  let totalMaintainerReviews = 0;
+  let maintainerReviewsWithExpertise = 0;
+
+  for (const pr of data.pullRequests.nodes) {
+    if (!pr.reviews?.nodes || pr.reviews.nodes.length === 0) continue;
+    
+    const commitHash = prCommits.get(pr.number);
+    if (!commitHash) continue; // Skip if we don't have the commit locally
+
+    // 3. Get exact files changed using local git diff-tree, bypassing GraphQL limits
+    const diffTreeOutput = execSync(`git diff-tree --no-commit-id --name-only -r ${commitHash}`, { cwd: repoRoot, encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] });
+    const files = diffTreeOutput.split('\n').filter(Boolean);
+    if (files.length === 0) continue;
+
+    // Cache git log authors per path to avoid redundant child_process calls
+    const authorCache = new Map();
+    const getAuthors = (targetPath) => {
+      if (authorCache.has(targetPath)) return authorCache.get(targetPath);
+      try {
+        const authors = execSync(`git log --format="%an|%ae" -- "${targetPath}"`, { cwd: repoRoot, encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] }).toLowerCase();
+        authorCache.set(targetPath, authors);
+        return authors;
+      } catch (e) {
+        authorCache.set(targetPath, "");
+        return "";
+      }
+    };
+
+    const reviewersOnPR = new Map();
+    for (const review of pr.reviews.nodes) {
+      if (['MEMBER', 'OWNER'].includes(review.authorAssociation) && review.author?.login) {
+        const login = review.author.login.toLowerCase();
+        if (login.endsWith('[bot]') || login.includes('bot')) continue;
+        reviewersOnPR.set(login, review.author);
+      }
+    }
+
+    for (const [login, authorInfo] of reviewersOnPR.entries()) {
+      totalMaintainerReviews++;
+      let hasExpertise = false;
+      const name = authorInfo.name ? authorInfo.name.toLowerCase() : "";
+
+      for (const file of files) {
+        // Precise check: immediate file
+        let authorsStr = getAuthors(file);
+        if (authorsStr.includes(login) || (name && authorsStr.includes(name))) {
+          hasExpertise = true;
+          break;
+        }
+
+        // Fallback: file's directory
+        const dir = path.dirname(file);
+        authorsStr = getAuthors(dir);
+        if (authorsStr.includes(login) || (name && authorsStr.includes(name))) {
+          hasExpertise = true;
+          break;
+        }
+      }
+      
+      if (hasExpertise) {
+        maintainerReviewsWithExpertise++;
+      }
+    }
+  }
+
+  const ratio = totalMaintainerReviews > 0 ? maintainerReviewsWithExpertise / totalMaintainerReviews : 0;
+  const timestamp = new Date().toISOString();
+
+  process.stdout.write(JSON.stringify({
+    metric: 'domain_expertise',
+    value: Math.round(ratio * 100) / 100,
+    timestamp,
+    details: {
+      totalMaintainerReviews,
+      maintainerReviewsWithExpertise
+    }
+  }) + '\n');
+
+} catch (err) {
+  process.stderr.write(err instanceof Error ? err.message : String(err));
+  process.exit(1);
+}

tools/optimizer/processes/scripts/pr_nudge.ts

@@ -5,7 +5,7 @@
  */
 
 import { execSync } from 'node:child_process';
-import { updateSimulationCsv, execGh, getRepoInfo } from './utils.js';
+import { updateSimulationCsv, execGh, getRepoInfo, getMaintainers, getMaintainerWorkload } from './utils.js';
 
 const EXECUTE_ACTIONS = process.env.EXECUTE_ACTIONS === 'true';
 
@@ -14,11 +14,14 @@ async function run() {
   console.log(`PR Nudge starting for ${owner}/${repo}... (EXECUTE_ACTIONS=${EXECUTE_ACTIONS})`);
 
   try {
-    // 1. Fetch community PRs that pass CI but need review
+    const MAINTAINERS = await getMaintainers();
+    const WORKLOAD = await getMaintainerWorkload();
+
+    // 1. Fetch community PRs
     const query = `
     query($owner: String!, $repo: String!) {
       repository(owner: $owner, name: $repo) {
-        pullRequests(first: 50, states: OPEN) {
+        pullRequests(first: 500, states: OPEN, orderBy: {field: UPDATED_AT, direction: ASC}) {
           nodes {
             number
             author { login }
@@ -28,6 +31,7 @@ async function run() {
             isDraft
             reviewDecision
             mergeable
+            assignees(first: 1) { nodes { login } }
             commits(last: 1) {
               nodes {
                 commit {
@@ -45,13 +49,20 @@ async function run() {
       }
     }
     `;
-    const output = execSync(`gh api graphql -F owner=${owner} -F repo=${repo} -f query='${query}'`, { encoding: 'utf-8' });
+    const output = execSync(`gh api graphql -F owner=${owner} -F repo=${repo} -f query='${query}'`, { encoding: 'utf-8', maxBuffer: 50 * 1024 * 1024 });
     const data = JSON.parse(output).data.repository;
     const prs = data.pullRequests.nodes;
 
     const actions = [];
     const now = new Date();
 
+    // Sort maintainers by workload (ascending)
+    const sortedMaintainers = MAINTAINERS
+      .filter(m => m !== 'TOTAL_MAINTAINERS')
+      .sort((a, b) => (WORKLOAD[a] || 0) - (WORKLOAD[b] || 0));
+
+    let mIndex = 0;
+
     for (const pr of prs) {
       if (['MEMBER', 'OWNER', 'COLLABORATOR'].includes(pr.authorAssociation)) continue;
       if (pr.isDraft) continue;
@@ -62,12 +73,24 @@ async function run() {
       const isConflicting = pr.mergeable === 'CONFLICTING';
 
       const updatedAt = new Date(pr.updatedAt);
-      const hoursSinceUpdate = (now.getTime() - updatedAt.getTime()) / (1000 * 60 * 60);
+      const daysSinceUpdate = (now.getTime() - updatedAt.getTime()) / (1000 * 60 * 60 * 24);
+      const hoursSinceUpdate = daysSinceUpdate * 24;
 
       const hasNudgeLabel = pr.labels.nodes.some(l => l.name === 'status/nudge');
       const hasConflictLabel = pr.labels.nodes.some(l => l.name === 'status/merge-conflict');
+      const hasAssignee = pr.assignees.nodes.length > 0;
 
-      // 1. Author Nudge for Conflicts
+      // 1. Terminal State: Close stale conflicts
+      if (isConflicting && hasConflictLabel && daysSinceUpdate > 14) {
+          actions.push({
+              number: pr.number,
+              type: 'close-conflict',
+              comment: `Hi @${pr.author?.login || 'author'}! This PR has had merge conflicts for over 14 days. We are closing it to keep the queue manageable. Please feel free to reopen it once you have resolved the conflicts and synchronized with the main branch.`
+          });
+          continue;
+      }
+
+      // 2. Author Nudge for Conflicts
       if (isConflicting && !hasConflictLabel) {
           actions.push({
               number: pr.number,
@@ -77,14 +100,26 @@ async function run() {
           continue;
       }
 
-      // 2. Maintainer Nudge for Ready PRs
+      // 3. Maintainer Action for Ready PRs
       if (isCiSuccess && isMergeable && pr.reviewDecision === 'REVIEW_REQUIRED') {
-        // Nudge maintainers if ready and no activity for 48 hours
-        if (hoursSinceUpdate > 48 && !hasNudgeLabel) {
+        if (!hasAssignee) {
+           // Assign a maintainer based on workload
+           const assignee = sortedMaintainers[mIndex % sortedMaintainers.length];
+           mIndex++;
+           WORKLOAD[assignee] = (WORKLOAD[assignee] || 0) + 1;
+
+           actions.push({
+               number: pr.number,
+               type: 'assign-reviewer',
+               assignee,
+               comment: `Hi @${assignee}! This community PR by @${pr.author?.login || 'author'} is ready for review (Mergeable + CI Success). Assigning to you based on current workload.`
+           });
+        } else if (hoursSinceUpdate > 48 && !hasNudgeLabel) {
+          // Nudge existing assignee if inactive for 48 hours
           actions.push({
             number: pr.number,
             type: 'maintainer-nudge',
-            comment: `Hi maintainers! This community PR by @${pr.author?.login || 'author'} has passed all CI checks and is mergeable. It has been open and inactive for over 48 hours. Could someone please take a look? @google-gemini/gemini-cli-maintainers`
+            comment: `Hi @${pr.assignees.nodes[0].login}! This PR is ready and has been inactive for over 48 hours. Could you please take a look?`
           });
         }
       }
@@ -96,15 +131,21 @@ async function run() {
     for (const action of actions) {
       try {
         if (action.type === 'author-nudge-conflict') {
-            execGh(`pr edit ${action.number} --add-label "status/merge-conflict"`, EXECUTE_ACTIONS);
+            await execGh(`pr edit ${action.number} --add-label "status/merge-conflict"`, EXECUTE_ACTIONS);
             simulationUpdates.set(action.number.toString(), { labels: 'status/merge-conflict' });
+        } else if (action.type === 'close-conflict') {
+            await execGh(`pr close ${action.number}`, EXECUTE_ACTIONS);
+            simulationUpdates.set(action.number.toString(), { state: 'CLOSED' });
+        } else if (action.type === 'assign-reviewer') {
+            await execGh(`pr edit ${action.number} --add-assignee "${action.assignee}" --add-label "status/nudge"`, EXECUTE_ACTIONS);
+            simulationUpdates.set(action.number.toString(), { assignee: action.assignee, labels: 'status/nudge' });
         } else if (action.type === 'maintainer-nudge') {
-            execGh(`pr edit ${action.number} --add-label "status/nudge"`, EXECUTE_ACTIONS);
+            await execGh(`pr edit ${action.number} --add-label "status/nudge"`, EXECUTE_ACTIONS);
             simulationUpdates.set(action.number.toString(), { labels: 'status/nudge' });
         }
 
         if (action.comment) {
-          execGh(`pr comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
+          await execGh(`pr comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
         }
       } catch (err) {
         console.error(`Failed to process PR #${action.number}:`, err);
@@ -114,7 +155,7 @@ async function run() {
     // 3. Update simulation
     await updateSimulationCsv('prs-after.csv', simulationUpdates);
 
-    console.log(`Processed ${actions.length} PR nudges.`);
+    console.log(`Processed ${actions.length} PR nudges/actions.`);
 
   } catch (err) {
     console.error('Error in PR Nudge:', err);

tools/optimizer/processes/scripts/stale_manager.ts

@@ -14,11 +14,11 @@ async function run() {
   console.log(`Stale Manager starting for ${owner}/${repo}... (EXECUTE_ACTIONS=${EXECUTE_ACTIONS})`);
 
   try {
-    // 1. Fetch open issues/PRs that might be stale
-    const query = `
+    // 1. Fetch open issues
+    const issueQuery = `
     query($owner: String!, $repo: String!) {
       repository(owner: $owner, name: $repo) {
-        issues(first: 200, states: OPEN, orderBy: {field: UPDATED_AT, direction: ASC}) {
+        issues(first: 1000, states: OPEN, orderBy: {field: UPDATED_AT, direction: ASC}) {
           nodes {
             number
             authorAssociation
@@ -28,7 +28,22 @@ async function run() {
             }
           }
         }
-        pullRequests(first: 200, states: OPEN, orderBy: {field: UPDATED_AT, direction: ASC}) {
+      }
+    }
+    `;
+    let issueOutput;
+    try {
+      issueOutput = execSync(`gh api graphql -F owner=${owner} -F repo=${repo} -f query='${issueQuery}'`, { encoding: 'utf-8', maxBuffer: 50 * 1024 * 1024 });
+    } catch (err) {
+      console.error('Failed to fetch issues from GitHub:', err);
+      process.exit(1);
+    }
+
+    // 2. Fetch open PRs
+    const prQuery = `
+    query($owner: String!, $repo: String!) {
+      repository(owner: $owner, name: $repo) {
+        pullRequests(first: 500, states: OPEN, orderBy: {field: UPDATED_AT, direction: ASC}) {
           nodes {
             number
             authorAssociation
@@ -52,16 +67,17 @@ async function run() {
       }
     }
     `;
-    let output;
+    let prOutput;
     try {
-      output = execSync(`gh api graphql -F owner=${owner} -F repo=${repo} -f query='${query}'`, { encoding: 'utf-8', maxBuffer: 10 * 1024 * 1024 });
+      prOutput = execSync(`gh api graphql -F owner=${owner} -F repo=${repo} -f query='${prQuery}'`, { encoding: 'utf-8', maxBuffer: 50 * 1024 * 1024 });
     } catch (err) {
-      console.error('Failed to fetch issues/PRs from GitHub:', err);
+      console.error('Failed to fetch PRs from GitHub:', err);
       process.exit(1);
     }
 
-    const data = JSON.parse(output).data.repository;
-    const items = [...data.issues.nodes.map(i => ({...i, type: 'issue'})), ...data.pullRequests.nodes.map(p => ({...p, type: 'pr'}))];
+    const issueData = JSON.parse(issueOutput).data.repository;
+    const prData = JSON.parse(prOutput).data.repository;
+    const items = [...issueData.issues.nodes.map(i => ({...i, type: 'issue'})), ...prData.pullRequests.nodes.map(p => ({...p, type: 'pr'}))];
 
     const now = new Date();
     const actions = [];
@@ -71,10 +87,23 @@ async function run() {
       const daysSinceUpdate = (now.getTime() - updatedAt.getTime()) / (1000 * 60 * 60 * 24);
       const isMaintainerOnly = item.labels.nodes.some(l => l.name === '🔒 maintainer only');
       const isStale = item.labels.nodes.some(l => l.name === 'Stale');
+      const needsInfo = item.labels.nodes.some(l => l.name === 'status/needs-info');
       const isCommunity = !['MEMBER', 'OWNER', 'COLLABORATOR'].includes(item.authorAssociation);
 
       if (isMaintainerOnly) continue; // Maintainer issues have their own lifecycle
 
+      // Special handling for needs-info: mark stale faster
+      if (needsInfo && !isStale && daysSinceUpdate > 7) {
+          actions.push({
+              number: item.number,
+              target: item.type,
+              type: 'label',
+              label: 'Stale',
+              comment: `Hi! This ${item.type} was marked as 'status/needs-info' but has had no activity for 7 days. We are labeling it as 'Stale'. It will be closed in 14 days if no further activity occurs. Thank you!`
+          });
+          continue;
+      }
+
       // Safeguard: Don't mark as stale if it's a PR ready for review (Maintainer bottleneck)
       if (item.type === 'pr') {
           const ciState = item.commits?.nodes[0]?.commit?.statusCheckRollup?.state;
@@ -124,14 +153,14 @@ async function run() {
         const simulationMap = action.target === 'pr' ? prSimulationUpdates : issueSimulationUpdates;
 
         if (action.type === 'label') {
-          execGh(`${cmdPrefix} edit ${action.number} --add-label "Stale"`, EXECUTE_ACTIONS);
-          simulationMap.set(action.number.toString(), { labels: 'Stale' });
+          await execGh(`${cmdPrefix} edit ${action.number} --add-label "${action.label}"`, EXECUTE_ACTIONS);
+          simulationMap.set(action.number.toString(), { labels: action.label });
         }
         if (action.comment) {
-          execGh(`${cmdPrefix} comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
+          await execGh(`${cmdPrefix} comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
         }
         if (action.type === 'close') {
-          execGh(`${cmdPrefix} close ${action.number}`, EXECUTE_ACTIONS);
+          await execGh(`${cmdPrefix} close ${action.number}`, EXECUTE_ACTIONS);
           simulationMap.set(action.number.toString(), { state: 'CLOSED' });
         }
       } catch (err) {

tools/optimizer/processes/scripts/triage_router.ts

@@ -5,7 +5,7 @@
  */
 
 import { execSync } from 'node:child_process';
-import { getMaintainers, execGh, getRepoInfo, updateSimulationCsv } from './utils.js';
+import { getMaintainers, execGh, getRepoInfo, updateSimulationCsv, getMaintainerWorkload } from './utils.js';
 
 const EXECUTE_ACTIONS = process.env.EXECUTE_ACTIONS === 'true';
 
@@ -15,13 +15,14 @@ async function run() {
 
   try {
     const MAINTAINERS = await getMaintainers();
-    console.log(`Fetched ${MAINTAINERS.length} maintainers.`);
+    const WORKLOAD = await getMaintainerWorkload();
+    console.log(`Fetched ${MAINTAINERS.length} maintainers and current workloads.`);
 
-    // 1. Fetch untriaged issues
+    // 1. Fetch untriaged issues (Increase limit to process the backlog)
     const query = `
     query($owner: String!, $repo: String!) {
       repository(owner: $owner, name: $repo) {
-        issues(first: 100, states: OPEN, labels: ["status/need-triage"]) {
+        issues(first: 1000, states: OPEN, labels: ["status/need-triage"], orderBy: {field: CREATED_AT, direction: ASC}) {
           nodes {
             number
             title
@@ -46,43 +47,53 @@ async function run() {
     const issues = data.issues.nodes;
 
     const actions = [];
-    let maintainerIndex = Math.floor(Math.random() * MAINTAINERS.length);
+    
+    // Sort maintainers by workload (ascending)
+    const sortedMaintainers = MAINTAINERS
+      .filter(m => m !== 'TOTAL_MAINTAINERS') // safeguard
+      .sort((a, b) => (WORKLOAD[a] || 0) - (WORKLOAD[b] || 0));
+
+    let mIndex = 0;
 
     for (const issue of issues) {
       if (issue.assignees.nodes.length > 0) continue;
 
       const body = issue.body || '';
-      const title = issue.title || '';
+      const title = issue.title.toLowerCase();
+
+      // Better categorization
+      const labelsToAdd: string[] = [];
+      if (title.includes('bug') || body.toLowerCase().includes('expected behavior')) {
+        labelsToAdd.push('type/bug');
+      } else if (title.includes('feature') || title.includes('enhancement') || body.toLowerCase().includes('proposed change')) {
+        labelsToAdd.push('type/feature');
+      }
 
       // Low quality check
       if (body.length < 50 || title.length < 10 || !body.includes('###')) {
         actions.push({
           number: issue.number,
           type: 'needs-info',
+          labelsToAdd: ['status/needs-info'],
+          labelsToRemove: ['status/need-triage'],
           comment: `Hi @${issue.author?.login || 'author'}! Thank you for the report. This issue seems to be missing some critical information or doesn't follow the template. Could you please provide more details? Labeling as 'status/needs-info' for now.`
         });
         continue;
       }
 
-      // Potential duplicate check (very naive but better than nothing)
-      if (title.toLowerCase().includes('duplicate') || title.toLowerCase().includes('same as #')) {
-          actions.push({
-              number: issue.number,
-              type: 'possible-duplicate',
-              comment: `Hi @${issue.author?.login || 'author'}! This issue might be a duplicate of another existing issue. Labeling as 'status/possible-duplicate' for maintainer review.`
-          });
-          continue;
-      }
-
-      // Assign to a maintainer (round-robin)
-      const assignee = MAINTAINERS[maintainerIndex % MAINTAINERS.length];
-      maintainerIndex++;
+      // Assign to the maintainer with the lowest workload
+      const assignee = sortedMaintainers[mIndex % sortedMaintainers.length];
+      mIndex++;
+      // Increment local workload tracker to keep distribution even during this run
+      WORKLOAD[assignee] = (WORKLOAD[assignee] || 0) + 1;
 
       actions.push({
         number: issue.number,
         type: 'assign',
         assignee,
-        comment: `Automated Triage: Assigning to @${assignee} for initial review. Please categorize and set priority.`
+        labelsToAdd: [...labelsToAdd, 'status/manual-triage'],
+        labelsToRemove: ['status/need-triage'],
+        comment: `Automated Triage: Assigning to @${assignee} based on current workload. Please categorize and set priority.`
       });
     }
 
@@ -91,19 +102,22 @@ async function run() {
 
     for (const action of actions) {
       try {
-        if (action.type === 'needs-info') {
-          execGh(`issue edit ${action.number} --add-label "status/needs-info" --remove-label "status/need-triage"`, EXECUTE_ACTIONS);
-          simulationUpdates.set(action.number.toString(), { labels: 'status/needs-info' });
-        } else if (action.type === 'possible-duplicate') {
-            execGh(`issue edit ${action.number} --add-label "status/possible-duplicate" --remove-label "status/need-triage"`, EXECUTE_ACTIONS);
-            simulationUpdates.set(action.number.toString(), { labels: 'status/possible-duplicate' });
-        } else if (action.type === 'assign') {
-          execGh(`issue edit ${action.number} --add-assignee "${action.assignee}" --remove-label "status/need-triage" --add-label "status/manual-triage"`, EXECUTE_ACTIONS);
-          simulationUpdates.set(action.number.toString(), { labels: 'status/manual-triage' });
-        }
+        const addLabels = action.labelsToAdd?.map(l => `"${l}"`).join(',') || '';
+        const removeLabels = action.labelsToRemove?.map(l => `"${l}"`).join(',') || '';
+        
+        let editCmd = `issue edit ${action.number}`;
+        if (addLabels) editCmd += ` --add-label ${addLabels}`;
+        if (removeLabels) editCmd += ` --remove-label ${removeLabels}`;
+        if (action.assignee) editCmd += ` --add-assignee "${action.assignee}"`;
+
+        await execGh(editCmd, EXECUTE_ACTIONS);
+        simulationUpdates.set(action.number.toString(), { 
+            labels: action.labelsToAdd?.join(', ') || '',
+            assignee: action.assignee || ''
+        });
         
         if (action.comment) {
-          execGh(`issue comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
+          await execGh(`issue comment ${action.number} --body "${action.comment}"`, EXECUTE_ACTIONS);
         }
       } catch (err) {
         console.error(`Failed to process issue #${action.number}:`, err);

tools/optimizer/processes/scripts/utils.ts

@@ -36,14 +36,17 @@ export async function getMaintainers(): Promise<string[]> {
     
     const lines = content.split('\n');
     for (const line of lines) {
-      if (line.startsWith('#') || !line.trim()) continue;
+      const cleanLine = line.split('#')[0].trim();
+      if (!cleanLine) continue;
+
       // Match @user or @org/team
-      const matches = line.match(/@[\w-]+\/[\w-]+|@[\w-]+/g);
+      const matches = cleanLine.match(/@[\w-]+\/[\w-]+|@[\w-]+/g);
       if (matches) {
         for (const match of matches) {
           if (match.includes('/')) {
-            // For team mentions, we might want to expand them, 
-            // but for now let's just skip them or handle them if needed.
+            // For teams, we should ideally expand them via gh api, 
+            // but for simulation/simple use, we'll just log it.
+            console.log(`[INFO] Found team ownership: ${match}. Skipping team expansion for now.`);
             continue;
           }
           maintainers.add(match.replace('@', ''));
@@ -52,6 +55,7 @@ export async function getMaintainers(): Promise<string[]> {
     }
     
     if (maintainers.size === 0) {
+      console.warn('No maintainers found in CODEOWNERS, using fallbacks.');
       return ['gundermanc', 'jackwotherspoon', 'DavidAPierce'];
     }
     
@@ -126,7 +130,14 @@ export async function updateSimulationCsv(
       for (const [colName, newValue] of Object.entries(update)) {
         const colIndex = header.indexOf(colName);
         if (colIndex !== -1) {
-          columns[colIndex] = newValue;
+          if (colName === 'labels') {
+            const existingLabels = columns[colIndex].replace(/"/g, '').split(',').map(s => s.trim()).filter(Boolean);
+            const newLabels = newValue.split(',').map(s => s.trim()).filter(Boolean);
+            const combined = Array.from(new Set([...existingLabels, ...newLabels]));
+            columns[colIndex] = `"${combined.join(', ')}"`;
+          } else {
+            columns[colIndex] = newValue.includes(',') ? `"${newValue}"` : newValue;
+          }
         }
       }
     }
@@ -136,10 +147,37 @@ export async function updateSimulationCsv(
   await fs.writeFile(filePath, newLines.join('\n'));
 }
 
+/**
+ * Reads maintainer workload from maintainer_workload.csv.
+ */
+export async function getMaintainerWorkload(): Promise<Record<string, number>> {
+  try {
+    const filePath = path.join(process.cwd(), 'maintainer_workload.csv');
+    const content = await fs.readFile(filePath, 'utf8');
+    const workload: Record<string, number> = {};
+    
+    const lines = content.split('\n');
+    for (let i = 1; i < lines.length; i++) {
+      const line = lines[i].trim();
+      if (!line || line.startsWith('TOTAL') || line.startsWith('RATIO')) continue;
+      
+      const columns = line.split(',');
+      const maintainer = columns[0];
+      const assignedIssues = parseInt(columns[1] || '0', 10);
+      const assignedPrs = parseInt(columns[2] || '0', 10);
+      workload[maintainer] = assignedIssues + assignedPrs;
+    }
+    return workload;
+  } catch (err) {
+    console.warn('maintainer_workload.csv not found or unreadable, returning empty workload.');
+    return {};
+  }
+}
+
 /**
  * Executes a gh command with logging and dry-run support.
  */
-export function execGh(command: string, execute: boolean) {
+export async function execGh(command: string, execute: boolean) {
   if (!execute) {
     console.log(`[DRY RUN] Would execute: gh ${command}`);
     return;
@@ -147,8 +185,11 @@ export function execGh(command: string, execute: boolean) {
 
   console.log(`Executing: gh ${command}`);
   try {
+    // Small delay to be nicer to the API and avoid race conditions if run concurrently
+    await new Promise(resolve => setTimeout(resolve, 100));
     execSync(`gh ${command}`, { stdio: 'inherit' });
   } catch (err) {
     console.error(`Failed to execute gh ${command}:`, err);
   }
 }
+

tools/optimizer/query.graphql

@@ -0,0 +1,15 @@
+query($owner: String!, $repo: String!) {
+  repository(owner: $owner, name: $repo) {
+    pullRequests(last: 100, states: MERGED) {
+      nodes {
+        number
+        reviews(first: 20) {
+          nodes {
+            authorAssociation
+            author { login, ... on User { name } }
+          }
+        }
+      }
+    }
+  }
+}