MediaMetz/gemini-cli
1
0
Fork 0
mirror of https://github.com/google-gemini/gemini-cli.git synced 2026-06-12 04:17:15 -07:00
Code Issues Packages Projects Releases Wiki Activity

feat(prompt-optimization): implement multi-objective evaluation metrics

Browse Source 
Established a Pareto-ready evaluation foundation for the Genetic-Pareto (GEPA)
optimizer, supporting simultaneous optimization of accuracy and density.

Key improvements:
- Core Architecture: Defined standardized `MetricResult` and `OptimizationDirection`
  types in `packages/core/src/evals/types.ts` to support multi-objective fitness.
- Centralized Config: Implemented `packages/core/src/evals/config.ts` with tunable
  weights and detailed documentation for scoring gradients.
- Tool Alignment Metric: Created `metrics/toolAlignment.ts` to measure functional
  accuracy, argument precision, and explicit shell avoidance.
- Token Frugality Metric: Created `metrics/tokenFrugality.ts` to measure and
  penalize conversational noise ("chatter") using a configurable threshold.
- Verification Suite: Added comprehensive unit tests for all metrics, achieving
  100% coverage of scoring logic and gradient steps.
- Project Integration: Relocated `schema.ts` to the core package for build safety,
  updated the data validator, and extended project-wide lint/format scripts.
This commit is contained in:
Abhijit Balaji
2026-03-02 14:10:45 -08:00
parent c0b463dbcf
commit 6c94c4d9ca
9 changed files with 458 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
package.json
+2 -2
View File
@@ -62,8 +62,8 @@
"release:version": "node scripts/version.js",
"telemetry": "node scripts/telemetry.js",
"data:validate": "tsx scripts/validate-data.ts",
"data:format": "prettier --write 'data/**/*.ts' 'data/*.json' 'scripts/validate-data.ts'",
"data:lint": "eslint 'data/**/*.ts' 'scripts/validate-data.ts'",
"data:format": "prettier --write 'data/**/*.ts' 'data/*.json' 'scripts/validate-data.ts' 'packages/core/src/evals/**/*.ts'",
"data:lint": "eslint 'data/**/*.ts' 'scripts/validate-data.ts' 'packages/core/src/evals/**/*.ts'",
"check:lockfile": "node scripts/check-lockfile.js",
"clean": "node scripts/clean.js",
"pre-commit": "node scripts/pre-commit.js"