feat(prompt-optimization): implement Data Layer MVP and Tool Alignment dataset

Established the "Heart" of the Prompt Optimization Pipeline by building a robust,
extensible data infrastructure and a high-fidelity golden dataset.

Key improvements:
- Core Schema: Defined the `Scenario` interface in `data/schema.ts` supporting
  multiple negative failure modes, platform-specific shell contexts (Unix/Win32),
  and strict tool-call typing.
- Optimization Manifest: Created `data/manifest.json` to define "No-Fly Zones"
  for the optimizer, protecting literal tool names and template variables, while
  providing descriptive context for validation.
- Tool Alignment Dataset: Authored 113 scenarios in `data/tool_alignment.jsonl`
  across 20 tools, focusing on "Built-in over Shell" preference. Heavily weighted
  `replace` (12) and `write_file` (10) to enforce surgical editing.
- Extensible Validator: Implemented `scripts/validate-data.ts` to provide
  real-time integrity checks and purpose-driven coverage reports.
- Project Integration: Added `data:validate`, `data:format`, and `data:lint`
  scripts to package.json and updated ESLint config to cover the data directory.
This commit is contained in:
Abhijit Balaji
2026-03-02 11:20:03 -08:00
parent 717660997d
commit c0b463dbcf
6 changed files with 403 additions and 1 deletions
+3
View File
@@ -61,6 +61,9 @@
"prepare:package": "node scripts/prepare-package.js",
"release:version": "node scripts/version.js",
"telemetry": "node scripts/telemetry.js",
"data:validate": "tsx scripts/validate-data.ts",
"data:format": "prettier --write 'data/**/*.ts' 'data/*.json' 'scripts/validate-data.ts'",
"data:lint": "eslint 'data/**/*.ts' 'scripts/validate-data.ts'",
"check:lockfile": "node scripts/check-lockfile.js",
"clean": "node scripts/clean.js",
"pre-commit": "node scripts/pre-commit.js"