Commit Graph

6 Commits

Author SHA1 Message Date
cocosheng-g
9da1542071 feat(ci): isolate workflow evals into independent nightly job
- Splits 'Evals: Nightly' into 'evals' (general capabilities) and 'workflow-evals' (specific workflow simulations).
- 'workflow-evals' runs only on 'gemini-2.5-pro' (the target model).
- 'evals' excludes workflow tests to prevent noise/skewed metrics on other models.
- Removes code-level 'targetModels' restrictions in favor of CI configuration.
- Updates aggregation script to handle skipped tests correctly (though exclusion avoids them).
2026-02-03 22:37:15 -05:00
Christian Gunderman
ba8c64459b ci: allow failure in evals-nightly run step (#17319) 2026-01-22 18:23:15 +00:00
Christian Gunderman
982a093791 Enable the ability to queue specific nightly eval tests (#17262) 2026-01-22 03:07:33 +00:00
Christian Gunderman
c43b04b44c Run evals for all models. (#17123) 2026-01-21 16:38:37 +00:00
Christian Gunderman
66e7b479ae Aggregate test results. (#16581) 2026-01-14 07:08:05 +00:00
Christian Gunderman
8030404b08 Behavioral evals framework. (#16047) 2026-01-14 04:49:17 +00:00