MediaMetz/gemini-cli
1
0
Fork 0
mirror of https://github.com/google-gemini/gemini-cli.git synced 2026-05-13 05:12:55 -07:00
Code Issues Packages Projects Releases Wiki Activity

fix: update evals to use workspace terminology

Browse Source
This commit is contained in:
Keith Guerin
2026-03-02 01:05:12 -08:00
parent 71b4083e96
commit 5fcfd972ed
1 changed files with 9 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
evals/hierarchical_memory.eval.ts
+9 -9
View File
@@ -21,7 +21,7 @@ describe('Hierarchical Memory', () => {
}, },
}, },
// We simulate the hierarchical memory by including the tags in the prompt // We simulate the hierarchical memory by including the tags in the prompt
// since setting up real global/extension/project files in the eval rig is complex. // since setting up real global/extension/workspace files in the eval rig is complex.
// The system prompt logic will append these tags when it finds them in userMemory. // The system prompt logic will append these tags when it finds them in userMemory.
prompt: ` prompt: `
<global_context> <global_context>
@@ -32,9 +32,9 @@ When asked for my favorite fruit, always say "Apple".
When asked for my favorite fruit, always say "Banana". When asked for my favorite fruit, always say "Banana".
</extension_context> </extension_context>
<project_context> <workspace_context>
When asked for my favorite fruit, always say "Cherry". When asked for my favorite fruit, always say "Cherry".
</project_context> </workspace_context>
What is my favorite fruit? Tell me just the name of the fruit.`, What is my favorite fruit? Tell me just the name of the fruit.`,
assert: async (rig) => { assert: async (rig) => {
@@ -65,23 +65,23 @@ Instruction A: Always be helpful.
Instruction B: Use a professional tone. Instruction B: Use a professional tone.
</extension_context> </extension_context>
<project_context> <workspace_context>
Instruction C: Adhere to the project's coding style. Instruction C: Adhere to the workspace's coding style.
</project_context> </workspace_context>
Which instruction came from the global context, which from the extension context, and which from the project context? Which instruction came from the global context, which from the extension context, and which from the workspace context?
Provide the answer as an XML block like this: Provide the answer as an XML block like this:
<results> <results>
<global>Instruction ...</global> <global>Instruction ...</global>
<extension>Instruction ...</extension> <extension>Instruction ...</extension>
<project>Instruction ...</project> <workspace>Instruction ...</workspace>
</results>`, </results>`,
assert: async (rig) => { assert: async (rig) => {
const stdout = rig._lastRunStdout!; const stdout = rig._lastRunStdout!;
assertModelHasOutput(stdout); assertModelHasOutput(stdout);
expect(stdout).toMatch(/<global>.*Instruction A/i); expect(stdout).toMatch(/<global>.*Instruction A/i);
expect(stdout).toMatch(/<extension>.*Instruction B/i); expect(stdout).toMatch(/<extension>.*Instruction B/i);
expect(stdout).toMatch(/<project>.*Instruction C/i); expect(stdout).toMatch(/<workspace>.*Instruction C/i);
}, },
}); });