docs(skill): add schema update step and cleanup workflow numbering in experimentation skill

This commit is contained in:
mkorwel
2026-02-19 21:07:47 -06:00
committed by Matt Korwel
parent a1db5f9faf
commit 24bbeb11c7
+12 -8
View File
@@ -10,6 +10,7 @@ This skill assists developers in adding net-new remote experiments (feature flag
## Key Files
- `packages/core/src/code_assist/experiments/flagNames.ts`: Definitions and metadata for all experiment flags.
- `packages/core/src/config/config.ts`: Unified configuration object where strongly typed wrappers for experiments are added.
- `packages/cli/src/config/settingsSchema.ts`: Schema definitions for `settings.json`.
## Core Pattern: Unified Configuration
@@ -19,6 +20,7 @@ Gemini CLI uses a unified `Config` object as the source of truth. Every experime
## Workflow: Adding a New Experiment
When a user asks to add a new experiment, follow these steps sequentially:
### 0. Research and Discovery
- **Locate Existing Flags:** Read `packages/core/src/code_assist/experiments/flagNames.ts` to see current IDs and naming conventions.
- **Check for Duplicates:** Ensure the proposed experiment doesn't already exist or overlap with another flag.
@@ -35,10 +37,16 @@ When a user asks to add a new experiment, follow these steps sequentially:
- Add a corresponding entry to `ExperimentMetadata`, including `description`, `type`, and `defaultValue`.
- **Note:** The key in `ExperimentFlags` (converted to kebab-case) will be the name used for CLI flags and Settings. For example, `MY_NEW_FEATURE` becomes `my-new-feature`.
### 3. Usage in Code
### 3. Update Settings Schema
- **File:** `packages/cli/src/config/settingsSchema.ts`
- To ensure IDE autocompletion and validation work for the new experiment in `settings.json`, you MUST add it to the `experimental.properties` section of the schema.
- Match the kebab-case name used in step 2.
- **Action:** After updating the file, run `npm run schema:settings` to regenerate `schemas/settings.schema.json`.
### 4. Usage in Code
- **Generic Method:** `config.getExperimentValue<Type>(ExperimentFlags.YOUR_FLAG_ID)`
- **Preferred Pattern (Strongly Typed Wrappers):** To maintain a clean and discoverable interface, you should add a strongly typed wrapper method in `packages/core/src/config/config.ts`. This allows other developers to easily find and use your experiment with proper type safety.
Example in `Config` class:
```typescript
isNewFeatureEnabled(): boolean {
@@ -47,10 +55,6 @@ When a user asks to add a new experiment, follow these steps sequentially:
```
- **Testing:** Add or update tests in `packages/core/src/config/config.experiment.test.ts` to verify the flag resolves correctly across all priority levels (CLI, Settings, Remote, Default).
### 4. Telemetry and Metrics (Crucial)
- **Dynamic Nature:** While `getExperimentValue` is dynamic and doesn't *require* a `Config` update for every flag, the wrapper pattern is highly encouraged for long-term maintainability.
- **CLI Override:** Users can override via `--experiment your-flag-name=value`.
- **Settings Override:** Users can override in their `settings.json`:
```json
@@ -59,11 +63,11 @@ When a user asks to add a new experiment, follow these steps sequentially:
}
```
### 3. Telemetry and Metrics (Crucial)
### 5. Telemetry and Metrics (Crucial)
- Prompt the user to think deeply about what metrics are necessary to evaluate the success or failure of the experiment.
- Ask questions like: "How will we know if this feature is working as intended?" or "What telemetry events should be fired when this new code path is executed?"
- Help them add the necessary telemetry calls to the codebase to capture these insights.
### 4. Branching and PR Preparation
### 6. Branching and PR Preparation
- If not already on a dedicated branch, help the user create a new git branch for this experiment (e.g., `git checkout -b exp/feature-name`).
- Remind them to run local tests and linting (`npm run lint:all`, `npm test` or `npm run preflight`) before preparing a Pull Request.