2026-01-08 10:43:35 -08:00
|
|
|
# Model routing
|
2025-11-18 08:09:55 -08:00
|
|
|
|
|
|
|
|
Gemini CLI includes a model routing feature that automatically switches to a
|
|
|
|
|
fallback model in case of a model failure. This feature is enabled by default
|
|
|
|
|
and provides resilience when the primary model is unavailable.
|
|
|
|
|
|
2025-12-01 11:38:48 -08:00
|
|
|
## How it works
|
2025-11-18 08:09:55 -08:00
|
|
|
|
2025-12-17 17:14:33 -05:00
|
|
|
Model routing is managed by the `ModelAvailabilityService`, which monitors model
|
|
|
|
|
health and automatically routes requests to available models based on defined
|
|
|
|
|
policies.
|
2025-11-18 08:09:55 -08:00
|
|
|
|
2025-12-17 17:14:33 -05:00
|
|
|
1. **Model failure:** If the currently selected model fails (e.g., due to quota
|
2026-01-15 07:59:31 -08:00
|
|
|
or server errors), the CLI will initiate the fallback process.
|
2025-11-18 08:09:55 -08:00
|
|
|
|
2025-12-17 17:14:33 -05:00
|
|
|
2. **User consent:** Depending on the failure and the model's policy, the CLI
|
|
|
|
|
may prompt you to switch to a fallback model (by default always prompts
|
|
|
|
|
you).
|
2025-11-18 08:09:55 -08:00
|
|
|
|
2026-01-20 14:43:43 -05:00
|
|
|
Some internal utility calls (such as prompt completion and classification)
|
|
|
|
|
use a silent fallback chain for `gemini-2.5-flash-lite` and will fall back
|
|
|
|
|
to `gemini-2.5-flash` and `gemini-2.5-pro` without prompting or changing the
|
|
|
|
|
configured model.
|
|
|
|
|
|
2025-12-17 17:14:33 -05:00
|
|
|
3. **Model switch:** If approved, or if the policy allows for silent fallback,
|
|
|
|
|
the CLI will use an available fallback model for the current turn or the
|
|
|
|
|
remainder of the session.
|
2025-11-18 08:09:55 -08:00
|
|
|
|
2025-12-01 11:38:48 -08:00
|
|
|
### Model selection precedence
|
2025-11-18 08:09:55 -08:00
|
|
|
|
|
|
|
|
The model used by Gemini CLI is determined by the following order of precedence:
|
|
|
|
|
|
|
|
|
|
1. **`--model` command-line flag:** A model specified with the `--model` flag
|
|
|
|
|
when launching the CLI will always be used.
|
|
|
|
|
2. **`GEMINI_MODEL` environment variable:** If the `--model` flag is not used,
|
|
|
|
|
the CLI will use the model specified in the `GEMINI_MODEL` environment
|
|
|
|
|
variable.
|
|
|
|
|
3. **`model.name` in `settings.json`:** If neither of the above are set, the
|
|
|
|
|
model specified in the `model.name` property of your `settings.json` file
|
|
|
|
|
will be used.
|
2025-12-01 11:38:48 -08:00
|
|
|
4. **Default model:** If none of the above are set, the default model will be
|
2025-11-21 09:54:17 -08:00
|
|
|
used. The default model is `auto`
|