LLM Backends
incomplianceRadar's NLP extraction and Q&A features run against a single
LlmProvider trait (crates/llm/src/lib.rs), selected at runtime via the
LLM_BACKEND environment variable — no code changes needed to switch.
| Backend | LLM_BACKEND | Requires | Notes |
|---|---|---|---|
| Ollama (local) | ollama (default) | ollama serve running locally | No API key, no data leaves the machine. Set OLLAMA_MODEL to any model you've pulled. |
| Anthropic (frontier) | anthropic | ANTHROPIC_API_KEY | Uses the Messages API. Set ANTHROPIC_MODEL to override the default. |
See .env.example in the repository root for all variables.
Adding a new backend
- Implement
LlmProviderincrates/llm/src/providers/<name>.rs. - Add a variant to
LlmBackendincrates/llm/src/config.rsand wire it intoLlmConfig::from_env. provider_from_env()incrates/llm/src/lib.rspicks it up automatically.
This is the same Repository/Strategy pattern used elsewhere in the codebase — callers depend on the trait, never on a concrete provider.