forgejo_admin/westside-ai-assistant
1
0
Fork 0
Code Issues 13 Pull requests Projects Releases Packages Wiki Activity Actions

Model provider switch: Anthropic / Ollama feature flag #16

New issue
Closed
opened 2026-03-29 03:46:42 +00:00 by forgejo_admin · 1 comment
forgejo_admin commented 2026-03-29 03:46:42 +00:00
Owner
Copy link

Type

Feature

Lineage

Standalone — scoped from session discussion. Anthropic is primary for now, Ollama (qwen2.5:7b) is the zero-cost fallback.

Repo

forgejo_admin/westside-ai-assistant

User Story

As Lucas (platform operator)
I want to switch between Anthropic and Ollama with an env var
So that I can use Anthropic when I have tokens and fall back to Ollama when I don't

Context

The AI engine was originally built with Anthropic SDK, then swapped to OpenAI SDK for Ollama. These are different SDKs with different request/response formats. This ticket adds a AI_PROVIDER env var ("anthropic" or "ollama") that selects which SDK and endpoint to use. Tool definitions, system prompt, conversation history, and confirmation flow stay the same — only the LLM client layer changes. Ollama uses OpenAI-compatible API at cluster-internal URL. qwen2.5:7b is already pulled into the Ollama pod.

File Targets

Files the agent should modify:

  • app/ai.py — abstract LLM client behind a provider interface. Two implementations: AnthropicProvider (anthropic SDK) and OllamaProvider (openai SDK). Selected by AI_PROVIDER env var.
  • app/config.py — add ai_provider: str = "anthropic", keep both anthropic_api_key/anthropic_model and ollama_url/ollama_model fields
  • app/health.py — readiness checks the configured provider's credentials
  • requirements.txt — both anthropic and openai as dependencies
  • tests/test_ai.py — test both provider paths

Files the agent should NOT touch:

  • app/basketball.py, app/confirmation.py, app/groupme.py

Acceptance Criteria

  • AI_PROVIDER=anthropic uses Anthropic SDK with ANTHROPIC_API_KEY and ANTHROPIC_MODEL
  • AI_PROVIDER=ollama uses OpenAI SDK with OLLAMA_URL and OLLAMA_MODEL
  • Tool definitions work with both providers (converted at the provider layer)
  • Conversation history and confirmation flow unchanged
  • Missing credentials for selected provider → clear error at startup
  • Default provider is "anthropic"

Test Expectations

  • Unit test: anthropic provider dispatches correctly
  • Unit test: ollama provider dispatches correctly
  • Unit test: invalid provider raises startup error
  • Run command: pytest tests/ -v

Constraints

  • Provider interface must be minimal — just async def complete(messages, tools) -> ProviderResponse
  • ProviderResponse normalized so confirmation.py and groupme.py don't care which provider is active
  • Keep it simple — two if/else branches, not a plugin system

Checklist

  • PR opened
  • Tests pass
  • No unrelated changes

Related

  • project-westside-ai-assistant — parent project
  • story-westside-ai-assistant-read-ops — must work with both providers
### Type Feature ### Lineage Standalone — scoped from session discussion. Anthropic is primary for now, Ollama (qwen2.5:7b) is the zero-cost fallback. ### Repo `forgejo_admin/westside-ai-assistant` ### User Story As Lucas (platform operator) I want to switch between Anthropic and Ollama with an env var So that I can use Anthropic when I have tokens and fall back to Ollama when I don't ### Context The AI engine was originally built with Anthropic SDK, then swapped to OpenAI SDK for Ollama. These are different SDKs with different request/response formats. This ticket adds a `AI_PROVIDER` env var ("anthropic" or "ollama") that selects which SDK and endpoint to use. Tool definitions, system prompt, conversation history, and confirmation flow stay the same — only the LLM client layer changes. Ollama uses OpenAI-compatible API at cluster-internal URL. qwen2.5:7b is already pulled into the Ollama pod. ### File Targets Files the agent should modify: - `app/ai.py` — abstract LLM client behind a provider interface. Two implementations: AnthropicProvider (anthropic SDK) and OllamaProvider (openai SDK). Selected by `AI_PROVIDER` env var. - `app/config.py` — add `ai_provider: str = "anthropic"`, keep both `anthropic_api_key`/`anthropic_model` and `ollama_url`/`ollama_model` fields - `app/health.py` — readiness checks the configured provider's credentials - `requirements.txt` — both `anthropic` and `openai` as dependencies - `tests/test_ai.py` — test both provider paths Files the agent should NOT touch: - `app/basketball.py`, `app/confirmation.py`, `app/groupme.py` ### Acceptance Criteria - [ ] `AI_PROVIDER=anthropic` uses Anthropic SDK with `ANTHROPIC_API_KEY` and `ANTHROPIC_MODEL` - [ ] `AI_PROVIDER=ollama` uses OpenAI SDK with `OLLAMA_URL` and `OLLAMA_MODEL` - [ ] Tool definitions work with both providers (converted at the provider layer) - [ ] Conversation history and confirmation flow unchanged - [ ] Missing credentials for selected provider → clear error at startup - [ ] Default provider is "anthropic" ### Test Expectations - [ ] Unit test: anthropic provider dispatches correctly - [ ] Unit test: ollama provider dispatches correctly - [ ] Unit test: invalid provider raises startup error - Run command: `pytest tests/ -v` ### Constraints - Provider interface must be minimal — just `async def complete(messages, tools) -> ProviderResponse` - ProviderResponse normalized so confirmation.py and groupme.py don't care which provider is active - Keep it simple — two if/else branches, not a plugin system ### Checklist - [ ] PR opened - [ ] Tests pass - [ ] No unrelated changes ### Related - `project-westside-ai-assistant` — parent project - `story-westside-ai-assistant-read-ops` — must work with both providers
forgejo_admin commented 2026-03-29 22:36:25 +00:00
Author
Owner
Copy link

Scope Review: NEEDS_REFINEMENT

Review note: review-647-2026-03-28

Issue body is well-scoped and template-complete. All 5 file targets verified against the repo. The sole blocker is missing backing notes in pal-e-docs for the traceability triangle:

  • [SCOPE] Create user story note story-westside-ai-assistant-read-ops (referenced on project page, does not exist)
  • [SCOPE] Create user story note story-westside-ai-assistant-write-ops (referenced on project page, does not exist)
  • [SCOPE] Create architecture notes (arch-domain-westside-ai-assistant, arch-dataflow-westside-ai-assistant, arch-deployment-westside-ai-assistant) — project page links to them but none exist. The arch:A2 board label has no clear mapping to these named notes.

These are project-level gaps affecting all items on board-westside-ai-assistant. Once resolved, this ticket is READY.

## Scope Review: NEEDS_REFINEMENT Review note: `review-647-2026-03-28` Issue body is well-scoped and template-complete. All 5 file targets verified against the repo. The sole blocker is missing backing notes in pal-e-docs for the traceability triangle: - **[SCOPE]** Create user story note `story-westside-ai-assistant-read-ops` (referenced on project page, does not exist) - **[SCOPE]** Create user story note `story-westside-ai-assistant-write-ops` (referenced on project page, does not exist) - **[SCOPE]** Create architecture notes (`arch-domain-westside-ai-assistant`, `arch-dataflow-westside-ai-assistant`, `arch-deployment-westside-ai-assistant`) — project page links to them but none exist. The `arch:A2` board label has no clear mapping to these named notes. These are project-level gaps affecting all items on `board-westside-ai-assistant`. Once resolved, this ticket is READY.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
forgejo_admin/westside-ai-assistant#16
No description provided.