#!/bin/bash
[rahul@fedora ~/projects]$ cat ai-orchestration-layer.md

AI-Orchestration-Layer

Python · Ollama / Gemini / OpenRouter / NVIDIA NIM · ChromaDB · Electron · Fedora Linux

Selene is a provider-neutral, tool-augmented AI routing and orchestration system written in Python. Rather than wrapping a single model, it sits between its interfaces, a shared agent runtime and several model backends, and it owns the decisions that sit in between: which model serves the current conversation, how a provider-specific request and stream get converted into one internal contract, when a tool should run, how much context is safe to send, and what happens when a route fails mid-turn.

The router supports a managed local Gemma 4 model through Ollama alongside Google Gemini, OpenRouter, NVIDIA hosted NIM and arbitrary OpenAI-compatible endpoints. Each conversation stores a stable model identifier such as local:default or gemini:gemini-2.5-flash, which the runtime resolves through a server-side registry that holds availability, capabilities, credentials and context limits before dispatching to the matching adapter. Every adapter returns normalised content, thinking metadata, tool calls, token usage, stop reasons and user-safe errors, so no other part of the system ever handles a provider-shaped payload. When a chat model fails, the same turn is retried down a bounded fallback chain, with each transition surfaced in the interface instead of failing silently.

On top of routing sits an agentic loop. The model emits structured tool-call requests rather than executing anything itself; a tool runner mediates every call using a registry whose metadata covers side effects, parallel safety, resource weight, cancellation, timeouts, output bounds and platform support. Independent read-only calls run concurrently and are fed back in the original order, while side-effecting and dependency-sensitive chains stay strictly ordered. A local retrieval-augmented generation vault handles long-term document memory — documents are chunked, embedded locally and stored in ChromaDB, with resumable checkpointed indexing for large PDFs and both vector search and exhaustive cursor-based reads.

Context is treated as a managed budget rather than an afterthought: preflight guards reserve output space before every routed call, history is summarised and compacted as usage climbs, and overflow is refused with a controlled warning instead of producing unstable output. The same orchestration core drives three interfaces — a browser UI with SSE streaming, concurrent conversations and collapsible thinking panels, a terminal TUI, and an Electron desktop build. Conversations, credentials, vault indexes and source documents stay on the host throughout, with the local Ollama route as the zero-key, privacy-preserving default.