Local LLMs Should Be Hardcoded Into Every Tool By Now

You want to hook a local LLM into a tool. Not a cloud API. Not some SaaS wrapper that phones home to three different data centers. Just your llama.cpp server running on localhost, minding its own business, serving completions on port 8081.

The tool in question? A security framework called KittySploit. The LLM? Qwen3.6-27B, quantized to Q6_K_XL, sitting on 131k context. Sounds straightforward.

It wasn't.

The Framework's Idea of "Local"

KittySploit's LocalLLMService had a vision. A beautiful, narrow vision: Ollama. Only Ollama. It spoke the Ollama dialect, hit /api/chat and /api/generate, and assumed every local inference server on earth was running Ollama.

llama.cpp doesn't speak Ollama. llama.cpp speaks OpenAI's REST API. It exposes /v1/chat/completions and /v1/completions because someone at the Ollama maintainers' offices clearly didn't consider that other people build inference servers too.

So here we are. The framework can talk to one flavor of "local." Not the flavor you happen to be running.

Classic.

Oh, And You Can't Authenticate Either

Because if endpoint incompatibility wasn't enough, there was the matter of API keys. llama.cpp's server supports Authorization: Bearer . The framework's LLM service? Didn't bother with headers. Didn't have a place to put a key. Didn't consider that someone might want to secure their inference endpoint.

It's 2026. The default assumption should be that your API needs auth. Not that it's sitting on your LAN, unprotected, hoping for the best.

The Fix Was Five Files

Here's what it took to make this work:

local_llm.py — Added _is_openai_endpoint() to detect /v1/chat/completions. Added _build_headers() to inject Authorization: Bearer. Rewrote query_json() and query_text() to build OpenAI-format request bodies and parse choices[].message.content instead of Ollama's response shape. Kept backward compatibility because apparently Ollama users still exist and we're nice to them.

mcp_kittysploit_bridge.py and workflow_core.py — Pass os.environ.get("KITTYMCP_OLLAMA_API_KEY") to the service. Because of course the env var is still called OLLAMA even though it now works with anything.

kittymcp_client.py and kittymcp_server.py — Added --ollama-api-key CLI flag. Same naming inconsistency. Same reason: don't touch working config names.

97 insertions. 29 deletions. Five files. A PR to the repo.

How You Actually Run It Now

export KITTYMCP_OLLAMA_ENABLED=1
export KITTYMCP_OLLAMA_ENDPOINT="http://127.0.0.1:8081/v1/chat/completions"
export KITTYMCP_OLLAMA_MODEL="qwen3.6-27b-ud-q6_k_xl"
export KITTYMCP_OLLAMA_API_KEY="your-key-here"

python3 kittyconsole.py
agent your-target.com --llm-local

The model boots. The framework connects. The agent runs. No cloud. No egress. No "we use your prompts for model improvement" fine print.

The Point

Every framework that claims to support "local LLMs" supports exactly one local LLM. And it's never the one you're running. Until someone opens a PR, it stays that way.

The good news: opening that PR is usually five files and a weekend.

The bad news: you still have to do it.

Open source tools that support "local LLMs" are like restaurants that say "all you can eat" — technically true, until you realize the buffet only has one food.