Upgrading to llama.cpp

Fri, 20 Feb 2026 00:00:00 +0000

The first time I tried llama.cpp

llama.cpp was one of the first inference servers available for running an LLM model. At the time, I wasn’t super impressed with AI but they were better than the old fashioned auto complete. I didn’t think a subscription was worth paying for but if I could run my own, i’d take it.

When I first got to the llama.cpp repo on github, running it still required compiling it from source code. Annoying but it wasn’t the end of the world. What actually stopped me was that there wasn’t something like hugging face available to pull models from. The models that were around were the hundreds of billion parameter versions or I could write code to create my own from training data.

Setting up Ollama server with Open WebUI

Sat, 20 Dec 2025 00:00:00 +0000

Ollama is a powerful AI model serving platform that makes it easy to run large language models locally. In this post, we’ll walk through setting up Ollama with Open WebUI in a Docker containerized environment, including GPU acceleration support. Visit Ollama’s website for more information.

What is Ollama?

Ollama is an AI platform that provides an easy way to run large language models locally. It offers:

Easy model management
GPU acceleration support
A clean API for model interactions
Containerized deployment

What is Open WebUI?

Open WebUI provides a web-based interface for interacting with Ollama models, making it easy to use AI models without needing to write code or deal with complex command-line interfaces. Visit the Open WebUI GitHub repository for more information.

Small Language Models on

Upgrading to llama.cpp

The first time I tried llama.cpp

Setting up Ollama server with Open WebUI

What is Ollama?

What is Open WebUI?