Ollama

While we wait for Apple Intelligence to arrive on our devices, something remarkable is already running on our Macs. Think of it as a locavore approach to artificial intelligence: homegrown, sustainable, and available year-round.

This week on NSHipster, we’ll look at how you can use Ollama to run llms locally on your Mac — both as an end-user and as a developer.

→ nshipster.com/ollama/

Apple introduced Apple Intelligence at WWDC 2024, but its delayed rollout has left its, AI for the rest of us, promise unfulfilled, while Ollama offers a robust solution for running large language models (LLMs) locally on Macs, likened to Docker for LLMs.

Ollama simplifies model management with commands to pull and run models like llama3.2, using Modelfiles for configuration and the OCI standard for distribution, powered by llama.cpp for efficient inference. It provides privacy, cost-free unlimited usage, low latency, and full control, with an HTTP API on port 11434 for integration via the Ollama Swift package, supporting text completions, chat interactions, and embeddings for semantic search.

The Nominate app exemplifies its use, renaming PDFs locally using Ollama, PDFKit, and Apple’s Natural Language framework. Developers are encouraged to leverage Ollama’s open-source models for AI-driven macOS apps, bypassing cloud API limitations and aligning with Swift’s capabilities for immediate innovation.