I no longer wish to rent my Rubber Duck

Full disclaimer, I’m neither a good dancer OR a good developer. Then again, that’s not what I’m here for.

Dabbling, dancing and developing

Since mid-2023, I’ve been dabbling in using Large Language Models (LLMs) to help me get things out of my brain. I don’t really use LLMs to come up with ideas — I tried it, they’re terrible at it, but they’ll tell you they’re great and if you can’t tell what good looks like, you’re doomed — but in a similar way a dancer uses a mirror to check their form and shape, or how a developer uses a rubber duck to chat through what they’re going to do.

My work predominantly revolves around creating compelling stories around why brands do what they do, and why their customers should give a shit. A lot of the time, it’s about taking away what isn’t working, rather than adding more to the pile. Sometimes this involves gaining a deep understanding of a business model, a business name or a dive into cultural significance and mythology.

My recent tours of curiousity

Being me, with my brain, I’m painfully curious. So much so, I’ve been indirectly referred to as “idiotic and meaningless” and “positively dangerous” by Convervative MPs. That’s a badge of honour.

More recently, I’ve been filling my brain with Ghanaian Culture (Kente cloth, Theodosia Okoh and flag symbolism), Greek Mythology (bay trees, Daphne, Apollo and transformation/female-empowerment), and true generational artisanal craft and the connections to the landscape and other trades. Next month, I’ll probably have moved onto swimming pools - but that’s another story.

If you’ve spoken to me, you’ll have probably heard me use the “carving an owl from a block of wood” analogy. You ‘just’ take away the bits that don’t look like an owl. Of course, I’m oversimplifying what I do for a living, but you get the picture.

LLMs can serve me the information (along with robust citations, don’t take outputs for facts, kids!) to help me shape my thinking, but it won’t solve it for me. LLMs over-promise and over deliver in entirely the wrong way. AI-generated posters are the perfect example of what Eric Morecambe would call “playing all the right notes, not necessarily in the right order”.

More recently, I’ve been using Anthropic’s Claude to help me do the repetitive tasks, admin and basic code reviews to help keep me focused on what my brain does best. I’ve used it to chat through ideas and hold a mirror up to my thinking or rubber-duck something out. But that’s all about to change.

Switching to local LLMs.

Just over a year ago, I bought an M4 Mac Mini and loaded it up to the gills with RAM (this was before the Great RAM shortage and price explosion), since then, I’ve been testing out different services: Hermes, Pi, OpenCode, Ollama, LLM Studio, etc, to see how I could work locally, rather than sending my thought-processes to a resource-intensive data centre.

I think I’ve finally cracked it, so today the switch-over starts. Here’s what I’ll be setting up and the key differences.

The benefits of a local LLM

Let’s cover some of the benefits in using a local model:

• Privacy — Everything stays on the Mac. Nothing is sent to a server.
• Cost — Free after hardware I already own. No per-token or subscription cost, however much I use it.
• Offline + always-on — Works with no internet, no rate limits, no service outages, no account.
• Control — I own the whole stack: swap models, fine-tune, set my own system prompt and loop in my own tools.

The big payoff here is speed, but I’m not interested in speed - I’m happy to set a few tasks and leave it to run and find something else to do. I just need to get the mundane out of the way, so I can focus on making something unmistakably human. And, with a top power draw of 80w (the power ceiling is 140w, but stress tests usually cap at around 80w for sustained use), I’m using far less power than sending things to cloud computing.

There may be a case for a hybrid approach, but that doesn’t feel like the leap that’s needed, and you know how I feel about the over-reliance on ‘convenience’.

From now, I’ll own my Rubber Duck.

–

Photo by Se. Tsuchiya on Unsplash

Hi! I'm James Kindred. I wrote this!

I have over 25 years of experience in brand strategy, design and technology. My experience in both founding brands and defining meaningful brand strategies for others has given me a unique perspective on how to help businesses grow.