Local LLMs for HR: Tools, Models and Guides

When I came up with the idea to test local LLMs this summer, I watched a lot of videos and read many blogs. Some were good, others just a waste of time. Below I’ve created an overview of the most helpful and informative ones, that will help you get up and running quickly.

Let’s start with the basics

How do LLMs work? And what is a token? What does inference mean? LLMs come with their own set of jargon. It takes a bit of getting used to, even though it’s not difficult. I found a good explanation in normal language in How Generative AI Works. Start here, so you understand the basics.

How can you run AI for free on your laptop? This video You Can Actually Run AI For Free is an excellent explainer on how to install a local LLM. The site has many more educational blogs and videos that are worth your while. Once you’ve watched this video, you’ll know exactly what to do.

Models

In the final report I wrote about proprietary and open-weight models. I also used the term frontier model a couple of times. Maybe you’ve even read about open-source models. This article provides a good overview of the differences and includes a ranking of open-weight models. LM Studio will suggest the models that fit your hardware, but even so, I recommend to familiarize yourself with the different options first, so you know which one you want.

And a reminder: You can use LLMs to answer questions about working with LLMs. So, if you are getting stuck, just ask the model how to proceed. They typically provide a good answer, or give you the exact code to enter.

Applications

You need an App to be able to work with a model. I had used Ollama before, but didn’t find it intuitive. This time I started with LM Studio and I liked it, also because it has a good number of settings to tune. Here’s the step-by-step install guide I used.

I later added: Atomic Chat, Ollama and OpenWorker but it’s not necessary to start with those and in fact, I would advise that you focus on LM Studio first.

If you prefer a command line interface, use Ollama. I set it up using this article Using Claude Code with Local GLM 5.2: A Complete Guide. The steps are easy to follow and t5he guide also explains how to tune different settings. Add Atomic Chat or OpenWorker only once you are ready to create agents.

If you like the Claude environment, you can also download the Claude Desktop App and run it on your laptop. You don’t need a Claude account. You just change the settings to point the App to work with one of the local models that you’ve already downloaded through LM Studio. There’s a little trick to make it work and here’s how. While I tried it myself, I found that Claude Desktop wasn’t as efficient in using memory as LM Studio, and the results weren’t any better.

Models

After you’ve install the App, you download a model. LM Studio and Ollama let you download models straight from the interface or the command line. LM Studio shows the list of models that will work well – you can still download the others. Atomic Chat and OpenWorker can then use the models you’ve already downloaded. Just point them to the folders where these models are stored. You can also download models from a site called Hugging Face, but it’s not necessary.

LM Studio and other apps often deliver results in *.md files. And since I was going open source anyway, I decided to install OnlyOffice to work with the output files. It’s an office suite with a document, spreadsheet and presentation editor. You can integrate LM Studio and OnlyOffice so you use use your LLM straight in the editor (like CoPilot etc.)

Settings

You can tune many settings to get the models to behave slightly differently. And that can become overwhelming. But some settings are more important than others and this article explains which settings to focus on to get the results you want.

As I wrote in the main article, the context length is an important setting to get right. It determines how much information a model can hold in memory. If the context length is too short, it won’t finish the task. But too much context costs VRAM, and the experience will quickly degrade. What to Do When Your Model Runs Out of Space helped me figure out how to select the optimal setting.

When I came back from vacation, Qwen 3.8 27B was released. I wrote how this model was super capable and made a real difference. But it’s also a model that is known for widely overthinking stuff. In my article, I wrote about the Reasoning setting. You can read here what happens if you don’t set that well.

And if your model worked fine for a while, but then became unruly (it happens), this article has some ideas on where to look to fix that (hint: the context window is king).

Agents

If you want to build an agent, there are programs available that already have the basics. Hermes Agent is one of them and it works with Atomic Chat. This step-by-step explainer talks you through the setup.  Both Hermes and Atomic Chat can work with skills, so that’s why I picked this combo for my (very small) agents.

I first ran my experiments offline, to understand exactly what was going on. But I also wanted to create an agent to find the latest HR news. And for that, I needed to give my laptop internet access. Yes, right now, the news about agents running amok on the internet is everywhere, but trust me, the agents you and I build are simply not powerful enough. If you want your agent to access the internet, here’s how to do it. Warning: it’s quite technical but scroll down to bullet 3 and you’ll get a non-developer setup for integrating web search in LM Studio. Don’t get me wrong: you will still need to know what you are doing, so this one is a bit more technical than the rest.

A few other good articles:

In Using LLMs for Agentic Coding you can read about the arguments for and against using open-weight models. I especially liked the “brain-rot” one: “Weaker models are better for the brain because you have to compensate their lack of intelligence and be more involved.”

The Token Price Index site shows the current price of tokens.

Mozilla compiled a report on the State of Open Source AI. It’s interesting to see how fast Open Source models are catching up in capability – and in usage.

And for a bit of fun: a tracker that shows if AI is profitable yet.