When it comes to AI, maybe ChatGPT or Gemini come to mind. There are other players like Perplexity, Claude, Grok and Mistral. In a booming market, there are a whole host of AI models out there, many of which don’t even require an internet connection.
Models that run without internet connections are called local AI models, and as the name suggests, they can be run on your own hardware. You don’t need to connect to OpenAI’s or Google’s servers to use those versions of ChatGPT or Gemini.
This brings both advantages and disadvantages. The major advantage is privacy. With local AI, you can have it analyze sensitive documents or ask it embarrassing questions, all without worrying if Big Tech is watching. It’s also unlimited. As long as you have power, you can continue asking questions, at least until you run out of memory.
I decided to make the most powerful PC I could in the smallest available hardware. Would it have been possible to use a larger server motherboard that can handle much more RAM? Sure. But I doubt that’s how most people would use AI at their homes or small businesses. I essentially opted for the equivalent of a powerful and compact gaming PC. Here are the specs, with approximate retail value:
- AMD Ryzen 9 9950X3D ($660)
- Nvidia RTX 5090 ($2,400)
- MSI MPG B650I Edge Wi-Fi motherboard ($290)
- 64GB Crucial Pro DDR5 RAM ($140)
- 2x 1TB Crucial Gen5 NVMe solid-state drives ($150 each)
- Corsair SF1000 power supply ($270)
- Fractal Design Terra 10.4-liter case ($180)
At a total price of $4,240, this is an expensive rig. But to power some of these local AI models, you need horsepower. Not all models require such expensive hardware. There are smaller models that perform really well that can also run on a decently powerful laptop. This includes OpenAI’s new GPT-OSS, which is essentially a local version of ChatGPT.
I believe that over time, the most powerful models will get more efficient. So, maybe a power-hungry version of DeepSeek R1 in 2025 can run on more modest hardware in 2027.
Read the full article here
