host ALL your AI locally

Introduction

In an age where AI is becoming increasingly central to various processes, having your own local AI server is a powerful and private solution. This approach allows you to utilize AI models without relying on cloud services, ensuring control over what your family or business can access. In this article, I’ll walk you through how I built a local AI server that not only serves as a personal assistant but is also customizable to suit the needs of my daughters.

Building the AI Server

The journey started as a personal venture but quickly evolved to involve my daughters' education. The AI server, which I affectionately named "Terry," offers a graphical user interface that's feature-rich, including chat histories, model switching, and even support for image generation with Stable Diffusion.

Selecting the Hardware

Building your own AI server can be done on a variety of hardware, but I went all out with a powerful setup:

Case: Leon Lee 011 Dynamic Evo XL
Motherboard: Asus x670 Creator Pro Art
CPU: AMD Ryzen 9 7950 X (4.2 GHz, 16 cores)
Memory: 128 GB of G.Skill Trident Z5 Neo (DDR5 6000)
GPUs: Two MSI Supreme 490s, liquid cooled, with 24 GB of memory each
Storage: Two Samsung 990 Pros (2 TB each)
Power Supply: Corsair AX 1600i (1600 watts)

Despite the labor and initial struggle with Ubuntu, I ultimately decided to install Pop OS, which worked flawlessly.

Getting Started with Local AI

To build your local AI server, you’ll need only a computer, be it a laptop or a desktop running Windows, Mac, or Linux. Having an NVIDIA GPU will enhance performance due to parallel processing capabilities. The first step is to download Olam to run AI models effectively.

For Linux Users: Install Olam using the command line.
For Windows Users: Install the Windows Subsystem for Linux (WSL) to get an excellent Linux distribution running on Windows.

Setting Up Olam

Once Olam is installed, you can test it through your web browser by accessing localhost at Port 11434. After verifying the connection, you can pull and run AI models like Llama 2, which allows for a responsive chat experience.

Building the Web UI

To enhance user experience and allow navigation through different models, setting up a web UI is crucial. One of the best options available is the Open Web UI. You'll need Docker to run it in a container, integrating it with Olam seamlessly.

Install Docker using terminal commands.
Pull the Open Web UI container using Docker and launch it to access a beautifully designed front end.

Model Customization and Control

As an administrator, you have control over which models your daughters can access. You can create custom model files to restrict capabilities further, ensuring educational interaction without the risks of cheating.

Adding Image Generation Capabilities

In addition to text-based models, you can install Stable Diffusion and the Automatic 1111 UI for image generation. This requires a few more prerequisites, but the output quality is impressive and can be accessed through your Open Web UI.

Integration with Note-Taking

Seamlessly integrating AI into your workflow is straightforward. I adapted my local AI server to work with Obsidian, my preferred note-taking application. This allows easy access to the AI assistant right in my notes, adding another layer of utility for my daughters while they study.

Conclusion

By following these steps, you can create a fully functional, private AI server that suits your and your family’s needs. This venture not only protects personal information but also fosters a learning environment that encourages curiosity and education without compromising integrity.

Keywords

Local AI server
Terry
Olam
Docker
Web UI
Model customization
Stable Diffusion
Obsidian
Private AI

FAQ

Q: What is Olam, and why do I need it?
A: Olam acts as the foundation for running AI models locally. It allows interaction with various AI systems without the need for internet access.

Q: Do I need a powerful computer to run a local AI server?
A: While a powerful setup is beneficial for handling larger models, you can start with a standard computer. Ensure it's equipped with an NVIDIA GPU for better performance.

Q: How can I restrict the AI models my children can access?
A: You can create custom model files that specify what the AI can and cannot assist with, ensuring educational use without the chance of cheating.

Q: Can I generate images with my local AI server?
A: Yes! By setting up Stable Diffusion alongside your text-based models, you can create images right on your local server.

Q: How does integration with Obsidian work?
A: You can use a community plugin in Obsidian to connect to your local AI server, allowing you to interact with the AI while taking notes or doing research.