My Private, Free AI Setup

This article is republished with permission from Wonder Toolsa newsletter that helps you discover the most useful sites and apps. Subscribe here.

Short on time? Read this 30-second summary of today’s post.

Download a free, private AI program to run on your computer. Use it offline without any subscription cost and avoid the risk of having sensitive information ingested into a large language model like ChatGPT, Claude, or Gemini. The latest versions of private AI tools like Jan run easily on my 2021 Mac laptop, cost nothing, and are easy to use. They’re a good alternative to costlier AI platforms.

🔰 Quick start guide

1. Download and install Jan for free. Other good free alternatives to consider include Msty, AnythingLLM, or LM Studio.
2. open Jan and pick an open-source large language model. The model you use impacts the AI’s response style. You can switch anytime. I use the v1 model.
3. try your first query. Here are a few quick mini prompts to start with:
   “Summarize the pros and cons of using AI for (specific task).”
   “Turn my rough notes below into a short summary and bullet points.”
   “Turn this angry email draft to my service provider into a constructive message more likely to generate a helpful response.”
4. adjust the app’s appearance settings, including font size and shortcuts.
5. Close other processor-intensive apps on your computer, like video editing tools, to reduce the likelihood of your computer slowing down.

🕵🏻 5 reasons to use private AI

• Save money: Avoid subscription fees by running AI models on your own computer. Generate unlimited responses without monthly charges.
• Keep your data private: Using private AI on your computer ensures no data is sent to or stored on big tech firms’ servers. No conversations leave your device. You can even run these tools offline.
  • For sensitive legal, medical, financial or personal issues, ask questions without worrying about your data ending up in a large language model’s training data.
• Work offline: Having full offline access is handy whether you’re traveling without Wi-Fi, working in a remote area, or hesitant to trust a random public network.
• Experiment with hundreds of open-source models: Choose an open-source large language model that suits you. Each is trained differently. Some are stronger at certain languages, others specialize in coding. New ones emerge regularly. Switch as often as you’d like. By contrast, ChatGPT, Claude, Copilot, and Gemini limit you to the platform’s own models.
  • Tip: use LM Arena to compare two models’ responses side by side.
• Reduce your environmental impact: If you run hundreds of daily prompts, a local AI app may mean less use of internet infrastructure and remote data servers.

💫 Jan is an excellent, free, private AI tool

• Platforms: Mac, PC, Linux

What I like about it

• Fast and easy to set up and use: Jan takes a minute to download and install. Using Jan is as easy as using ChatGPT, Claude, or any other chatbot, although you do have to make an initial decision about which model to use.
• Assistants: Create customized AI helpers for various purposes. One for translating Chinese, another for coding. Task it to “Act as a software engineering mentor focused on Python and JavaScript. Provide detailed explanations with code examples. Use markdown formatting for code blocks.”
• Projects: Organize queries into distinct folders for easy access to subjects of interest without searching through hundreds of threads.
• Integrations: Link Jan to Canva, Todoist, Linear, or other tools using MCP (model context protocol) connections.
• Documentation and resources: Lots of useful documentation, including a handbook and blog.

What’s Next: Jan AI is developing mobile versions for iOS and Android and adding integrations to link Jan to other services.

🩺A Jan case study

Becky Leea senior technical writer, uses Jan to explore health questions she wants to keep private.

“I have a chronic illness I’m struggling to get diagnosed,” she emailed me. “So I created an assistant to help interpret test results and brainstorm possible explanations for my symptoms. Obviously, it’s super important to take this with a grain of salt (a chatbot is absolutely no substitute for a doctorHowever, this helps bubble up conditions I can research further on my own, and it also generates questions I can ask my actual doctor.”

▪ More free AI options for Mac, PC, or Linux

Msty

The free version of this well-designed app has multiple unique features. Unlike Jan, which is completely free, Msty also has paid advanced features.

Its best free features include:

• A built-in prompt library with hundreds of options.
• Special focus and zen modes that strip away side menus.
• Create multiple personaswhich are assistants with distinct personalities. Each can adopt a different style or approach in answering your queries.
• Knowledge Stacks let you import document collections for analysis. These can include PDFs, Word documents, PowerPoints, spreadsheets, lists of YouTube links, or even an Obsidian vault.
• Advanced featureslike multistep automations, require a paid subscription. I’ve only used the free version. It’s easy to use, powerful, and well designed. I chose the Gemma 3.

AnythingLLM

Like Jan, this is a straightforward open-source AI app that’s a good option for novice AI users.