You can run a private AI companion without a recurring cloud-model subscription by keeping both inference and the chat interface on your own computer. The practical design has two separate parts: a local model runtime, such as Ollama or LM Studio, and an interface, such as SillyTavern for character-based conversations or Open WebUI for general assistant chat. Download the software and model while connected, select the local endpoint, prepare any optional local services, then verify the setup after disconnecting the network.
This removes mandatory cloud inference charges, but it is not the same as running for free. You provide the computer, storage, electricity, model files, and maintenance. Privacy also depends on configuration: a locally installed interface can still call cloud models, hosted search, identity providers, or other external services unless you disable them or block outbound traffic.
As an Amazon Associate I earn from qualifying purchases.
What a “local AI companion” actually is
A companion is not a single application. It is a chain of components that can run on one computer:
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →- Model files: the downloaded language model that generates replies.
- Runtime or server: the software that loads the model and exposes an inference endpoint. Ollama and LM Studio are two documented local options.
- Conversation interface: the application that sends messages to the runtime and displays responses. SillyTavern and Open WebUI serve different styles of use.
- Character or system configuration: prompts, character cards, memory settings, and conversation history that shape behavior.
- Optional local services: embeddings and document extraction for retrieval, or speech models and voices for audio conversations.
SillyTavern describes itself as an interface installed on your machine; it still requires an LLM backend. Its character cards define behavior and support persistent character conversations. Open WebUI is a self-hosted interface that can connect to Ollama or another OpenAI-compatible local endpoint. Keeping these layers separate makes troubleshooting easier: a chat window can be working while the model server, document retriever, or speech component is not.
#1 Best Overall
- Solid state performance with up to 800MB/s read speeds in a portable drive. (Based on internal testing; performance may be lower depending on host device, interface, usage conditions and other factors. 1MB=1,000,000 bytes.)
- Back up your content and memories on a storage solution that fits seamlessly into your mobile lifestyle.
- Take it with you on your adventures—up to two-meter drop protection means this durable drive can take a beating. (Based on internal testing.)
- Secure it to your belt loop or backpack for extra peace of mind thanks to the tough rubber hook.
- From Sandisk, a brand professional photographers trust to take on assignments.
Choose the companion experience first
Character-driven conversations with SillyTavern
Choose SillyTavern when the companion’s identity, style, scenario, and long-running roleplay matter most. A character card can contain a name, personality, description, greeting, example dialogue, and behavioral instructions. The card shapes prompts, but it does not contain the language model itself. You must connect SillyTavern to a local backend and select that backend for every conversation.
General assistant chat with Open WebUI
Open WebUI is better suited to a general assistant: questions, writing, coding, and conversations organized like a conventional chat service. It can connect to Ollama and other OpenAI-compatible endpoints. Its offline guidance stresses that every enabled dependency must be local if you want a genuinely offline setup. Local inference alone does not make web search, document extraction, speech, authentication, or another connected feature private.
Desktop-first model management with LM Studio
LM Studio combines a desktop interface with local model management and a local server. Its offline documentation says that chatting with downloaded models, chatting with documents, and serving inference locally can work without connectivity. Searching for models, downloading models or runtimes, and checking for updates still require a connection.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Runtime-first deployment with Ollama
Ollama is a local runtime that downloads and serves models for compatible interfaces. You can use it directly or connect it to Open WebUI and other clients. Ollama’s privacy policy states, “We don’t see your prompts or data when you run locally.” That statement applies to local operation; it does not turn a cloud model or an external tool enabled in your interface into a local service.
| Approach | Best fit | What you manage | Offline considerations |
|---|---|---|---|
| SillyTavern + local runtime | Persistent characters and roleplay | Character cards, chat history, backend connection, model files | Character chat can be offline after the model and all required components are downloaded; cloud connectors must remain disabled |
| Open WebUI + Ollama | General assistant conversations | Ollama model, Open WebUI configuration, optional retrieval or speech services | Each optional service must use local models and prepared dependencies |
| LM Studio alone or as a server | Desktop-oriented local chat and model testing | Model downloads, runtime components, context settings, local server | Downloaded models can be used offline; discovery, downloads, and update checks cannot |
What you need before installing anything
A computer matched to the model
Resource requirements vary with model weights, context length, runtime overhead, and whatever else is running on the host. SillyTavern’s project documentation recommends an NVIDIA 3000-series graphics card with at least 6GB of VRAM for local inference. Treat that as a project recommendation, not a universal minimum or a promise of a particular response speed. A model that fits in memory may still respond too slowly for an interactive companion, especially with a large context or several local services running at once.
Check these variables before choosing a model:
- System RAM and GPU memory: the model and runtime need room, and the operating system and interface need room too.
- Model size and quantization: larger weights generally require more memory. A smaller or quantized model may fit where a larger one does not, with possible differences in quality.
- Context length: longer conversation history increases memory use. A companion that keeps extensive history can require more resources than a short-chat assistant.
- Response-speed target: occasional thoughtful replies tolerate more latency than a spoken, real-time exchange.
- Concurrent services: retrieval, speech recognition, text-to-speech, and document extraction consume additional memory and processing capacity.
Disk space and persistent data
Model files, runtime packages, conversation databases, embeddings, document indexes, and caches all occupy storage. The required amount depends on the models and features you select, so there is no responsible universal SSD capacity to quote. If your internal drive has adequate free space, you do not need another drive. An external SSD is an option only when model files and persistent data do not fit comfortably on the existing disk; verify that your runtime supports the chosen storage path and keep the drive connected whenever the model is needed.
Network access for preparation
Plan one connected preparation phase. You need internet access to install applications, search for and download models, obtain runtime components, and retrieve updates. Once those files are present, basic local inference can be tested without internet access. Downloading first is not a contradiction of privacy: it is the step that moves the model onto your machine.
Recommended Free Tools
Rank #2
- MADE FOR THE MAKERS: Create; Explore; Store; The T7 Portable SSD delivers fast speeds and durable features to back up any endeavor; Build your video editing empire, file your photographs or back up your blogs all in an instant
- SHARE IDEAS IN A FLASH: Don’t waste a second waiting and spend more time doing; The T7 is embedded with PCIe NVMe technology that brings fast read and write speeds up to 1,050/1,000 MB/s¹, making it almost twice as fast as the T5
- ALWAYS MAKE THE SAVE: Compact design with massive capacity; With capacities up to 4TB, save exactly what you need to your drive – from large working files to game data and everything in between
- ADAPTS TO EVERY NEED: Whether using a PC or mobile phone, count on the T7 for extensive compatibility²; It’s a true team player when it comes to heavy-duty application usage or file-saving
- HI RESOLUTION VIDEO RECORDING: Record Ultra High Resolution (4K 60fs) videos directly onto the T7 Portable SSD with your favorite camera or mobile devices; Supports iPhone 15 Pro Res 4K at 60fps video and more³
Step-by-step setup
1. Decide what “companion” means for you
- For a defined personality, roleplay, or a persistent fictional character, start with SillyTavern.
- For a general-purpose assistant with organized chats, start with Open WebUI.
- If you want a desktop application that handles model discovery and a local server, evaluate LM Studio.
- If you prefer a separate runtime that other interfaces can call, evaluate Ollama.
Write down the features you actually need: plain text chat, long memory, document questions, speech input, speech output, or web access. Every additional feature adds another dependency to prepare and another possible route out of the machine.
2. Install the local runtime
- Install Ollama or LM Studio from its current official distribution while online.
- Allow the runtime to store model files in a location with sufficient free space. If you use removable storage, configure and mount it before downloading.
- Confirm that the runtime starts and exposes its local endpoint. The exact address and port depend on the application and your configuration.
- Keep the runtime’s model list local. Do not select a hosted or cloud variant simply because it appears beside local models.
LM Studio can also serve inference locally for another interface. Ollama can provide the backend while Open WebUI supplies the browser interface. Either arrangement is valid; the important check is that the endpoint selected by the interface belongs to the runtime on your computer.
3. Download a model while connected
- Use the runtime’s model browser or documented import process to choose a model compatible with your hardware.
- Account for the model’s weights, expected context size, and any separate runtime or tokenizer downloads.
- Download the files completely before disconnecting. An interrupted download can leave an incomplete or unusable model.
- Send a short test prompt and confirm that the reply is generated by the local runtime.
Do not infer compatibility from the model name alone. Check the model’s memory guidance and then observe whether your computer remains responsive during generation. A model that technically loads may still be unsuitable for a companion if every reply takes too long or causes the system to swap heavily.
4. Install and connect the interface
- Install SillyTavern or Open WebUI while online.
- Open the interface’s connection or provider settings.
- Select the local Ollama, LM Studio, or OpenAI-compatible local endpoint, depending on your runtime.
- Choose the downloaded local model from the provider’s model list.
- Send a test message and inspect the provider label, server address, or status indicator to ensure it is not a cloud model or hosted endpoint.
Open WebUI’s offline tutorial specifically instructs users to select a local model rather than an Ollama cloud model or another hosted endpoint. A locally opened web page is not proof of local inference; the endpoint setting is the proof.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errors5. Add the companion’s personality
In SillyTavern, import or create a character card and review its greeting, personality, scenario, example dialogue, and system instructions. Keep the card understandable and test one behavior at a time. In Open WebUI, use a system prompt or model-specific preset for personality and boundaries. Store conversation history on local storage and decide how much history to include in each request, because a larger context increases resource use.
6. Prepare document retrieval or memory separately
Plain chat does not validate retrieval. If you want the companion to answer from private notes, you need local embedding and document-extraction components, plus a local index. Open WebUI’s offline guidance calls out these dependencies explicitly. Download the embedding model and extraction tools while online, import a small test document, and verify that the answer cites or reflects that document after you disconnect.
Do not treat a conversation database as the same thing as retrieval. Conversation history can be sent back to the language model as context; a document index requires its own embedding and search pipeline. Both must be stored and backed up locally if they contain important or sensitive material.
Rank #3
- Note: Magsafe is not available in this version
- High-speed Data Transfer: Lexar external SSD ES3 supports USB 3.2 Gen 2 up to 1050MB/s read and 1000MB/s write to transfer files fast for more efficient work. (Performance may be lower if not supporting USB 3.2 Gen 2 on Mac and other systems)
- Wide Compatibility: Lexar Portable SSD ES3 compatibility with iPhone 17 series (Not supported on iPhone 14 and older models), Android mobile devices, laptops, cameras, Xbox X|S, PS4, PS5, gaming console, and more
- On The Go: Lexar external solid state drive ES3's thin, stylish, and durable design, weighs 42g and is only 10.5mm thick, making it smaller than a card and easily fits in your pocket. It comes with a Type-C cable for plug-and-play convenience
- Data Safety First: Lexar SSD ES3 includes Lexar DataShieldTM 256-bit AES encryption software to protect files
7. Prepare speech only if you need it
Voice interaction adds speech-recognition models, text-to-speech voices, audio libraries, and often additional system permissions. Download every model and dependency before going offline, then test microphone input and audio output without a network connection. A successful text chat says nothing about whether the speech stack is local. If any voice feature calls a hosted API, it remains a network dependency.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
8. Test the complete workflow before disconnecting
- Restart the computer with the runtime, interface, and persistent storage configured as they will be used normally.
- Start a new conversation and continue an existing one.
- Test the longest context you expect to use, not just a one-line prompt.
- Exercise document retrieval, speech, or other optional features individually.
- Disable the network or disconnect the cable and repeat the same tests.
- Record which features fail so you know which component still expects an external service.
LM Studio’s documentation distinguishes downloaded-model chat from model discovery, downloads, and update checks. Offline testing should therefore include startup, inference, and the optional features you intend to rely on, but not downloading a new model or checking for updates.
How private and offline is the setup?
Local inference is a specific claim
When prompts are processed by a model loaded on your computer, the inference request can remain on that computer. LM Studio states, “Nothing you enter into LM Studio when chatting with LLMs leaves your device.” Ollama similarly says it does not see prompts or data when you run locally. These are vendor statements about local modes, not an independent network audit of every installation or every plug-in.
A local interface can still use remote services
Open WebUI’s documentation makes the boundary clear: “Running the interface locally does not make those services local.” Cloud models, hosted web search, remote embedding providers, external document extractors, telemetry endpoints, authentication services, and plug-ins can all create network traffic. Review each enabled integration instead of assuming that a localhost browser address guarantees privacy.
Offline settings are not a firewall
Open WebUI also warns, “These variables do not block outbound connections.” Configuration flags can disable features, but they are not a network perimeter. If your requirement is that no request leaves the computer, use operating-system firewall rules, container network restrictions, or physical network isolation in addition to disabling cloud features. Test the restriction after configuration rather than relying on a label such as “offline mode.”
Free tools Windows power users keep installed
One-click scans. No signup required.
Privacy checklist
- Confirm the model provider is Ollama, LM Studio, or another endpoint bound to your machine.
- Remove cloud-model options and hosted search tools from the interface.
- Use local embedding, extraction, speech-recognition, and text-to-speech components.
- Review OAuth, analytics, update checks, and plug-ins that may contact external servers.
- Apply firewall or container rules when outbound traffic must be blocked.
- Keep sensitive conversation files, character cards, indexes, and backups on storage you control.
- Recheck settings after software updates, because new integrations or defaults can change the network boundary.
Choosing a model without overbuying hardware
Start with the computer you have and the experience you want. A small model with a moderate context may provide a better conversational experience than a larger model that constantly swaps to disk. Conversely, a writing or reasoning task may justify a larger model if you accept slower replies. There is no single “best” model for every companion, and the cited documentation does not establish a universal model, speed, or memory threshold.
| Decision | What to compare | Practical implication |
|---|---|---|
| Model scale | Weight size and quantization versus available RAM and VRAM | Select a model that loads without exhausting memory; leave capacity for the interface and operating system |
| Context | Expected conversation length and memory strategy | Longer history consumes more resources; summarize or limit history when necessary |
| Speed | Acceptable delay for typed or spoken replies | Voice use generally demands lower latency than occasional text questions |
| Features | Text only versus retrieval, speech, or multiple services | Each added component needs storage, downloads, and processing capacity |
| Privacy | Local endpoint versus cloud fallback | Disable automatic fallback so a failed local model does not silently send a prompt elsewhere |
SillyTavern’s 6GB VRAM recommendation for an NVIDIA 3000-series card is a useful project-specific starting point, not a guarantee that every model or context will run well. OpenClaw’s documentation likewise describes resource use as dependent on model weights, context size, runtime, and other host activity. Use those qualifications when assessing an upgrade.
Rank #4
- MADE FOR THE MAKERS: Create; Explore; Store; The T7 Portable SSD delivers fast speeds and durable features to back up any endeavor; Build your video editing empire, file your photographs or back up your blogs all in an instant
- SHARE IDEAS IN A FLASH: Don’t waste a second waiting and spend more time doing; The T7 is embedded with PCIe NVMe technology that brings fast read and write speeds up to 1,050/1,000 MB/s¹, making it almost twice as fast as the T5
- ALWAYS MAKE THE SAVE: Compact design with massive capacity; With capacities up to 4TB, save exactly what you need to your drive – from large working files to game data and everything in between
- ADAPTS TO EVERY NEED: Whether using a PC or mobile phone, count on the T7 for extensive compatibility²; It’s a true team player when it comes to heavy-duty application usage or file-saving
- HI RESOLUTION VIDEO RECORDING: Record Ultra High Resolution (4K 60fs) videos directly onto the T7 Portable SSD with your favorite camera or mobile devices; Supports iPhone 15 Pro Res 4K at 60fps video and more³
Common failure modes and fixes
The interface opens, but no reply appears
- Check that the runtime process is running and that the interface points to its current local endpoint.
- Verify that a model is downloaded and selected, rather than merely listed in a catalog.
- Read the runtime log for a memory allocation or port error.
- Reduce context length or choose a smaller model if the host cannot load the current one.
The reply is extremely slow
- Watch memory and swap usage while generating.
- Close unrelated applications and disable optional services temporarily.
- Reduce the context window and test a smaller or more heavily quantized model.
- Do not assume a faster graphics card is required until you know whether memory capacity or another bottleneck is the limiting factor.
The companion forgets earlier details
- Inspect the interface’s context and history settings.
- Check whether the character card or system prompt is being applied to the current chat.
- Use a deliberate memory or summary method instead of endlessly increasing context length.
- For private documents, verify that local embeddings and retrieval are actually enabled; chat history alone does not create a document index.
Document questions fail offline
- Confirm that the embedding model and extraction component were downloaded before disconnecting.
- Check that the document index is stored on persistent local storage.
- Test a plain local chat and retrieval separately to identify which service is missing.
- Look for a configured hosted embedding or extraction provider and replace it with a local component.
Voice stops working without internet
- Confirm that speech-recognition and text-to-speech models, voices, and native dependencies are installed locally.
- Check microphone permissions and the selected audio devices.
- Inspect the voice provider setting for a hosted API URL.
- Run a text-only conversation to separate the language-model problem from the audio pipeline.
A prompt still leaves the computer
- Review provider, search, plug-in, authentication, and telemetry settings.
- Look for cloud-model fallback when the local runtime is unavailable.
- Apply firewall or container egress restrictions; application settings alone do not block outbound connections.
- Disconnect the network and watch which feature fails, then remove or replace that dependency.
Cost: no subscription is different from zero cost
A local companion can eliminate recurring charges for hosted inference when every conversation uses your own runtime and model. That does not establish a total saving. You may already own suitable hardware, or you may need a graphics-card upgrade, a desktop computer, or additional storage. Electricity, replacement hardware, downloads, backups, and your configuration time also have value. The available documentation does not provide a reliable hardware-amortization, energy, or payback figure, so a specific savings claim would be misleading.
Think of the trade-off as a cost structure rather than a “free” label:
- Upfront: computer capability, storage, and time spent installing and configuring.
- Recurring: electricity, maintenance, backups, and occasional model or software updates.
- Avoided: subscription fees for cloud inference, provided you do not use cloud models or hosted add-ons.
- Variable: model size, context, speech, retrieval, and the number of simultaneous services.
Buy hardware only after measuring the workload you actually need. A new GPU or computer is sensible when the selected model cannot load or respond acceptably on the current machine; it is not a prerequisite for every local companion.
Maintenance and safe operating practices
Back up the parts that define the companion
Back up character cards, system prompts, conversation databases, document indexes, configuration files, and any locally created memory. Keep a copy separate from the machine, but protect it because these files may contain intimate conversations or private documents. Test restoring a small backup before you depend on it.
Update deliberately
Updates require a connected maintenance window. Read the current release notes, back up first, update one component at a time, and rerun the offline test afterward. A runtime update can change model compatibility; an interface update can add a provider or alter a default. Do not update automatically during a session that must remain disconnected.
Protect the local endpoint
A service bound only to the local machine is less exposed than one listening on every network interface. Use the runtime’s documented bind and access controls, avoid forwarding its port to the public internet, and restrict other devices on your network when the conversations are sensitive. Local processing protects prompts from a cloud provider; it does not protect an openly exposed server from someone who can reach it.
A practical decision checklist
- Define whether you need character roleplay, general assistance, retrieval, speech, or all four.
- Inventory RAM, GPU memory, free disk space, operating-system permissions, and expected response speed.
- Choose SillyTavern or Open WebUI for the interface, then choose Ollama or LM Studio for the local runtime.
- Download the model and every optional local dependency while online.
- Connect the interface to the local endpoint and disable cloud fallbacks.
- Configure the character card, system prompt, history, and persistent storage.
- Test plain chat, long context, retrieval, and speech separately.
- Disconnect the network and repeat the features you intend to use.
- Inspect external integrations and add firewall or container restrictions if no outbound traffic is acceptable.
- Back up the configuration and schedule controlled maintenance updates.
The result is a companion that can operate without a cloud subscription and, when all dependencies are local, without an internet connection during use. Its privacy and performance depend on the endpoint, model, hardware, optional services, and network controls you actually configure—not on the fact that the interface appears in a browser on your own computer.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
