← All articlesPublished on August 30, 2026• Written byShubham Chavan

Sluely: The Always-On AI Assistant That Never Leaves Your Machine

There's a category of AI tool that quietly exploded over the last eighteen months: desktop apps that sit in the background, watch your screen, listen to your calls, and surface help the moment you need it. You've probably seen one in a screen share. You've definitely seen the marketing: a translucent overlay whispering the perfect answer during an interview, a sales call, a negotiation. It's a genuinely useful category. It's also, almost without exception, built on a trade you're never quite asked to consent to: convenience for visibility.

Every screen you share with one of these tools, every word said on every call, every document you have open while it's running: all of it gets shipped to somebody else's server to be processed. Maybe it's deleted afterward. Maybe it's retained "to improve the product." Maybe it's fine. But you don't get to know for sure, and you definitely don't get to change it.

Sluely exists because that shouldn't be the only option on the table.

This post is a long one, deliberately. I wanted to write down, in full, what Sluely is, why it's built the way it is, how the pieces actually work under the hood, who it's for, and where it's going. If you only want the short version: Sluely is a local-first, always-on AI assistant that runs on your machine, powered by Ollama, built around one non-negotiable premise: the assistant that watches your screen and listens to your meetings all day should be the one piece of software on your computer you never have to think twice about trusting.

Everything below is the long version.

Table of Contents

1. The problem with the current always-on AI category
2. What "always-on" actually means, and why most tools aren't
3. The core idea behind Sluely
4. Architecture: how Sluely actually works under the hood
5. Why local, why Ollama — the technical case
6. Why local, why Ollama — the philosophical case
7. Choosing a model: a practical guide
8. Feature walkthrough
9. Who Sluely is actually built for
10. Privacy and security, in detail
11. Performance, latency, and hardware realities
12. How Sluely compares to the cloud-based competition
13. What's live today vs. what's on the roadmap
14. The bigger argument: why local models are ready now
15. Frequently asked questions
16. Getting started

1. The problem with the current always-on AI category

If you've spent any time looking at this space, you've noticed the pattern. A wave of tools launched over the last year or two, all converging on roughly the same idea: capture what's happening on your screen and in your audio, run it through a large language model, and surface the answer in an overlay before the moment passes. Some are aimed at interviews. Some are aimed at sales calls. Some try to be more general "meeting copilots." The framing differs, the underlying mechanics are nearly identical.

And to be fair, the mechanics work. Watching a screen and listening to audio in real time, then feeding that context to a capable LLM, produces genuinely useful, genuinely fast help. That part of the pitch isn't hype: it's real, and it's the reason this category has grown as quickly as it has.

The part that doesn't get talked about enough is what has to happen for that convenience to exist: your screen and your audio have to leave your machine. Almost every tool in this category is built on a cloud pipeline. The app on your desktop is mostly a thin capture-and-display layer; the actual thinking happens on someone else's infrastructure, using someone else's model, under someone else's data policy.

For a lot of use cases, that's a fine trade. Nobody's especially precious about a note-taking bot summarizing a routine standup. But "always-on" changes the calculus. An always-on assistant, by definition, isn't just present for the meetings you'd be comfortable recording: it's present for everything:

• The technical debugging session where you paste in a production database credential without thinking about it.
• The candid strategy call where you say something you'd never want logged anywhere.
• The moment you have a document with someone else's personal information open in another tab.

An occasional-use tool can get away with a permissive privacy policy because you only invoke it occasionally. An always-on tool that's watching everything, all day, doesn't get that same grace: and most of the current crop of tools in this category are built as if it does.

There's also a quieter, more mundane problem: cost and control. Cloud-based always-on tools are metered by nature: a subscription, a per-minute cap, a tiered plan that gates "unlimited" behind the most expensive option. That's a reasonable business model for a company running inference on GPUs it's paying for. But it also means the tool you're relying on all day, every day, has a marginal cost attached to every single interaction, and that cost structure inevitably shapes what the product is allowed to do. Rate limits. Session caps. Features locked behind a plan tier. None of that is malicious: it's just what happens when the intelligence lives on someone else's meter.

Sluely was built to remove both of those constraints at once: nothing leaves your machine, and nothing is metered, because the model runs where you run it.

There's a third, less obvious problem worth naming too: dependency risk. When the intelligence behind a tool you rely on all day lives entirely on someone else's servers, your ability to keep using that tool is contingent on that company's continued existence, continued willingness to serve your use case, and continued pricing decisions. Companies in this space get acquired, pivot, shut down free tiers, or get priced out of their own infrastructure costs and pass that along. A tool you've built real workflow dependency on, that could disappear or change unrecognizably with a single pricing-page update, is a fragile foundation for something meant to be "always-on" in your working life.

2. What "always-on" actually means, and why most tools aren't

Here's something worth being precise about, because the phrase "always-on" gets used loosely in this space. Most tools that call themselves always-on are actually reactive, session-based tools that happen to be quick to invoke. You open the app, it starts a session, it watches and listens for the duration of that session, and when the session ends, so does its context. The next time you open it, it starts from zero. That's a fast, low-friction tool: but it isn't actually "always on" in any meaningful sense. It's "quick to turn on."

Real always-on behavior is a different design problem entirely. It means:

• Context persists across sessions, not just within them. If you were debugging a specific issue this morning and you come back to it after lunch, an always-on assistant should still have that thread, not force you to re-explain everything.
• The assistant doesn't require an explicit "start." A tool that's genuinely always-on is running quietly in the background the whole time you're working, ready to be invoked at any point.
• It generalizes across contexts, not just one. A tool built specifically for "interview help" or "sales calls" is a specialized session tool wearing an always-on label. A genuinely always-on assistant needs to be equally useful whether you're in a meeting, reading documentation, debugging code, or drafting an email.
• It can be proactive, not just reactive. Most session-based tools only ever respond to a direct trigger. An assistant that's actually present throughout your day has the opportunity to notice things without being asked: that you've hit the same error three times, that a meeting is running over a topic you flagged earlier, that a document you're reading contradicts something from a call last week.

This is the gap Sluely is built to close. Not "a faster session tool," but software that's actually present across your day the way a good human assistant would be.

3. The core idea behind Sluely

Strip away the feature list and the core idea behind Sluely is simple: run the same category of "watch, listen, help" assistant that's become popular over the last year, but flip the deployment model. Instead of a thin client talking to a cloud API, Sluely runs its inference locally, through Ollama, on your own hardware.

That one architectural decision cascades into almost everything else about the product:

1. Zero per-token cost: No artificial rate limits designed around margin protection.
2. True continuous background execution: There is no meter running while it sits idle.
3. Hardware-tailored model selection: Pick models based on your own GPU and preferences, not vendor profit margins.
4. Absolute on-device privacy: The answer to "where does my screen and audio go?" is simple: nowhere.

None of this is a rejection of what cloud-based tools do well. The overlay-based, screen-aware, real-time-answer pattern is genuinely good UX, and Sluely adopts that pattern deliberately. The disagreement isn't with the interaction model: it's with where the thinking happens.

4. Architecture: how Sluely actually works under the hood

It's worth walking through the actual pipeline, because "runs locally via Ollama" undersells how many moving pieces have to work together for an always-on assistant to feel instant rather than clunky.

[Screen Capture] ──> [OCR / Change Detection] ──┐
                                                 ├──> [Structured Context Buffer] ──> [Trigger Layer] ──> [Local Ollama Inference] ──> [Low-Friction Overlay]
[System Audio]   ──> [Local Speech-to-Text]   ──┘

4.1 Capture layer

Sluely's capture layer runs continuously in the background and has two inputs: your screen and your system/microphone audio.

• Screen capture: Samples what's on screen, extracts relevant content through OCR for text-heavy windows or lightweight change detection when static. Keeps a rolling representation of what is currently in view rather than archiving heavy video frames.
• Audio capture: Pulls audio directly from your machine's audio stream. No bot joins the call, and no integration per meeting platform is needed. Whatever plays through your speakers or mic is accessible locally to the pipeline.

4.2 The context buffer

Raw screen and audio data needs distillation before reaching the language model. The context buffer maintains a rolling, structured window of recent activity:

• Separates current screen content from recent speech transcripts.
• Tags semantic context (e.g., "terminal session", "code editor", "meeting conversation").
• Weights recent events higher than older content while constantly pruning stale data.

4.3 Trigger layer

Not every moment needs a response:

• Active triggers: Direct invocation via global hotkey or typed prompt, responding immediately against the current context buffer.
• Passive triggers: Watches for specific signals: repeated compiler error messages, long conversational pauses, or conflicting statements. The threshold for passive surfacing is intentionally conservative to prevent interruption fatigue.

4.4 Local inference via Ollama

The trigger passes the context slice to your locally running Ollama instance. No internet round-trip, no per-token billing, and no reliance on third-party uptime. The output streams back locally onto your screen.

4.5 Overlay and output

Output surfaces through a lightweight, non-modal overlay designed for peripheral awareness. It sits at the edge of your screen without taking focus or disrupting your primary task.

5. Why local, why Ollama — the technical case

Setting philosophy aside, the technical advantages of running on Ollama over a hosted API are clear:

• No marginal cost per interaction: Run continuously without watching a billing dashboard.
• Zero network dependency for the core loop: Flaky internet or offline flights don't break your assistant.
• Zero data residency ambiguity: Eliminates compliance and NDA risks when dealing with proprietary source code or private customer details.
• Model flexibility: Swap freely between ultra-fast 7B/8B models for real-time speed and 32B+ models for deep reasoning.
• Predictable latency: Performance is determined by your hardware, not cloud provider traffic spikes or throttling.

6. Why local, why Ollama — the philosophical case

An assistant watching your screen and listening to your microphone all day is fundamentally one of the most invasive software categories imaginable. That level of access demands an uncompromised standard of trust.

"Trust us, we have a privacy policy" is insufficient for continuous background capture across your entire professional life.

When the answer to "Where does my data go?" is "It never leaves your machine," that is not a policy promise: it is an architectural fact.

Furthermore, local execution guarantees true software ownership. A remote provider can pivot, hike prices, or deprecate APIs, but software running on your own hardware stays under your control permanently.

7. Choosing a model: a practical guide

Model selection with Sluely is fully customizable via Ollama:

8. Feature walkthrough

• Live, in-context answers: Grounded in your immediate screen and audio context.
• Automatic meeting notes: Captures key decisions and action items with zero manual typing during calls.
• Zero-bot audio capture: Operates invisibly on your local audio stream without awkward call invites.
• Low-friction overlay: Non-intrusive UI designed for rapid glanceability.
• Full model autonomy: Change your model anytime with a single ollama run command.
• Completely unmetered: No subscriptions, no usage caps, no tier gates.
• Full offline support: The core loop functions seamlessly with no internet connection.

8.5 Design philosophy behind the overlay

The overlay is built on restraint:

• Positioned peripherally to avoid obstructing active windows.
• Dismissible instantly with zero penalty.
• Concise, scannable responses matched to the moment.
• Conservative passive triggers to ensure zero annoyance.

9. Who Sluely is actually built for

• Engineers & Developers: Automatically grasps terminal outputs, error stacks, and IDE context without endless copy-pasting.
• Founders & Operators: Holds unified context across investor updates, product specs, and customer calls.
• Busy Professionals in Back-to-Back Meetings: Delivers instant summaries and action items privately.
• Researchers & Students: Cross-references complex documents and papers live on screen.
• Privacy-Conscious Builders: Anyone who loves the vision of an always-on copilot but refuses cloud surveillance.

10. Privacy and security, in detail

• What stays local: Screen capture, OCR, audio feeds, context buffers, and LLM inference all remain strictly on-device.
• What you control: Model parameters, prompt rules, trigger thresholds, and data retention.
• Legal responsibility: Recording consent laws apply regardless of processing location; users must adhere to local regulations when capturing multi-party calls.
• Architectural security: True privacy enforced by absence of network transmission rather than written terms of service.

11. Performance, latency, and hardware realities

Local execution creates honest hardware dependencies:

1. GPU Acceleration: Discrete GPUs (Nvidia RTX 3000/4000 series, Apple Silicon M-series) provide near-instantaneous streaming.
2. Audio Transcription Speed: Local speech-to-text efficiency is critical for maintaining real-time responsiveness.
3. Compute Allocation: Active generation utilizes local resources during inference while resting light during idle monitoring.

12. How Sluely compares to the cloud-based competition

12.5 Tradeoffs at a Glance

12.6 Common Objections Answered

"Won't a local model be worse than frontier cloud models?"
For everyday context grounding, meeting notes, and live assistance, modern 8B–14B models perform exceptionally well while safeguarding your privacy.

"Isn't setup more work?"
Installing Ollama takes just a few minutes, a small tradeoff for permanent data privacy and zero monthly fees.

13. What's live today vs. what's on the roadmap

• Live Today: Real-time screen sampling, on-device audio stream capture, structured rolling context buffer, local Ollama streaming, and lightweight overlay.
• On the Roadmap: Dynamic task mode-switching (coding vs. calls), multi-day persistent episodic memory, and expanded proactive triggers.

14. The bigger argument: why local models are ready now

Two years ago, running useful models locally on consumer hardware was impractical. Today, open-weight models (Llama 3, Qwen 2.5, Mistral) rival previous frontier models on standard reasoning tasks.

"Always-on" no longer has to mean "always uploaded."

Because inference runs on hardware you own, Sluely has no subscription fees, no token quotas, and no platform lock-in.

15. Frequently Asked Questions

16. Getting started

Sluely is actively being built and refined. The core loop — private, on-device, and unmetered — is live and functional.

If you'd like early access, want to follow development updates, or discuss the architecture, get in touch on my personal website.