Smart autocomplete, document summarisation, semantic search, auto-categorisation, anomaly detection — the AI features with real retention impact vs the chatbots and image generators developers build to impress in demos and users ignore in production.
Every SaaS product built in 2025 or 2026 has added an AI feature. Most of them added a chatbot. The chatbot sits in the bottom-right corner of every page, has a friendly name, and gets used by 3% of the user base in the first two weeks. Then 0.8%. Then the team stops looking at the metric.
The problem isn’t that AI is overhyped. The problem is that developers choose AI features that are impressive to demonstrate and invisible to users in normal usage. A chatbot is impressive in a demo: you type a question, the AI answers, the audience nods. In a real product, your user is trying to complete a task. They’re not going to stop, open a chat window, formulate a question, wait for a response, and then return to what they were doing. That’s three context switches for a feature that, in the best case, saves them thirty seconds.
This post is about the gap between AI features that generate demo applause and AI features that generate retention. The difference is almost always the same: one is a separate surface the user has to opt into; the other is embedded in the task the user was already doing.
The Pattern That Determines Whether an AI Feature Gets Used
Before the feature list: a single principle that predicts almost every usage outcome.
AI features that get used are invisible until they’re needed, then they save the user from doing something they were going to do anyway. AI features that don’t get used require the user to change how they work in order to access the AI.
High adoption: Low adoption:
───────────────────────────────────── ─────────────────────────────────────
Appears in the existing workflow Requires opening a new panel or modal
Saves a step the user was about to take Offers a capability the user wasn't seeking
Improves something already happening Adds something new that requires learning
Fails silently when wrong Fails visibly with confusing output
Faster than the manual alternative Comparable speed to the manual alternative
Run any planned AI feature through this filter before building it. If the feature requires the user to change their workflow to benefit from it, usage will be low regardless of how impressive the capability is.
Features Users Actually Use
Smart Autocomplete and Inline Suggestions
Not search autocomplete — text completion. The AI watches what the user is typing in a form field, document, or message and offers to complete it. The user presses Tab or clicks Accept. The task continues.
The usage pattern is effortless: the suggestion appears, the user evaluates it in a glance, accepts or ignores. No context switch. No workflow change. The worst outcome is that the suggestion is wrong and the user keeps typing exactly as they would have without it.
Where it works best:
→ Subject lines and email drafts
→ Task titles and description fields
→ Comment and response templates
→ Product descriptions with established patterns
→ Code in developer tools (this is proven at massive scale)
→ Form fields where common values are predictable from context
The implementation reality: you don’t need a frontier model for this. Smaller, faster models tuned on your domain data produce better suggestions than large general models and respond in the 50–200ms window that makes the suggestion feel natural rather than laggy. The latency target is under 300ms from keystroke to suggestion. Above that, the suggestion appears after the user has already continued typing and becomes a distraction.
// Laravel endpoint for inline suggestion
class AutocompleteController extends Controller
{
public function suggest(Request $request): JsonResponse
{
$request->validate([
'field' => ['required', 'string', Rule::in(['title', 'description', 'subject'])],
'partial' => ['required', 'string', 'max:500'],
'context' => ['sometimes', 'array'], // surrounding form data for context
]);
// Rate limit: max 10 suggestion requests per minute per user
// Autocomplete fires on every pause — this prevents runaway API costs
if (RateLimiter::tooManyAttempts("autocomplete:{$request->user()->id}", 10)) {
return response()->json(['suggestion' => null]);
}
RateLimiter::hit("autocomplete:{$request->user()->id}", 60);
$suggestion = (new AutocompleteAgent(
field: $request->input('field'),
partial: $request->input('partial'),
context: $request->input('context', []),
))->prompt($request->input('partial'));
return response()->json([
'suggestion' => $suggestion['completion'] ?? null,
]);
}
}
The rate limiting on autocomplete is essential. Autocomplete fires on every typing pause — without a rate limit, a user typing a 200-word description triggers 30–50 API calls. The limit returns null (no suggestion) when exceeded, which the frontend handles gracefully by simply showing nothing.
Document and Content Summarisation
Users upload documents, receive long reports, work through lengthy comment threads, or inherit tickets with 80 comments from three team members who no longer work at the company. They need the gist. They don’t want to read the whole thing to find out if it’s relevant.
Summarisation is the AI feature with the clearest ROI because it directly replaces time the user was visibly spending. Before: 20 minutes reading a 40-page report. After: 2 minutes reading the summary, 5 minutes on the sections flagged as relevant. The time saved is measurable and the user notices it.
Usage patterns with high adoption:
→ "Summarise this thread" button on long comment chains (5+ comments)
→ Document upload → immediate summary in the sidebar
→ "What changed since I last visited" on collaborative documents
→ Meeting notes → action items extraction
→ Customer support ticket → summary for the next agent picking it up
→ Long email thread → "catch me up" before the user replies
The feature doesn’t need to be perfect. A summary that captures 80% of the key points and is wrong about one nuance is still faster than reading the whole document. Users calibrate their trust over time — they learn what the summariser handles well and what needs verification. The trust level needed to adopt the feature is lower than the trust level needed to fully replace reading. Build for adoption first.
// Summarisation agent — triggered by explicit user action
class DocumentSummariser implements Agent
{
use Promptable;
public function __construct(
private readonly string $documentType, // 'ticket', 'report', 'thread', 'email'
private readonly int $targetLength, // 'brief' | 'detailed'
) {}
public function instructions(): Stringable|string
{
$lengthGuide = $this->targetLength === 'brief'
? '3-5 bullet points, each under 20 words'
: '2-3 paragraphs covering context, key points, and action items';
return "Summarise the provided {$this->documentType} as {$lengthGuide}.
Extract and list any action items separately.
If the document is too short to meaningfully summarise (under 200 words), say so.
Use plain language — no jargon unless it appears in the source.";
}
}
Semantic Search
Keyword search finds documents that contain the exact words you typed. Semantic search finds documents that are about what you meant. The user searches for “Q3 revenue performance” and gets results for “third quarter financial results,” “sales metrics July through September,” and “quarterly earnings review” — all of which match the intent without matching the keywords.
Users don’t understand how it works. They just notice that search started returning useful results for queries that used to return nothing. Adoption is passive — the user doesn’t change their behaviour, the feature improves the outcome of the behaviour they already had.
Where the delta between keyword and semantic search is largest:
→ Internal knowledge bases where the same concept has multiple names
→ Customer support ticket history ("the login problem" vs "authentication error")
→ Product catalogs with inconsistent naming conventions
→ Legal and compliance document libraries
→ Any domain with significant jargon or technical terminology
→ Personal note-taking tools (search notes by meaning, not by exact phrase used)
The implementation with Laravel and a vector database:
// Generate embedding on document save
class Document extends Model
{
protected static function booted(): void
{
static::saved(function (Document $document) {
GenerateDocumentEmbedding::dispatch($document);
});
}
}
// The job that generates and stores the embedding
class GenerateDocumentEmbedding implements ShouldQueue
{
public function __construct(private readonly Document $document) {}
public function handle(EmbeddingService $embedder): void
{
$embedding = $embedder->embed($this->document->content);
DocumentEmbedding::updateOrCreate(
['document_id' => $this->document->id],
['embedding' => json_encode($embedding)],
);
}
}
// Semantic search at query time
class SemanticSearchService
{
public function search(string $query, int $tenantId, int $limit = 10): Collection
{
$queryEmbedding = $this->embedder->embed($query);
// pgvector cosine similarity search
return Document::query()
->join('document_embeddings', 'documents.id', '=', 'document_embeddings.document_id')
->where('documents.tenant_id', $tenantId)
->selectRaw('documents.*, 1 - (document_embeddings.embedding <=> ?) as similarity', [
json_encode($queryEmbedding),
])
->having('similarity', '>', 0.75)
->orderByDesc('similarity')
->limit($limit)
->get();
}
}
The 0.75 similarity threshold is the number to tune per domain. Too low and irrelevant results appear. Too high and valid conceptual matches are excluded. Start at 0.75 and adjust based on user feedback about false positives and missed results.
Auto-Categorisation and Tagging
Users hate tagging. They know they should tag their content, tickets, contacts, or transactions. They don’t do it consistently because it’s friction with no immediate reward. The result: a growing corpus of untagged, unsearchable, unfiltered content that gradually becomes less useful.
Auto-categorisation removes the friction entirely. When a ticket is created, it’s categorised. When a document is saved, it’s tagged. The user never sees the process. They see that their content is already organised when they go looking for it.
Where auto-categorisation drives measurable retention:
→ Support tickets → auto-assign to queue (billing, technical, account)
→ Email → auto-label by topic or client
→ Expenses → auto-assign to budget category
→ CRM contacts → auto-segment by behaviour or attributes
→ Content → auto-tag for filtering and discovery
→ Bug reports → auto-prioritise and assign component
The implementation pattern: categorise asynchronously, not in the user’s request path. The user creates the ticket; the categorisation happens in a queued job. The ticket appears categorised by the time the user refreshes or looks at the list view. The AI doesn’t slow down the create action.
class TicketCreatedListener
{
public function handle(TicketCreated $event): void
{
CategoriseTicket::dispatch($event->ticket);
}
}
class CategoriseTicket implements ShouldQueue
{
public function __construct(private readonly Ticket $ticket) {}
public function handle(): void
{
$response = (new TicketClassifier)->prompt($this->ticket->body);
$this->ticket->update([
'category' => $response['category'],
'priority' => $response['priority'],
'component' => $response['component'],
'sentiment' => $response['sentiment'],
]);
// Allow users to correct the categorisation
// Track corrections to improve the classifier over time
TicketCategorised::dispatch($this->ticket);
}
}
The correction tracking is the compounding value: every time a user overrides the auto-categorisation, you have a labelled training example. Over time, the classifier gets better on your specific domain. This doesn’t happen with general AI features — it only happens with features that learn from your users’ corrections.
Anomaly Detection and Smart Alerts
Users don’t monitor dashboards. They look at dashboards when something is already wrong. What they actually want is to be told when something needs attention — when a metric is moving in a way that’s unusual for their specific patterns, when an event happens that’s outside their normal distribution.
The feature looks like this: “Your support ticket volume is 3× higher than usual for a Tuesday afternoon. The spike started 47 minutes ago. The top three tickets mention ‘login error.'” The user didn’t have to check the dashboard. The dashboard checked itself.
Where anomaly detection drives engagement rather than notification fatigue:
→ Revenue or conversion rate departures from weekly patterns
→ Error rate spikes in application monitoring
→ Customer churn signals in user behaviour data
→ Inventory levels approaching threshold faster than usual
→ Queue depth growing beyond normal patterns
→ Response time degradation before it becomes a user-visible outage
The implementation distinction between useful and annoying anomaly detection: the alert has to include context. “Error rate is high” creates alert fatigue. “Error rate for the /checkout endpoint is 12× above the 30-day hourly average for this time of day, having started 23 minutes ago” is actionable. The AI’s job isn’t to detect the anomaly — simple statistical methods do that. The AI’s job is to synthesise the context: what’s unusual, by how much, since when, and what else is happening at the same time.
class AnomalyContextualiser implements Agent
{
use Promptable;
public function instructions(): Stringable|string
{
return 'You are an anomaly analyst. Given a detected anomaly and surrounding context,
write a 2-3 sentence alert that explains: what is unusual (with specific numbers),
since when it started, and what related signals might explain it.
Be specific. Do not use vague language like "significantly" or "notably."
If the context doesn\'t support a cause, say the cause is unknown.';
}
}
Features Developers Build That Users Don’t Use
The Chatbot
The most-built and least-used AI feature. The problem isn’t the technology — it’s the workflow fit. A chatbot requires the user to:
- Notice the chat icon
- Decide they have a question
- Open the chat panel
- Formulate a natural language question
- Wait for a response
- Evaluate whether the response answers their question
- Ask a follow-up if it doesn’t
- Return to their original task
That’s 8 steps for a feature that, in the best case, replaces reading a help article. Users who would complete those 8 steps are the same users who would have found the help article themselves. The users who most need help — the confused, frustrated, unfamiliar users — are least likely to engage with a chatbot.
The chatbot gets impressive demo metrics: session length, messages sent, “AI interactions.” It gets poor outcome metrics: task completion rate for users who interact with it vs those who don’t, or retention difference between users who use it and comparable users who don’t.
When a chatbot actually makes sense:
→ The product IS a conversational interface (AI writing tool, AI research assistant)
→ The use case requires multi-turn clarification before an action (complex configuration)
→ The user population is explicitly seeking conversational guidance (onboarding)
→ There is no underlying UI that could expose the capability more naturally
For most SaaS products, the answer to “can we add a chatbot?” is: what task were you hoping the chatbot would help users complete? Then build the AI directly into that task instead.
The Image Generator
Image generation is genuinely impressive technology. It’s also irrelevant to the task the user was doing in your project management tool, your CRM, your accounting software, or your HR platform.
The usage data from products that added generic image generation: median user generates 1.2 images in the first week, 0 in subsequent weeks. The ones who use it more than once are using it to generate avatars or cover images — a use case that already has better alternatives (Unsplash, dedicated generators, their existing brand assets).
Image generation belongs in products where images are the primary work artifact: design tools, content creation platforms, marketing asset builders. In those contexts, it’s deeply useful. In the product management or SaaS app context, it’s a feature that sounds good on the pricing page and gets used once.
The AI-Generated Report Nobody Asked For
Weekly AI summaries emailed to users who didn’t request them. “Here’s your AI-generated activity summary for the week.” Open rates: 12%. Click rates: 2%. Users who take action based on the summary: 0.3%.
The failure mode: the summary is accurate and nobody cares. The information it contains is either already known (the user was the one who did the work) or not actionable (historical data presented without a prompt to act on it). The AI feature generated output. It didn’t generate value.
The contrast: an AI summary triggered by an event the user cares about. “Your highest-value customer hasn’t logged in for 14 days. Here’s a summary of their last 5 sessions and the last support ticket they raised.” That’s specific, timely, and creates an obvious action. The generic weekly summary is none of those things.
The Feature That Requires Users to Write Good Prompts
Any AI feature that requires the user to understand prompt engineering to get good results will be used by 5% of users and beloved by them. The other 95% will try it once, get mediocre output, and conclude that “AI doesn’t work for this.”
The product’s job is to write the prompts. The user’s job is to provide the input data and receive the output. An “AI assistant” text box where users type free-form requests places the prompt engineering burden on users. They don’t want to learn how to prompt — they want the thing to work.
What this means in practice:
→ Structured inputs not free-form prompts
"Generate a product description" with fields for: product name, target audience,
key features, tone — not a blank text box
→ Guided modes before advanced modes
Button-driven interface for common tasks first
"Advanced" mode for power users who want free-form input
→ Show the output before the prompt
Users trust AI features they see working, then learn to direct them
They distrust AI features they have to learn before seeing them work
The Decision Framework
Before building any AI feature, three questions:
1. Is this embedded in something users are already doing?
→ If yes: low friction, high adoption potential
→ If no: requires workflow change, expect < 10% adoption
2. What does the user do differently today that they won't need to do tomorrow?
→ The answer should be a specific, named task (tag a ticket, search for keywords,
read a full document before knowing if it's relevant)
→ If the answer is vague ("get AI-powered insights"), the value is vague too
3. What happens when the AI is wrong?
→ If the failure is invisible or easily corrected: build it
→ If the failure is embarrassing, data-sensitive, or hard to reverse:
build a confirmation step or don't build it yet
The features with the clearest yes/yes/benign answers: autocomplete (embedded in typing, replaces finishing a sentence, wrong suggestions are ignored), auto-categorisation (embedded in content creation, replaces manual tagging, wrong categories are corrected), semantic search (embedded in search, replaces zero results, wrong results are ignored in favour of better ones).
The features with no/vague/uncertain answers: the generic chatbot (separate surface, replaces an alternative users already know, wrong answers damage trust), the AI report nobody asked for (not embedded in anything, replaces nothing users were doing, wrong summaries erode trust in the product).
What Retention Data Actually Shows
The products that have seen measurable retention lift from AI features — verifiable from public case studies, investor letters, and product teardowns — share three characteristics:
The AI feature is invoked automatically, not by a user action. The user doesn’t decide to use it. It runs when the relevant event happens and presents the result.
The output replaces something the user was going to do manually anyway. Not a new capability. A faster version of an existing task.
The output is short, specific, and contains a number or a named entity. “3 action items” not “several things to do.” “Revenue dropped 18% vs last Tuesday” not “revenue is lower.” Specificity signals that the AI actually processed the data rather than generating a generic response.
The chatbot satisfies none of these. Smart autocomplete, summarisation on long content, semantic search, auto-categorisation, and anomaly alerting satisfy all three. That’s the list.