The hardest part of a proactive assistant is not generating more. It is knowing when not to speak.

At a glance

  • What it does — reads signals from a user's email, calendar and workspace, works out whether anything is genuinely worth their attention, and only then says something.
  • Shape — a pnpm/Turbo monorepo with four deployable pieces: apps/mobile (React Native), apps/server (TypeScript and Node over MongoDB and Redis, seventy-five domain modules), apps/ai (Python, FastAPI, LiteLLM, twenty-nine capability families) and apps/web (Next.js on Cloudflare Workers — the public site at hellolila.app, in sixteen languages), plus packages/shared, character-core and design-tokens for the contracts and the character between them.
  • Scale — approximately 9,500 commits since February 2026 (9,550 on the main branch at the 4 September 2026 count), single-author across the whole repository. That count is mine, self-counted in a private repository I own, and nobody else has verified it.
  • Enforcement point — one delivery gate, twelve ordered checks, every rejection recorded with the specific reason it was rejected for.
  • Measurement — an evaluation harness that scores restraint as its own dimension, running in CI alongside unit, integration and chaos tests.

The product problem

A proactive assistant has an asymmetric cost function. Surfacing something useful earns a little trust. Interrupting at the wrong moment loses a lot, and users do not give a second chance to a notification stream they have already learned to ignore. Once attention has been trained away, it does not come back.

The common design generates candidate insights and then filters them against a threshold. That fails twice. The threshold is a single scalar standing in for many different reasons to stay quiet, so it cannot express "not this person", "not at 03:00", or "not in week one". And nothing records why anything was suppressed, so the suppression behaviour — the most important behaviour in the product — is the one part of the system that cannot be improved.

LILA separates the two questions completely. Is this worth saying is a reasoning problem. Should it be said now, to this person, on this surface is a policy problem. They live in different places in the codebase, and only one of them involves a model.

What I owned, and what I did not

I am the sole author. Every architectural decision, every line of the gate, the context assembler, the memory promotion path, the gateway and the evaluation harness is mine, and I can defend or be argued out of any of them.

What I did not build is the intelligence itself. Inference is bought: OpenAI, Anthropic and Gemini for language, and hosted providers for speech. The system's contribution is not the model — it is the routing, restraint, budgeting and governance around a model, which is where the product behaviour actually comes from. Stating that plainly matters more here than on either of my employed projects, because "I built an AI assistant" is the easiest sentence in this industry to say and the least informative.

System view

The boundary the diagram is drawn around: apps/server owns decisions and state, apps/ai owns inference and holds no domain state.

The server holds everything that has to be reasoned about later: the signal bus and correlation, the decision engine, governance, the gate, layered context assembly, memory, and the queues, timeouts and retries underneath them. The AI service holds no domain state at all. Every model request enters it through one path — a router that maps a capability such as summarize to a model tier, a fallback chain that walks a sequence of models, a per-provider circuit breaker that tells the chain which vendors to skip, and a cost tracker that attributes each request back to a user.

Because the inference service is stateless, it can be scaled, throttled, failed over or given a different provider mix without product logic changing. The cost of that is real: two languages, two deploy surfaces, a shared contracts package to keep in step, and a network hop on every model call.

The module map, briefly

The two modules that carry the architecture are proactive — the delivery gate, suggestion bundling, decision logging and push throttling — and proactive-intelligence, which holds the signal bus, correlation, the decision engine, email ingestion and the metrics over all of it. Around them: context for layered assembly, token budgeting and cache-prefix hashing; memory for episodic storage, entity resolution, retrieval, promotion and reactivation; governance for agent budgets, conflict detection, escalation and procedural learning; graph for a knowledge graph over entities and relations; rag, continuity, detection and workflows; mcp for tool use over the Model Context Protocol; byok for users who supply their own provider credentials; cost-dashboard for per-user inference cost attribution; and privacy and compliance for consent, deletion and data-handling guarantees. Underneath sits the infrastructure layer: AI clients with tracing and token counting, queues, caches, Redis, budgets, monitoring, migrations and the reliability primitives below.

Capabilities behind a registry, not endpoints

The Python service exposes its twenty-nine capability families behind a registry rather than as a set of ad hoc endpoints — among them chat, email_triage, pattern_detection, summarize, extraction, rerank, embedding, transcription, tts, moderation, memory_extract, contextual_preamble, daily_planning and web_search, plus generation capabilities for briefings, evening and weekly reflections, morning rituals and guided meditations.

Each capability declares what it needs and which model tier it wants. That is the mechanism by which model selection, cost tracking, fallback and observability are handled once, in the gateway, rather than reimplemented per feature — and it means adding a capability does not involve touching reliability code. Feature code asks for summarize; it never names a model.

What happens when a provider fails

Fallback and circuit breaking are designed together, because they only work as a pair. A chain that retries a provider already known to be failing is not resilience, it is a latency multiplier that turns one vendor's outage into a slow outage for everyone; the breaker's job is to tell the chain what to skip. Provider identity is parsed from the model string, so a single chain can span vendors — a Gemini model, an OpenAI model and an Anthropic model in sequence — with the breaker tracking each vendor separately.

Below the gateway the primitives are deliberately boring: bounded queues so a slowdown cannot become an outage through unbounded memory growth, explicit per-call timeouts rather than unbounded waits, retries that the breaker gets a say in, and defined reduced-capability paths instead of exceptions escaping to the user. The intent is that a provider having a bad afternoon degrades one capability rather than taking the product down — and that degradation is a state the system knows it is in, rather than silently worse output nobody noticed.

One pipeline, one gate, two exits — and the reject exit is the one that is instrumented.

Heterogeneous events normalise into a common signal shape, which is what makes adding a fourth source cheap. Correlation then groups related signals across source and time: a meeting that moved, the thread about that meeting, and the document edited just before it are one situation rather than three, and grouping first is the only thing that stops the system emitting three notifications about one event. Pattern detection mixes deterministic rules with a model capability — rules for what is cheap and certain, the model for what needs reading intent — and returns a claim, a confidence and the evidence it rests on. Governance sits above individual suggestions with agent budgets, conflict detection, escalation rules and procedural learning. Everything then meets the gate.

The feedback loop closes at the gate rather than at generation. Dismissals and engagement feed a behaviour profile that the gate reads on the next decision, which means the system learns when a particular user wants to hear from it separately from what it should notice. Those are different questions and they improve at different rates: what is worth noticing is a property of the signals, while when to say it is a property of one person. Feeding dismissals back into generation would confuse the two and make the model quieter about the wrong things.

Decisions that shaped it

Splitting decisions and state from inference

The server owns decisions and state; the Python service owns inference and holds nothing. The alternative — one service, with model calls wherever they are needed — is faster to build and much more common.

The trade-off I accepted was a second language, a second deployment, a contracts package that must be kept honest, and cross-process latency on every call. What it bought is that provider strategy, rate limits, cost attribution and reliability behaviour all live in one place, and none of the product logic knows which vendor answered.

The direction of the split follows the ecosystems rather than a preference. Most of the code is domain logic, and TypeScript with a shared contracts package serves a React Native client better than a Python server would; the inference tooling, on the other hand, lives more naturally in Python. Splitting along that line means neither side is written in the language that suits the other.

One gate, twelve ordered checks, reason-coded rejections

Every suggestion passes through a single decision point, in a fixed order: an engine kill-switch, then consent, profile enabled, operating mode, global snooze, per-type mute, intensity, quiet hours, trust ramp, dismissal cooldown, adaptive timing, daily budget. Adaptive timing is the one check that does not reject: if a user has historically dismissed suggestions in this hour of the day, and the sample is large enough to trust, the gate strips the push surface and lets the suggestion wait in the feed instead. Rejection returns the specific reason and logs it. Allowance returns the set of surfaces — feed, push, chat or voice — the suggestion should be routed to.

Three things made this worth the chokepoint. It is the only way to answer "why was the user not told", because with checks scattered through a pipeline the answer is "somewhere, something returned false". Ordering becomes a visible design decision rather than an emergent one: consent must precede everything, and quiet hours must precede the daily budget, or an urgent overnight item spends budget it was never eligible to spend. And a new rule is one insertion — the dismissal cooldown was added long after the original design and cost a single change rather than an audit, and the Redis-backed engine kill-switch that now precedes consent, added in July 2026 as a production hard-stop, cost exactly the same.

The check I would defend hardest is the trust ramp: for the first seven days a new user only hears from the system at high priority. A proactive assistant is at its least calibrated exactly when the user is deciding whether to trust it, so being quiet early buys permission to be useful later.

The cost of one gate is that it is a chokepoint everything must pass, its ordering is maintained by hand, and it grows.

Context assembly as an explicit, deterministic allocation

A model call has a fixed token budget and there are always more plausibly relevant things to include than will fit. LILA treats that as an allocation problem across five retrieval layers — session, conversation, memory, graph and RAG — with ratios chosen per use case: a chat turn weights conversation and memory, a briefing weights memory and retrieval, and a speech-to-speech voice turn takes none of them because latency dominates depth.

Two properties are deliberate. Allocation is deterministic: where two layers tie for a residual token, the tie breaks by position in a declared array rather than by map iteration order, so the same inputs produce the same context every time. And rendering is cache-aware: context renders into stable blocks with a cache-prefix hash, so the prompt prefix stays byte-identical between turns and provider-side prompt caching actually hits instead of missing on invisible whitespace. That needs a separate cache adapter for each of the three providers, because their caching contracts differ materially.

Determinism was a requirement rather than a nicety because non-reproducible context makes the evaluation harness meaningless — you would be measuring noise. The ratios themselves are hand-set defaults, and the table in code says so in a comment rather than presenting them as derived.

Memory that is promoted rather than accumulated

Episodic memories are candidates, not records. Entity resolution collapses the same person or project referenced three different ways into one entity; a promotion path moves durable facts into long-term storage; a reactivation path brings dormant memories back when they become relevant again. Memory that only grows is memory whose retrieval quality decays as it grows.

The retrieval change that fed this was rolled out in shadow mode: the RAG memory adapter ran against the live path, produced results and logged them while changing nothing a user could see, until the results justified promoting it to serving traffic. It is slower than shipping the change, and it is the pattern I would use for any retrieval change again.

Making silence measurable

Prompt changes feel like improvements, and "the output seems better" is worth very little — particularly for a proactive system, where the most important behaviour is one you cannot see by looking at outputs.

The harness is a small amount of machinery holding a lot of judgement: datasets for pattern detection (schedule, social, productivity and edge cases) and for insight wording (tone safety and privacy); written rubrics for pattern detection, insight wording and confidence calibration; a runner that executes a dataset against the AI service and scores it against its rubric; a judge model for the text-quality dimensions; and pytest fixtures so all of it runs in CI. It runs three ways — a dry run that validates datasets without making any model calls, a single dataset in isolation, and a single insight judged inline against a named rubric while iterating on wording.

The harness scores pattern detection against a five-dimension rubric: accuracy, confidence calibration, evidence, product truth and silence. Product truth catches accurate observations the product should not make; a pattern can be real, well-evidenced and still something a user would find intrusive to have been noticed. Silence asks whether, where nothing was worth surfacing, the system correctly surfaced nothing.

Silence cannot be scored with ordinary datasets, because an input-to-expected-output pair cannot express "and here, correctly, nothing happened" — so those cases never get written, and the metric quietly rewards a system that talks too much. The datasets are built from the opposite direction: situations that look like they contain a pattern but do not, or contain one too weak or too personal to act on. The expected output is nothing, and the system is penalised for speaking.

That changed the product. Restraint stopped being an implicit hope in a prompt and became a behaviour with a number attached, which is the only reason I trust the gate's thresholds at all.

The product, in hand

Three screens from the React Native client, captured from a development build — the "Loading from Metro" banner along the top is the React Native dev server, left in deliberately rather than cropped out.

Left to right: the Today screen, where the four entry points are spoken instructions rather than forms; the model settings, where simple messages can run entirely on the device against a catalogue of local models with explicit RAM budgets; and the Skills screen, whose footnote says the quiet part out loud — more skills means more memory. Tap any screen to enlarge.

Where it stands

As of September 2026 LILA is a store-ready product whose release is being held on purpose. The public site, hellolila.app, is live in closed beta in sixteen languages with an early-access request rather than a download button. On Apple's side a 1.0 version sits prepared in App Store Connect with twenty-one builds uploaded through August; on Google's side the listing is written and builds run on the internal track. Pricing — a free tier and a Pro tier with a fourteen-day trial — is applied and verified in both stores across 171 regions. Push runs through a second transport with a proactive journey configured, the data-protection surface is built (right-of-access export, account deletion, sensitive-context opt-ins, recordings transcribed then deleted) and a launch-readiness run on 27 August executed 16,040 tests, of which 15,787 passed and every one of the 203 failures was classified as parked or known debt rather than left unexplained. I have not pressed submit, because the proactive engine and the personal-apps line are not where I want them to be on day one, and a product whose whole thesis is restraint should not launch impatiently.

Three currents of work define its present shape.

The first is a personal-apps line: small applications a user describes in natural language and Lila builds inside itself — records, formulas, timers, photo-to-record capture, automations — under a rule I keep repeating in the code: mini-apps propose, Lila disposes. Every proposal from an app still goes through the same gate as every other suggestion.

The second is the conversational operating layer: voice commands that operate the application itself — an action registry, a catalogue of screens the assistant can navigate and demonstrate, guided tours, interruptible speech — and, underneath it, a research strand I run deliberately dark: a semantic runtime that learns to read any valid Lila capability contract rather than being taught the current tools, measured on sealed sets in a sandbox and switched off in production until it clears the bars I wrote down before training.

The third is a line of work I halted, and the halt is the part worth recording. LILA speaks its users' languages properly — output is grammatically inflected per locale, backed by per-language morphology data that took hundreds of shipped data batches to build, down to Slavic participles. In May 2026 I stopped that programme deliberately: the substrate was ahead of anything user-facing that consumed it, and building deeper into a subsystem nobody could touch yet was momentum, not progress. The ship path took its place, and the sixteen-locale server that shipped in August is the disciplined version of the same ambition — a language sensor that abstains rather than guesses, because every detection is persisted. It is the same lesson as the feature I chose not to ship, applied to my own favourite subsystem.

What is measured, and how

This is a private system built by one person, and it has no published user-facing outcome metrics — so there are none on this page and I will not manufacture any. What exists is the regression signal, and it runs in CI:

  • Dataset integrity — the runner executes every dataset in dry-run mode with no model calls.
  • Pattern-detection quality — datasets executed against the AI service and scored against the rubric above.
  • Insight wording — graded by a separate model against a written rubric covering tone, judgement-free phrasing, privacy, evidence and actionability.
  • Server unit and integration tests — Vitest across seventy-five modules; the 27 August 2026 launch-readiness run executed 16,040 tests across the monorepo.
  • Chaos scenarios — including clock-skew tests, because quiet hours, snooze windows, trust ramps and daily budgets are all time-dependent, and time-dependent logic fails in ways ordinary tests do not reach.

The number I actually watch day to day is not how many notifications were sent. It is the shape of the rejection log: of everything the system considered surfacing, how much was suppressed, and by which rule. That distribution is diagnostic in a way a send count is not. Heavy suppression on quiet hours means detection is running at the wrong time of day rather than that the rule is wrong. Heavy suppression on the daily budget means correlation is under-grouping. Heavy suppression on intensity means confidence calibration is drifting.

Limitations, and what I would change

  • The LLM judge is not calibrated against human raters. It is the weakest part of the harness. Judges drift and are sensitive to rubric wording, so I treat their scores as a regression signal — has this got worse — rather than as an absolute measure of quality. Establishing human agreement on a sample is the obvious next piece of work and it is not done.
  • The context ratios are hand-set. They should be driven by measured retrieval utility per layer. The default table is frozen in code with a comment recording that it is a starting point rather than an optimum, and metrics-driven retuning is scheduled separately. It is the largest piece of unfinished thinking in the system.
  • The gate's daily budget was an in-memory counter keyed by user and date until August 2026 — correct on a single instance, wrong the moment the service scaled horizontally. It now reserves a slot with an atomic Redis increment and falls open to the in-memory map only when Redis is unavailable; the fail-open is a deliberate choice, recorded in the file, and it means a Redis outage briefly loosens the cap rather than silencing the product.
  • Correlation is weak where grouping needs world knowledge. It handles obvious relationships well and misses the ones that require knowing how two things relate in the world rather than in the data. It is the part I would rebuild first.
  • Nothing here has been validated against users. The beta is closed and the store submission is held, so this remains true in September 2026. The gate's defaults — the intensity levels, the seven-day ramp, the shape of quiet hours — are argued from the rejection distribution and from my own use, not from research with people who are not me. A restraint model that is internally coherent can still be calibrated to the wrong person, and I would want that tested before I trusted the defaults for anyone else.
  • Knowing when not to speak — why the suppression model, not the generation model, is the interesting half of a proactive assistant.