Lens, coaching in the moment
A coach on smart glasses for children with autism: it names the social moment, checks the child's plan and shows one short instruction.
Goals: initiation, reciprocity, regulation. Full instructions. Coach greetings, answering and asking back.
Replaylunch at school · no microphone, free
conversation
the glasses
decisions
Try a lineClef-flash classifies it live
Liveyour microphone · Nova-3 → Clef-flash → plan → cue
heard
the glasses
- clip
- —
- speech
- —
- decide
- —
- cost
- $0.000000
Audio goes to Workers AI. Not stored. Stops after 2 minutes.
What
Lens is a social coach for children with autism, on smart glasses. It listens to a conversation, names the kind of moment, checks the child's own learning plan, and shows one short instruction the child can act on now.
How
- model
- Nova-3 + Clef-flash
- id
@cf/deepgram/nova-3@cf/cloudflare/clef-flash- runs on
- Cloudflare Workers AI, through this site's Worker
- costs
- about $0.0006 per 3 s of audio; replay is free
- Classify. Nova-3 turns speech into words. Clef-flash names the moment with one of six categories from social-skills programs: initiation, reciprocity, reading the room, navigation, regulation and repair.
- Personalise. The child's plan decides if this moment is a goal, and how much help to give. A mastered skill gets no cue.
- Respond. The glasses show what to do, in at most three short lines. The cue gets shorter as the child improves.
Why
Earlier coaching glasses taught children to recognise emotions. The gap the research leaves is acting in context: what to do, right now, in this conversation. Lens puts the research methods into the moment, for one child's plan. The goal is a coach the child stops needing.
Try
- Play the replay on the cloud path, then on the edge path. Watch the delay.
- Switch the plan to "Nearly independent" and replay. Most cues go quiet.
- Type a line a classmate might say: "Can I sit with you?"
Limits
- The learners are made up. Real plans stay private.
- Replay times are the design budgets, not live measurements. In a recorded test on the edge path, cues came 1.4 s after the moment, on average over 17 cues.
- The live mic sends 3-second clips through a copy of the pipeline on this site. The production Lens is separate.
- Lens never coaches a child to hide who they are.
The pipelinedesign budget per step
| step | what happens | where | budget |
|---|---|---|---|
| hear | the glasses send audio every 500 ms; Nova-3 turns it into words, with who said what | edge | 200–500 ms |
| 1 classify | Clef-flash names the moment with a category from social-skills research | edge | about 600 ms |
| 2 personalise | the child's learning plan decides: is this a goal, and how much help to give | Durable Object | under 1 ms |
| 3 respond | a short, literal instruction appears on the glasses, then fades as the child improves | edge → glasses | 10–30 ms |
Moving the loop from the cloud to the edge cut the delay from 1.6–5.8 s to 1.2–3.0 s. The hosted version is SocialLens on Cloudflare. Next, the research moves out of the prompt and into the model's weights: thousands of example moments train a small model that runs on the glasses in under 100 ms. That is the pattern behind Callozum.
Cues that fadefull → short → one word → a dot
- initiation: "Say 'hi' back and their name." → "Hi + name" → "Greet" → "•"
- reciprocity: "Answer, then ask one back." → "Answer, ask back" → "Ask back" → "•"
- reading the room: "They look busy. Wait a moment." → "Wait a moment" → "Wait" → "•"
- navigation: "Ask to join: 'Can I play too?'" → "Ask to join" → "Join" → "•"
- regulation: "Breathe in slowly. Count to three." → "Slow breath" → "Breathe" → "•"
- repair: "Say: 'Sorry, I meant…'" → "Fix it: 'I meant…'" → "Repair" → "•"
The research7 sources
Write to me about the problem you want to fix.
- emailEmail contact@meirdick.com
- linkedinLinkedIn in/mdick
- githubGitHub meirdick
- substackSubstack aistrategyconsultant
- rssRSS /rss.xml