Joe Tom Ryan

some things about me:

  • i'm an ai engineer who ships production llm systems end to end, from architecture and backend code to deployment and reliability in the field.
  • right now i'm a deployed engineer at avoca ai (yc w23), owning enterprise deployments of real-time voice and sms agents.
  • before this i was founding ai engineer at 100x, where i built sally, and i founded medai, computer vision used in 30+ clinics.
  • i also build cortex, an open-source memory layer for ai agents.

experience (click a row for the full story)

  1. jun 2026 – now avoca ai deployed engineer · yc w23 voice + sms agents →
  2. jul 2025 – may 2026 100x founding ai engineer 50k+ conversations / mo →
  3. feb 2024 – jul 2025 medai founder & ai engineer 30+ clinics →
  4. open source cortex memory for ai agents sub-5ms search →

pick a book (or press 1–6)

← back

experience · jun 2026 – present · remoteAvoca AI

deployed engineer · yc w23 · applied ai & platform engineering

i own enterprise deployments of real-time voice and sms agents end to end: agent design, crm integrations, typescript backend features, and the observability and incident systems behind them. i sit between customers and core engineering, turning field failures into scoped tickets, design docs and production changes.

by the numbers

scheduling failure codes 19 a structured failure taxonomy i built, so every failed booking carries a reason you can query, alert on and fix.
crm & scheduling systems integrated 4+
  • servicetitan
  • housecall pro
  • pestpac
  • fieldroutes
  • + others
via apis, adapters and webhooks, with platform features that keep agent behaviour consistent across crms.

a call, end to end

customercall or sms→ voice agentvapi · elevenlabs→ prompts & blueprintsknowledge-base variables→ tool callsbooking logic→ crm writeservicetitan · housecall pro · …

when it breaks

structured telemetry19-code taxonomy→ slis / sloslog-based metrics→ datadog monitorslogs · apm→ dedupe & severitynoise reduction→ incident.iorouting→ root causeexact config or commit

what i do

  • enterprise ownership. own multiple enterprise accounts end to end, from agent build and crm integration to launch and production reliability.
  • llm agent engineering. configure and deploy production llm agents: prompt and blueprint composition, tool wiring, knowledge-base variables and booking logic across the voice platform layer.
  • integrations & platform. integrate agents with crm and scheduling systems, and design platform features that cut manual onboarding work.
  • production debugging. debug distributed failures spanning telephony, the llm, tool calls and crm writes using logs, traces, sql and code-path analysis, and attribute regressions to the exact config change or commit.
  • backend engineering. ship features and fixes in a typescript monorepo (next.js, postgres, inngest), covered by tests and type checks and merged through pr review.
  • observability & on-call. build structured event telemetry, slis/slos, log-based metrics and datadog monitors; design alert dedupe, severity policy and incident.io routing.

stack

  • typescript
  • next.js
  • postgres
  • inngest
  • vapi
  • elevenlabs
  • twilio
  • datadog
  • incident.io
  • sql
← back

experience · jul 2025 – may 2026 · remote100x

founding ai engineer · sally, conversational & real-time voice ai

i architected and shipped sally, a full-stack ai sales agent that talks to leads in real time for 30+ u.s. enterprise clients, and the guardrails that keep it honest.

by the numbers

conversations a month 50K+ real-time sales conversations handled by sally across clients.
u.s. enterprise clients 30+
lift in lead-to-close conversion 38% conversion indexed to 100 before sally.
fewer hallucinations 60% guardrail layers, structured output validation and automated regression testing.
speech-to-speech latency 500–800ms from the moment a caller stops talking to the moment sally starts, with streaming stt/tts and neural vad turn-taking.

one turn of a sally call

caller speaks→ neural vadturn-taking→ streaming stt→ llm orchestrationprompt chaining · tool routing · context windowing→ guardrailsstructured output validation→ streaming tts→ caller hears

the whole loop: 500–800ms.

also at 100x: enterprise automation

delivered enterprise automation on custom mcp servers with multi-model orchestration (gpt-4, claude, gemini) on fastapi and kubernetes, automating lead processing and crm enrichment to lift client arr/mrr.

what i did

  • architected and shipped sally, a full-stack ai sales agent: llm orchestration with prompt chaining, tool-use routing, context windowing and eval pipelines.
  • real-time voice with streaming stt/tts and neural vad turn-taking at 500–800ms speech-to-speech latency, driving a 38% lift in lead-to-close conversion.
  • hallucination-reduction framework: guardrail layers, structured output validation and automated regression testing, cutting hallucination rates by 60%.
  • infrastructure on aws with distributed tracing, ci/cd and docker/kubernetes.

stack

  • python
  • langgraph
  • langchain
  • fastapi
  • streaming stt / tts
  • neural vad
  • mcp
  • gpt-4 · claude · gemini
  • aws
  • docker
  • kubernetes
← back

founder · feb 2024 – jul 2025MedAI

founder & ai engineer · clinical computer vision

computer vision that clinicians actually used: diabetic retinopathy screening and cbct-based dental implant planning, built and clinically deployed across 30+ clinics.

by the numbers

screening accuracy 99%+ diabetic retinopathy screening.
retinal images trained on 50K+ with gan augmentation to widen the training set.
lower per-patient cost 55% per-patient screening cost, indexed to 100.
clinics using it 30+

retinopathy screening

retinal image→ gan augmentation50k+ images→ efficientnetclassifier→ dr grade→ clinician

dental implant planning

cbct scan→ 3d modelvtk→ ml-assisted plan→ dentist

what i did

  • founded medai and built its computer vision from model to clinic.
  • diabetic retinopathy screening with efficientnet and gan augmentation on 50k+ retinal images, reaching 99%+ accuracy.
  • clinically deployed across 30+ clinics, with 55% lower per-patient cost.
  • cbct-based dental implant planning for dentists.

stack

  • computer vision
  • efficientnet
  • gans
  • vtk
  • python
← back

open source · buildingCortex

persistent semantic memory layer for autonomous ai agents

an open-source, mcp-native memory server. agents remember across sessions, link what they learn into a knowledge graph, and consolidate experiences into lasting memory.

by the numbers

search latency <5ms hybrid retrieval over the whole memory store.
steps in memory consolidation 7 episodic memories consolidate into lasting knowledge.

how a memory is found

query→ sqlite-vecdensefts5 bm25sparse→ reciprocal rank fusion→ cross-encoderrerank→ memories<5ms

how a memory is stored

new memory→ spacy nerentities→ jaro-winklerentity resolution→ knowledge graphself-linking→ episodic memory7-step consolidation

stack

  • python
  • fastmcp
  • bge-small-en
  • sqlite
  • sqlite-vec
  • fts5
  • spacy
  • claude agent sdk

this is where my research interests start: what happens when an agent carries its past forward?

← back

chapter iWork

i don't build demos. i build ai systems that run in production.

jun 2026 – now

avoca ai · deployed engineer

enterprise deployments of real-time voice and sms agents for a yc w23 company: agent design, crm integrations, typescript backend and the observability behind it.

read the full story →
jul 2025 – may 2026

100x · founding ai engineer

built sally, a real-time voice ai sales agent: 50k+ conversations a month, 30+ u.s. enterprise clients, 38% conversion lift, 60% fewer hallucinations.

read the full story →
feb 2024 – jul 2025

medai · founder & ai engineer

clinical computer vision in 30+ clinics: retinopathy screening at 99%+ accuracy and 55% lower per-patient cost, plus cbct-based implant planning.

read the full story →
← back

chapter iiProjects

most production work lives in private repos at 100x.inc. here's what i can show.

← back

chapter iiiEducation

b.e. computer science · sir m. visvesvaraya institute of technology, bangalore · 2022 – 2026

i had a good college life: friends, the basketball team, late nights that had nothing to do with exams. but most of my time didn't go into studying for grades. it went into building things real people used.

i believed then, and still do, that you learn more from shipping software and watching it fail than from any textbook. so i spent college building startups and real-time systems, and spending weekends at hackathons.

four years, side by side

half my degree overlapped with running a company or building one.

startups

building, not memorising

i founded medai while in college and took clinical computer vision into 30+ clinics. in my final year i joined 100x as a founding ai engineer and shipped a voice agent handling 50k+ conversations a month. both taught me more than any semester did.

won several

hackathons

a lot of my weekends went to hackathons, and i won multiple of them. win or lose, every one forced me to ship something real in a weekend, which is a skill no lecture teaches.

philosophy

learning through failure

grades measure what you remember. building measures what you can do. i'd rather break a production system at 2am and learn exactly why than ace a test on it.

campus

outside the classroom

played basketball for the college team and started karuna, a drive to get puppies vaccinated.

certified

certifications

  • anthropic claude ai architect
  • anthropic claude developer
  • ai ethics & safety
  • google ai certification
← back

chapter ivResearch

no papers yet. these are the questions i keep coming back to. if you're working on them, i'd love to talk.

interest

ai safety

i ship agents that act on their own in front of real customers. that makes safety a practical question for me, not an abstract one: how do you know what an agent will do, and how do you keep it that way?

  • how do you evaluate an agent whose behaviour depends on what it remembers?
  • can an agent's memory be audited, corrected and made to forget?
  • when should an autonomous system stop and hand control back to a human?
interest

consciousness through memory

cortex started as infrastructure: give agents memory that persists. but persistent memory raises a bigger question. if a system carries its past forward, reflects on it and changes because of it, what is it becoming? i think deep memory architectures are one of the more concrete paths toward something like machine consciousness.

  • what is the smallest memory architecture that gives an agent a sense of continuity over time?
  • how should episodic memories consolidate into semantic knowledge, the way they do in us?
  • can an agent build a model of itself from its own history?
← back

chapter vLife

what i do when i'm not deploying things. (sometimes still deploying things.)

obsession

building startups

my favourite hobby is also my work. i love building things at scale that help a lot of people, whether that's screening in 30+ clinics or 50k+ conversations a month.

founded

karuna

karuna is a drive i started to get puppies vaccinated, so they survive their first months without catching parvo, lepto and other diseases that kill so many of them early.

court

basketball

played for my college team. (tap it)

underwater

aquascaping

building nature underwater. what fascinates me is the science: managing nitrates, the nitrogen cycle and co2 so plants grow, and watching a glass box develop into a mature ecosystem of its own.

daily

training

working out and staying athletic. it keeps the rest of this running.