MomTest AI
Founders validating their own product ideas tend to ask leading questions — "Would you use this?" — which produces false-positive validation. MomTest AI replaces the human interviewer with an AI that's structurally constrained to follow Rob Fitzpatrick's "Mom Test" methodology.
Problem
Founder-led interviews are prone to leading questions that produce false-positive validation. MomTest AI removes the human interviewer from that failure mode entirely, running the interview itself under hard behavioral constraints.
Flow
A four-phase pipeline, orchestrated with LangGraph state machines and a two-layer guard system that keeps the interview itself honest.
- 01Intake — a conversational chat where the LLM extracts the riskiest assumption behind the idea, in at most 8 questions.
- 02Generate — a LangGraph pipeline (parse → validate → retry → critique) turns the intake conversation into a structured ResearchBrief and an 8–10 question InterviewScript.
- 03Interview — the script runs live with the participant; every message passes through a zero-cost regex filter first, escalating only ambiguous cases to an isolated LLM checker with no conversation history, blocking 20+ leading-question patterns at near-zero cost, with a bounded reflection-based retry when a message is flagged.
- 04Analyze — the transcript is deterministically grounding-checked (quotes matched against the actual transcript text, no extra LLM call) before being classified into an evidence-strength taxonomy and turned into a decision recommendation.
Technologies
Notable decisions
Two-layer guard system
A zero-cost regex filter handles the obvious cases, escalating only ambiguous messages to an isolated LLM checker with no conversation history — blocking 20+ leading-question patterns at near-zero cost instead of running every message through an LLM.
Deterministic grounding check before classification
Quotes pulled into the final report are matched against the actual transcript text with no additional LLM call, catching hallucinated quotes before the evidence-strength classification step ever runs.
LangGraph state machines for structured generation
Both the script-generation and analysis phases run as parse → validate → retry → critique pipelines, rather than a single unstructured LLM call, so malformed output gets caught and corrected before it reaches the participant or the report.
Result
Deployed and running on AWS with automated daily backups and infrastructure monitoring. The two-layer guard architecture catches leading questions before they reach the participant, and the grounding verifier catches hallucinated quotes in the final report without any additional LLM call.