Ryan Smith
production2026-08

The Questions That Are Really About a House

Watches a caregiver forum three times a day and emails only the questions worth answering

280 in 31 daysQuestions captured
14 (5%)Flagged as relevant
under 25 secondsRun time

The Problem

A public caregiver forum posts a steady stream of questions from families dealing with aging parents. A handful of those are people asking exactly what a probate property guide answers: what to do with the house, who has authority to sell it, what happens to the deed. The rest are about burnout, incontinence, and difficult siblings. Finding the useful ones meant opening the forum by hand and reading past a few hundred posts that had nothing to do with property.

What It Does

Three times a day it loads the forum's question listing and pulls out every question title, its topic tags, and its link. It checks each one against a running ledger and discards anything already seen. Whatever is new goes to a language model with a scoring rubric that rates it 0 to 10 for how close the asker is to someone with a house to deal with, with a bonus for anything mentioning Florida. Every new question gets written to the ledger with its score and a one-line reason. If anything in that batch scored 7 or higher, a single digest email goes out with those questions ranked, linked, and explained.

A publisher or operator whose audience announces itself in public forums but is buried in unrelated posts.

Outcome

In 31 days it captured 280 distinct questions and flagged 14 as close to the target reader, removing roughly 95% of the reading. Runs finish in under 25 seconds and the pipeline has run clean since launch. It was built as a proof of concept and no outreach was made to any of the people it surfaced.

Screenshots

The digest email: one question scored 9 out of 10 out of three captured that run, with the model's one-line reason under it.
The digest email: one question scored 9 out of 10 out of three captured that run, with the model's one-line reason under it.
The digest email: one question scored 9 out of 10 out of three captured that run, with the model's one-line reason under it.

The digest email: one question scored 9 out of 10 out of three captured that run, with the model's one-line reason under it.

The ledger, one row per question, with the score and reason the model assigned. Green marks the rows that cleared the threshold.
The ledger, one row per question, with the score and reason the model assigned. Green marks the rows that cleared the threshold.
The ledger, one row per question, with the score and reason the model assigned. Green marks the rows that cleared the threshold.

The ledger, one row per question, with the score and reason the model assigned. Green marks the rows that cleared the threshold.

Six days of run history: three runs a day, every one succeeded, each finishing in under 25 seconds.
View full size
Six days of run history: three runs a day, every one succeeded, each finishing in under 25 seconds.
Six days of run history: three runs a day, every one succeeded, each finishing in under 25 seconds.

Six days of run history: three runs a day, every one succeeded, each finishing in under 25 seconds.

How It's Built

Trigger:Cron at 08:47, 12:26, and 15:51 Central, plus a manual trigger for the 5-page backfill and a second manual trigger for scoring rows that predate the scoring rubric
Components:22

A schedule node sets a page count, and a code node expands that into one URL per forum page (page 1 bare, pages 2+ with a query param). An HTTP node fetches each page as raw text with a browser user-agent, batched one at a time with a 1.5-second gap and three retries. A regex parser walks the HTML, pulling question ID, title, and topic tags out of each anchor, then slices the block between one question link and the next to attribute topics to the right question. Parsed questions are deduped against the Google Sheet ledger itself rather than against workflow state, so the dedupe survives re-imports and restarts. Survivors are chunked 15 at a time into an OpenAI chat-completions call with a JSON-mode response and the scoring rubric in the system prompt. A join node maps scores back onto the original items by question ID, defaulting to 0 with a visible reason when the model omits one. Scored rows append to the sheet, then a digest builder filters to the threshold, sorts by score, and renders an HTML table that only sends when the count is above zero. A parallel backfill branch reads the whole sheet, filters to rows with an empty score, scores them with the identical rubric, and writes back matched on question ID so it is safe to re-run.

By the numbers

distinct questions captured
280
capture window
2026-08-15 to 2026-09-14 (31 days)
new questions per day after backfill
6.1 average, range 1 to 14
backfill haul (5 pages, first run)
97
questions scoring 7 or higher
14 of 280 (5%)
questions scoring 0
143 of 280 (51%)
unscored rows in ledger
0
run duration
1.4 to 22.1 seconds
questions per scoring API call
15
scoring cost
negligible, not tracked

Key Decisions

Three runs a day rather than one

So a new post gets spotted fast enough to respond during work hours

Alert threshold of 7

A jumping-off point rather than a hard line. The score is a frame of reference for how close a question is to the ideal customer, not a verdict

A small, cheap model for the scoring

The job is simple: decide whether the question involves a house that may need to be sold. That does not need a high-reasoning model

Dedupe against the Google Sheet, not workflow state

The ledger is the source of truth, so re-importing or restarting the workflow cannot produce duplicate rows

Throw a loud error when the parser matches zero questions, with byte counts and the first 800 characters of HTML in the message

A silent zero-result run looks identical to a quiet day on the forum. The error message exists to say which of the two happened

Keep the backfill scoring branch separate from the daily path

Scores could be added to a ledger that already held hundreds of unscored rows without re-scraping anything

What broke & how it was fixed

The parser was rewritten to v3 after the site served minified HTML with unquoted attribute values, so matching on quoted href attributes returned nothing. The fix made the quotes optional in the pattern and treated whitespace or a closing bracket as the terminator. Nothing else has broken since.

Where else this applies

The shape is: a public source that posts more than anyone will read, a cheap ledger that remembers what has already been seen, and a scoring pass that decides what earns attention. Swap the forum for job boards, permit filings, RFP portals, or a subreddit and the rest of the pipeline is unchanged. The rubric is the only part carrying domain knowledge, and it lives in one editable block of text rather than in code.