Date: 2026-05-13 (depth mode, catalog-wide fake-proof sweep + durable safeguard)
Two ships:
_bulk_gen.py - added audit_fake_proof() as a post-generation validation step that rejects LLM output containing fake-proof violations + strengthened the prompt with specific examples of what NOT to invent.Audited all 62 placeholder JSONs against an expanded regex pattern set. Broader pattern catches:
FAKE_PROOF_PATTERNS = [
(r"over \d{2,}\,?\d{3,}\+? (SaaS|customers|users|teams|companies|practices|accounts|operators|founders|clinics|firms|contracts|conversations|deals|sessions|emails|stores|brands)", "corpus-claim"),
(r"trained on (signals from |behavioral data from |patterns from |real )?(over )?\d+\,?\d{3,}", "training-corpus"),
(r"\d{2,},\d{3}\+?\s+(?:SaaS\s+|active\s+)?(customers|users|teams|companies|practices|accounts|operators|founders|clinics|firms|contracts|deals|brands|programs)", "count-claim"),
(r"used by \d{2,}", "used-by-claim"),
(r"powering \d{2,}", "powering-claim"),
(r"trained on real (\w+\s+)+", "vague-corpus"),
]Pattern set is filtered against false-positive contexts (ICP definitions, demo examples, "for shopify stores doing X").
Audit results before iter 66 fixes: 4 slugs with violations (iter 65 fixed 2, iter 66 fixed 2).
iter 66 fixes:
creator-revenue-ai: "Scans 12,000+ active brand programs and ranks fit by audience overlap, category, and past deal sizes" replaced with "Queries public creator-economy sponsor databases (Passionfroot, Sponsy, Hashtag Paid index) and ranks fit by audience overlap, category, and historical deal sizes pulled from public creator disclosures." The new version names specific real public sources instead of claiming an invented 12,000-program corpus.
rebook-ai: "Trained on real rebooking conversations, not generic IVR scripts" replaced with "Built with the cadence and language patterns of real service-business rebooking calls, not generic IVR scripts." Removes the implicit "we have a private corpus" claim while keeping the operator-voice positioning.
Post-iter 66 audit result: 0 slugs with violations across all 62 placeholder JSONs. Catalog is now fake-proof-clean.
Not flagged (false positives confirmed legit):
_bulk_gen.pyTwo enhancements to prevent future bulk-gen invocations from re-introducing fabrications:
Enhancement A: post-generation audit reject
Added audit_fake_proof(slug, ph_dict) function that scans every string field in the generated placeholder JSON for the fake-proof patterns. If any violations are found, the script REJECTS the output (does not save the placeholders.json, does not render, does not deploy) and saves the rejected JSON to /tmp/<slug>-fakeproof.txt for manual inspection.
This means future bulk-gens cannot silently ship fabrications. The worst case is a rejection + manual prompt revision, not silent corruption of the catalog.
Enhancement B: strengthened prompt language
Old prompt constraint:
- ABSOLUTELY NO fake customer counts, fake testimonials, fake revenue numbers.
- This is a Wishdeal Factory listing without live customers yet.New prompt constraint (added concrete examples + named exceptions):
- ABSOLUTELY NO fabricated proof: do NOT invent customer counts ("100,000 SaaS accounts"), training-corpus claims ("trained on 10,000 contracts"), "used by X teams", "powering Y customers", or any phrase implying we have data or users we do not have.
- If the product uses public reference data (NVCA forms, public benchmarks, etc.), name the specific source. Do not vaguely claim "trained on real X" unless you can name a specific public source.
- Demo example values (e.g., "Acme Corp, $48,000 ACV" in DEMO_INPUT_VALUE) are fine. ICP definitions are fine. Customer-count claims and training-corpus claims are NOT fine.The strengthened prompt names the EXACT patterns that violated the rule in iters 58 outputs ("100,000 SaaS accounts" and "10,000 contracts" are now explicit don'ts). Plus it explicitly carves out ICP definitions + demo examples as PERMITTED, so the model does not over-correct and refuse to generate concrete buyer scenarios.
Both enhancements work together: the prompt makes violations less likely, the audit catches them when they happen anyway.
/Users/wes/factory-templates/creator-revenue-ai-placeholders.json (12,000-programs fabrication replaced with named public sources)/Users/wes/factory-templates/rebook-ai-placeholders.json (vague training-corpus claim replaced with cadence/language framing)/Users/wes/factory-templates/_bulk_gen.py (audit_fake_proof function + post-gen reject + strengthened prompt)/srv/sites/factory/builds/creator-revenue-ai/index.html/srv/sites/factory/builds/rebook-ai/index.htmlOver two iters: discovered 4 fake-proof violations from the iter 58 bulk-gen (silent for 10 days), cleaned them, and installed a durable safeguard so future bulk-gens cannot reintroduce the bug.
This is the same arc the iter 60-62 sequence took with HTML-entity em-dashes: find a silent bug, fix the surface, then patch the source generator + add a validation step so it does not happen again.
The factory's content invariants are now defended at multiple layers:
| Invariant | Surface defense | Source defense |
|---|---|---|
| No em-dashes (Unicode) | Sweep cron every 15 min | All 12 generators clean (iter 62) |
| No em-dashes (HTML entity) | Sweep cron extended (iter 61) | All 12 generators clean (iter 62) |
| No tagline = name | Catalog renders updated taglines | adoptability-score.py patched (iter 62) |
| No name = slug | Catalog renders updated names | adoptability-score.py patched (iter 62) |
| No stale counts | Manual catch + regen | 3 generators patched (iter 63) |
| No fake-proof claims | Catalog manually swept this iter | _bulk_gen.py audit + prompt (iter 66) |
Every content invariant now has both a surface-fix mechanism and a source-fix mechanism. Future bulk-gens will be caught before they ship.
The audit-and-source-fix cycle is complete for the known invariants. Next leverage moves:
Recommended: option 3 (audit per-product subpages). The main index pages are now clean. The subpages (~6 per product × 60+ products = 360+ subpages) are a different layer that has not been systematically audited.
The factory's "invisible bugs" surface area is now systematically managed. The next focus is conversion polish on individual products, not structural durability.