Most AI marketplaces are dressed up like real businesses. Ten checks will tell you if you are looking at one. None of this requires technical expertise. The whole audit fits in 30 minutes. It works on us, and it works on competitors.
You found a marketplace. The catalog looks rich. The branding is clean. There are testimonials and an Adoptability score on every card. You are considering paying for a dossier or a build. Before you do, run this audit.
None of these checks require running curl, reading code, or knowing what JSON-LD is. They are surface-level questions you can answer in 2-3 minutes each. If the marketplace fails three of them, walk. If it fails five, run.
Wishdeal Factory passes its own audit. We will tell you where each check lives on this site. We will also tell you the one place we currently fail.
A serious marketplace publishes its own quality metrics. Not just "we audit our content" but a live dashboard. Look for a page called Quality Report, Status, Audit Results, or similar.
Where we publish ours: /factory/quality-report/. Shows fake-proof audit (currently 0 hard, 0 soft), em-dash invariant status, broken-tagline count, health-endpoint pass-rate, and 10 content invariants. Refreshed every 30 minutes from real audit output.
Red flag if the marketplace claims quality but does not publish anything live. A static "we maintain quality" page is not an audit. A dynamic dashboard is.
An autonomous catalog claim ("we ship things overnight" / "our data is always current") deserves verification. Look for a page that shows when the content last updated.
Where we publish ours: /factory/cron-status/. 131+ cron jobs listed with last-run timestamps. The page itself regenerates every 15 minutes. If you reload after 20 minutes, timestamps shift visibly.
Red flag if "fresh content" is claimed but the catalog has not changed in weeks. Check the changelog, the "recently added" list, and individual product modified-dates if visible.
This is the biggest tell. Most marketplaces promise too much. A serious one names its limits.
Where we say ours: /factory/honest/. The opener: most ideas in this catalog will not become your next business. We disclose: zero live customer revenue claimed for any product, distribution is the hard part, the median idea has zero validated demand signal. The page exists to reduce refund risk by setting expectations correctly.
Red flag if a marketplace has no disclaimer page, or the "About" is pure marketing. Look for a page that names the failure modes. If it doesn\'t exist, the marketplace either has not thought about its failure modes or is hiding them.
"Used by 500+ companies." Where? Which 500? Can you click any of them? Are the logos clickable to real customer URLs? Is the customer name a verifiable business?
Most marketplaces have no real customers and use stock-customer logos or vague claims. The honest test: pick one named customer, search them, find them on LinkedIn, see if they have actually mentioned the marketplace publicly.
What we say: "Zero live customer revenue." We have one product that graduated to its own domain (Counsel AI / intakecounsel.com), and we document that case explicitly at /factory/playbooks/counsel-anatomy/. No other product has a live customer claim because no other product has live customers.
Red flag if a marketplace claims customer counts (50+, 500+, 10,000+) without naming any. Real customer counts come with at least 5 real names you can verify.
Many "marketplaces" are one person with a polished website. That is fine, but the marketplace should say so. If the "Team" page has stock photos and made-up names, leave.
What we have: /factory/about-the-builder/ names one person (Wes Lemos) plus six other shipped products. No team page pretending we are bigger.
Red flag if the team page has more than 3 names and they all look like AI-generated headshots. Reverse-image-search one of the photos. Real teams have findable LinkedIn profiles.
Read the public teaser of three different products. Are they actually different? Do they name different ICPs, different workflows, different pain points? Or are they templated content with one section swapped per product?
The depth of the free tier is a near-perfect proxy for the depth of the paid tier. If the teaser is generic, the dossier will be generic. If the teaser names a specific buyer with a specific pain in their own words, the dossier probably does too.
Sample three of ours: bookkeeper-ai, nurture-ai, roofing-ai. Each names a specific ICP, a specific workflow disrupted, and a specific pain point. They are not interchangeable.
Red flag if three free teasers are 90 percent identical and only the product name changes.
What happens if you pay and the dossier is bad? Most marketplaces have a 30-day money-back guarantee, an instant refund flow, or a refund-via-email path. None means risk.
Read the FAQ or terms page. Look for the word "refund" specifically. If you cannot find a refund path in 30 seconds of searching, treat that as a no-refund marketplace.
What we offer: 30-day money-back guarantee on $5 unlocks, surfaced at every product unlock CTA. The $99 adopt tier has the same guarantee. The operator partnership tier has scoped refund clauses per engagement.
Red flag if the refund policy is buried, vague, or "case-by-case." That is no policy.
A live catalog has a public changelog. Not a marketing blog. An actual list of what changed and when.
Where ours is: /factory/changelog/ lists every iteration. /factory/log/ship-logs/ is the index of 91+ individual ship logs, each with the iter\'s shipped changes, the why, and the next-iter queue. The receipts are public.
Red flag if the only "changelog" is a quarterly product update blog. Real catalogs change daily. If the change rhythm is silent, the catalog is probably abandoned.
Use your browser\'s find-in-page (Cmd-F / Ctrl-F) on three sample product pages and search for these phrases:
| Phrase | Why it is a red flag |
|---|---|
| SOC 2 certified | Most marketplaces do not have SOC 2. Stating it falsely is a fabrication. |
| trained on real X data | Most "AI products" have no trained model. The marketing copy is aspirational. |
| used by 500+ companies | Customer counts without names are usually invented. |
| in 18 months we | Time-based fake history. Check the actual launch date. |
| our pilot customer | Often a placeholder for "we have not shipped to anyone yet." |
| 100% accuracy | No AI is 100% accurate. Claiming it suggests not-yet-built. |
What we did about these: We built an audit that catches them automatically. /factory/playbooks/seventy-fabrications/ tells the story of catching 70 of these in our own catalog and fixing them at source.
This one is for the technical buyer. A serious catalog publishes a few public endpoints (state, health, adoptability scores, teaser content) and documents them honestly. The presence of these endpoints means the catalog is a real system, not a static brochure.
Where ours is: /factory/api-docs/ documents 17 endpoints (5 static read endpoints plus 12 live application endpoints). The honest framing section names what we do not promise (no SLA, no rate limits, no v1 contract).
Red flag if a marketplace claims to be data-driven but publishes no machine-readable data. That is a brochure, not a system.
Out of 10 checks, here is the rubric:
| Pass count | Verdict |
|---|---|
| 8 to 10 | This is a real catalog. The risk is normal product risk, not vendor risk. |
| 5 to 7 | Mixed signal. Spend the $5 unlock, judge from the dossier itself. The catalog is invested but immature. |
| 3 to 4 | The marketplace is marketing-first, audit-second. Buy only if the specific product is unique enough that audit infrastructure does not matter to you. |
| 0 to 2 | Walk. The catalog is a brochure dressed up as a system. |
The Wishdeal Factory passes 10 of 10 checks. The one place we currently fail by some definitions: zero live customers. We are an autonomous studio in operation, not a community of buyers, and the catalog explicitly says so on /honest/. By "no live customer revenue" rule we fail any check that says "show me real money." But that is a separate test from the marketplace-audit test above.
If you want to verify any of this audit on us specifically, the surfaces are at /quality-report/, /cron-status/, /api-docs/, /log/ship-logs/, and the playbook library at /playbooks/.
One important caveat: this audit checks whether the catalog is honest and well-maintained. It does NOT check whether the product is right for you. A catalog can pass all 10 checks and still not have the idea that fits your situation. A catalog can fail several checks and still have one product that is a perfect fit.
Treat this audit as the vendor-risk filter, not the product-fit filter. After this passes, the product-fit question is its own conversation, covered in /factory/playbooks/reading-adoptability/ (how to read the score) and /factory/playbooks/skip-these-dossiers/ (which products to actively avoid).
The 30-minute audit is what you run before you spend the first dollar. The 10-minute Adoptability reading is what you run before you spend the second dollar.