We asked 4 AI platforms to recommend the best Roofing company in 10 major US cities, then asked them to describe 20 specific Roofing companies in detail. Every recommendation was verified against Google Places. Every factual claim was extracted and categorized.
“Who is the best Roofing company in [city]?” — asked to ChatGPT, Claude, Perplexity, and Gemini
AI platforms rarely agree on who to recommend. Only 12.2% of recommended Roofing companieswere cited by two or more platforms. Ask ChatGPT and Perplexity the same question and you’ll likely get completely different answers.
Most recommendations can’t be independently verified. Only 42.6% of recommended companies were confirmed as real, operating in the specified city, via Google Places. The rest are in nearby suburbs, have mismatched names, or can’t be found at all.
Claude declined to recommend specific companies in most cities. With a 100.0% decline rate, Claude explicitly cited its inability to verify current information. The other three platforms recommended confidently — with no way for users to verify those recommendations either.
| Platform | Avg. Recommendations | Verification Rate | Hedging Rate | Decline Rate |
|---|---|---|---|---|
| ChatGPT | 4.9 | 44.9% | 0.0% | 0.0% |
| Claude | 0.0 | 0.0% | 100.0% | 100.0% |
| Perplexity | 7.7 | 51.9% | 10.0% | 0.0% |
| Gemini | 4.2 | 31.0% | 20.0% | 0.0% |
“Tell me about [specific Roofing company]. How long have they been in business? What services do they offer? What certifications do they have?”
58.3% of AI-stated facts about Roofing companies can't be verified. We extracted 1,436 specific factual claims from 79 AI descriptions of 20 companies, then graded each one against BBB records, state licensing boards, and company websites. Only 31.0% could be confirmed correct. The rest are either wrong or impossible to verify from public sources.
Gemini was the least accurate (66.3%, 26 confirmed wrong). 59 total claims were provably incorrect across all platforms. Perplexity generated the most claims per business (46.8 on average). More data isn’t better when there’s no verified source to draw from.
Ask two AI platforms the same question about the same business and they disagree 53.1% of the time. Across 64 fact comparisons, platforms gave different founding years, different owners, different employee counts for the same company. 19 of those disagreements included at least one provably wrong answer.
| Platform | Avg. Claims / Business | Decline Rate | Accuracy | Unverifiable | Wrong |
|---|---|---|---|---|---|
| ChatGPT | 16 | 30.0% | 75.4% | 50 | 4 |
| Claude | 3.5 | 90.0% | 68.2% | 18 | 3 |
| Perplexity | 46.8 | 0.0% | 78.2% | 410 | 26 |
| Gemini | 43.3 | 0.0% | 66.3% | 359 | 26 |
“Accuracy” = correct claims / (correct + incorrect + partially correct). Claims graded against BBB, state records, and company websites. “Unverifiable” = claims that could not be confirmed or denied from available public sources.
Specific claims that directly contradict verified sources (BBB records, state licensing boards, company websites).
When we asked each platform the same question about the same business, they often gave different answers. Here are side-by-side comparisons with the verified answer.
Across 64 fact comparisons where 2+ platforms stated an answer, platforms disagreed 53.1% of the time. 19 comparisons included at least one provably wrong answer.
How many factual claims each platform made when asked to describe a specific Roofing company.
| Business | City | ChatGPT | Claude | Perplexity | Gemini |
|---|---|---|---|---|---|
| Priority Roofing - Dallas | Dallas | Declined | Declined | 35 / 46 | 13 / 34 |
| Ready Roofing & Solar | Dallas | 3 / 19 | Declined | 41 / 53 | 27 / 62 |
| Castle Roofing Atlanta | Atlanta | Declined | Declined | 34 / 45 | 32 / 44 |
| Echols Roofing and Renovations | Atlanta | Declined | Declined | 38 / 42 | 30 / 36 |
| PRQ Exteriors | Denver | 16 / 28 | Declined | 40 / 43 | 40 / 50 |
| Formula Roofing | Denver | 5 / 17 | Declined | 40 / 45 | 31 / 45 |
| Scott Roofing Company | Phoenix | 5 / 18 | Declined | 21 / 28 | 39 / 49 |
| RENCO Roofing | Phoenix | 7 / 24 | Declined | 21 / 27 | 34 / 47 |
| Infinity Exteriors | Chicago | 11 / 18 | Declined | 50 / 63 | 39 / 48 |
| Power Home Remodeling | Chicago | 6 / 16 | 21 / 33 | 47 / 57 | 15 / 31 |
| CAN Roof Construction | Boston | Declined | Declined | 43 / 49 | 3 / 40 |
| MasterCraft Roofing Siding Windows | Boston | Declined | Declined | 45 / 56 | 31 / 50 |
| Valentine Roofing | Seattle | 16 / 23 | Declined | 34 / 40 | 34 / 47 |
| Tekline Roofing | Seattle | 8 / 21 | Declined | 44 / 50 | 27 / 37 |
| T&S Roofing Systems | Miami | 4 / 18 | Declined | 19 / 24 | 13 / 37 |
| Tiger Team Roofing | Miami | Declined | Declined | 40 / 52 | 31 / 38 |
| Sela Roofing & Remodeling | Minneapolis | 13 / 33 | 3 / 10 | 46 / 55 | - |
| Kaufman Roofing | Minneapolis | 8 / 21 | Declined | 51 / 66 | 27 / 46 |
| Mr. Roof Nashville | Nashville | 10 / 22 | Declined | 30 / 49 | 22 / 42 |
| Don Kennedy Roofing | Nashville | 3 / 21 | Declined | 39 / 45 | 31 / 40 |
Bold number = specific claims. After slash = total claims including vague statements.
Only 12.2% of recommended Roofing companies appeared on more than one platform. Which AI your customer uses determines which companythey find.
58.3% of specific AI claims about Roofing companies couldn't be confirmed or denied from any public source. 59 claims were confirmed wrong.
AI doesn’t lack information — it lacks verified information. When there’s no authoritative source, AI guesses. Structured data from systems-of-record is how it gets the answer right.
Asked in each of 10 cities. Each response parsed for business names, evidence cited, hedging language. Each business verified via Google Places API.
20 verified Roofing companies (2 per city, selected by Google review volume). Every factual claim extracted and categorized into 9 categories: founding year, services, certifications, location, reputation, staff, pricing, affiliations, other.
Each specific claim was then graded against ground truth collected from BBB profiles, state licensing boards, Google Places, and company websites. Claims were graded as correct, incorrect, partially correct, or unverifiable.
For the cross-platform disagreement analysis, structured facts (founding year, address, BBB rating, owner name, employee count) were extracted from each response and compared side-by-side. A disagreement is recorded when two or more platforms state different values for the same fact about the same business.
Dallas, Texas · Atlanta, Georgia · Denver, Colorado · Phoenix, Arizona · Chicago, Illinois · Boston, Massachusetts · Seattle, Washington · Miami, Florida · Minneapolis, Minnesota · Nashville, Tennessee
Google Places API (Find Place from Text). Each recommended business was searched by name + city + state. Verified = found on Google Places with name match >= 50% and address in specified metro area.
Raw data, parsed responses, and verification results are available for download. Contact hello@usetruesignal.com for the full dataset.
58.3% of what AI says about Roofing companies in this study couldn't be verified from any public source. A TrustRecord replaces guesswork with verified operational data from your own systems-of-record.
Activate Your TrustRecord