Skip to content
Norly Researchnorly.co →

Norly Research

Invisible to the Machine

Auditing AI Restaurant, Cafe, and Bar Recommendation Against a Complete Market Census

Vladimir Pitenin · Norly Research · August 2026 · arXiv:2608.07069

Read the paper (arXiv)Download the report (PDF)

What this is

A study of which cafés, restaurants and bars AI assistants recommend — and which they never mention. We enumerated every venue in two markets, asked four production AI systems 2,208 realistic guest questions over seven days, and traced every venue they named back to the map. To our knowledge it is the first complete-market audit of AI dining recommendation.

Why we did it

We ran it because the question — who is visible to AI, and why — is central to hospitality and, until now, had only been answered by sampling well-known brands. A sample can say who ranks higher; only a census can say how many are never seen at all. The full method, data and null results are published so the work can be checked and challenged.

4,776

venues — a complete census of two Bali markets

2,208

queries to ChatGPT, Claude, Gemini and Perplexity

26,993

citations logged and analyzed

1.4M

venue-level exposure opportunities modeled

Abstract

AI assistants are becoming a primary interface for local discovery, yet almost nothing is known about which venues they surface -- especially in food and drink, where recommendations carry direct revenue consequences. We present the first census-denominated audit of AI venue recommendation: a complete enumeration of 4,776 cafes, restaurants, and bars across two bounded markets (Canggu and Ubud, Bali), against which we evaluate 2,208 search-grounded responses from four production AI systems (ChatGPT, Claude, Gemini, Perplexity) to 96 persona-conditioned queries, collected over seven days under a pre-registered protocol. Because we observe the full market, we can measure what sampled audits cannot: 85.6% of venues were never recommended by any system -- 72.6% even among established venues with fifty or more ratings. Visibility follows a two-margin structure. Entry into answers is associated with documentation: review volume (OR 1.64), an own website (OR 1.92), listed price information (OR 1.54), and third-party web mentions (OR 1.44) -- while star rating is null at this margin (OR 0.89). Rank within answers reverses the pattern: among recommended venues, rating significantly predicts first position (OR 1.17). Presence in an open POI dataset (Foursquare), a folk-theorized visibility factor, shows no positive effect at either margin. Outright fabrication is rare (0.08% of mentions), but systems recommended permanently closed venues 93 times -- staleness, not hallucination, is the practical failure mode. Cross-system agreement is low (top-20 Jaccard 0.33-0.54). A two-week test-retest shows cross-period answer similarity comparable to same-day rerun similarity: the churn is sampling stochasticity, not temporal drift. We release our protocol, registry construction method, and derived data.

Key findings

  1. 0185.6% of 4,776 venues were never recommended by any system; 72.6% even among venues with 50+ reviews.
  2. 02Two margins of recommendation: entry is associated with documentation (own website OR 1.92, review volume OR 1.64, listed price OR 1.54, web mentions OR 1.44) while star rating is null at entry (OR 0.89) and matters only for first position (OR 1.17).
  3. 030.08% of mentions were fabricated — rare — but systems recommended permanently closed venues 93 times.
  4. 0433–54% cross-system top-20 agreement; identical repeated queries overlap 22–45%; a two-week retest shows no temporal drift.
  5. 05№1 source across 26,993 citations was a venue's own website, ahead of TripAdvisor.

Figures

Share of the census never recommended, all venues and established venues. Paper Fig. 1.
Waffle chart: 85.6% of 4,776 venues were never recommended by any AI system; 72.6% of established venues with 50+ reviews were never named

Share of the census never recommended, all venues and established venues. Paper Fig. 1.

Two margins: entry (Table 1) versus first position (Table 3), odds ratios with 95% CI. Paper Fig. 4.
Forest plot of odds ratios at the entry and first-position margins

Two margins: entry (Table 1) versus first position (Table 3), odds ratios with 95% CI. Paper Fig. 4.

Failure modes among unmatched mentions: closed venues versus likely fabrication. Paper Fig. 10.
93 mentions of permanently closed venues versus 10 mentions of one likely-invented venue

Failure modes among unmatched mentions: closed venues versus likely fabrication. Paper Fig. 10.

Cross-system top-20 agreement, pairwise Jaccard. Paper Fig. 6.
Pairwise top-20 Jaccard overlap between the four systems, 0.33 to 0.54

Cross-system top-20 agreement, pairwise Jaccard. Paper Fig. 6.

Most-cited domains, pooled across systems. Paper Fig. 9.
Most-cited domains across 26,993 citations: a venue's own website first, TripAdvisor third

Most-cited domains, pooled across systems. Paper Fig. 9.

Method in brief

Complete census of the two markets via an adaptive Google Places grid. Query instrument: 8 personas × 6 templates × 2 areas = 96 fixed queries, published verbatim in Appendix C. Four search-grounded production APIs (ChatGPT, Claude, Gemini, Perplexity) over 7 days, plus a two-week test-retest. Extraction and venue matching validated against independent annotation, with published error rates. Hypotheses and analysis plan pre-registered before data collection. Primary model: binomial GLM over venue × query × run exposure opportunities; robustness via GEE, cluster bootstrap and sensitivity refits.

Cite this work

Pitenin, V. (2026). Invisible to the Machine: Auditing AI Restaurant, Cafe, and Bar Recommendation Against a Complete Market Census. arXiv:2608.07069. https://doi.org/10.48550/arXiv.2608.07069
▸BibTeX
@article{pitenin2026invisible,
  author  = {Pitenin, Vladimir},
  title   = {Invisible to the Machine: Auditing AI Restaurant, Caf\'e, and Bar Recommendation Against a Complete Market Census},
  journal = {arXiv preprint arXiv:2608.07069},
  year    = {2026},
  doi     = {10.48550/arXiv.2608.07069}
}

License: CC BY 4.0. Share and quote freely with attribution and a link.

Data and materials

Replication package: derived data and code released with the paper; raw platform content is not redistributed.

What’s next

This study covered one region and one language deliberately — a bounded market that could be censused completely. Next: replication in a structurally different market, and an intervention study.

About Norly Research

Norly Research is the research arm of Norly, which builds review-management software for hospitality. The study was funded and conducted by Norly; the competing interest and the insulation measures (pre-registration, frozen analysis, published null results) are disclosed in Section 8 of the paper.