My city pages lost to their country page. Why thin pages with generic titles get skipped, what search and AI engines need to pick a page, and the steps.
My site has a page for every city I’ve travelled to and written about. The city pages weren’t showing up for searches like “Luxor”. Only the Egypt country page did.
AI answer engines such as ChatGPT search and Perplexity face the same choice from the other side. When someone asks a question, they search, fetch a few pages and quote the ones that answer it best. If a page loses that choice, it doesn’t get quoted.
This is why my city pages were losing, what search and AI engines need before they’ll pick a page, and the steps, with examples from two sites: my own, and BHyve’s marketing website.
Why the city pages were skipped
The causes were on the pages themselves.
They were thin. Each city page was a short story. A search engine choosing a page for a query looks for the one that answers it most fully, and a few paragraphs of story rarely win that. The country page covered the whole trip, so next to a slim city page it could easily look like the better answer.
Their titles were generic. Every city page used the same pattern, like “Luxor, Egypt: travel story”. The description under it was just the opening lines of the story. Neither said what the page offered: what to see, what it costs, how to get around. A title is one of the strongest hints an engine has about what a page is for, and “travel story” matches nothing people actually search for.
Neither cause is about markup or robots files. Those matter, but only once the page deserves to be picked.
What an engine needs to pick a page
Search engines and AI answer engines go through the same three steps, and a page can fail at any of them.
- Find it. The page has to be in the index the engine searches. Engines find pages through links and through maps of the site: a sitemap for crawlers, and, as a newer idea, a plain-text summary for language models called llms.txt.
- Fetch it. The engine’s crawler has to be allowed in, by robots.txt and by whatever sits in front of the site, such as a CDN.
- Choose it. Out of every page that could answer, the engine picks the one that says most clearly what it’s about and answers most directly. Much of that comes down to a specific title and summary, direct answers short enough to quote, a date that shows how current the facts are, and structured data: a block of machine-readable JSON, using the schema.org vocabulary, that says what the page is and what it mentions.
Finding and fetching are plumbing. Choosing is where pages win or lose, and it’s where my city pages lost.
The same causes on a company website
BHyve’s marketing website had a different version of the same problem. When I rebuilt it, the weak spots were about telling engines which page is which. Old URLs that no longer existed were dead ends for anyone, person or crawler, arriving from an old link or search result. Redirecting them sends both to the page that replaced them. Some pages had duplicate main headings (H1s), and a page’s H1 is its own statement of what it’s about, so duplicates blur it. The rebuild also added structured data, an llms.txt and canonical tags, which name the one main URL when a page can be reached at several.
Each of those fixes helps an engine find the right URL and tell what each page is for.
The steps
- Make the page worth choosing. Before touching any markup, make each page the fullest answer to the question it’s for. On my site, every slim city page grew into a full guide.
- Write a specific title and summary, in the words people search with. The Luxor page is now “Luxor in two days: temples, tombs, costs and tips”. Its summary lists what’s actually on it: the West Bank tombs, a guide at Karnak, the sunrise balloon, the ferry and ticket prices.
- Answer questions directly, on the page. Each city page now has a short Quick answers section, with questions like how to cross the Nile in Luxor, in sentences that make sense when quoted alone. I mark them up as FAQPage for any parser that reads it. Google stopped showing FAQ rich results in Search in May 2026, so the answers on the page are what does the work.
- Describe the page for machines, and date it. Structured data says what the page is. Mine describes each city page as a blog post about a place, with a list of the sights it mentions. Show an updated date on the page, and match it in the meta tags and the sitemap, because on a page full of prices, how current they are is part of the answer.
- Keep one URL and one main heading per page. Redirect old URLs permanently (301 or 308) so engines treat the new one as the replacement, keep one H1, and add a canonical tag.
- Publish a map of the site. A sitemap for crawlers, and an llms.txt for language models: a Markdown file with the site’s name, a short summary and a list of key pages. Mine points to an llms-full.txt that carries every page as Markdown. Keep expectations low. llms.txt is a proposal, not a standard, and the big AI companies have said little about how they use it.
- Let the right crawlers in, and check what sits in front. Search crawlers and training crawlers are separate. OAI-SearchBot puts pages in ChatGPT search, PerplexityBot in Perplexity and Claude-SearchBot in Claude’s search, while GPTBot and ClaudeBot collect training data. If being quoted is the goal, the search crawlers are the ones that count. My robots.txt names them all, so a block elsewhere is easy to spot against it. robots.txt can’t overrule a CDN, which can turn a bot away before the request reaches your site. Cloudflare, for one, has blocked AI crawlers by default on new domains since July 2025, so check what your settings do today, not what you remember them doing. I tested mine by requesting the site as each AI crawler, and every one got through.
- Tell engines when a page changes. IndexNow is a protocol that lets a site notify search engines such as Bing and Yandex the moment a page is added or updated, instead of waiting for a recrawl. Google doesn’t take part, and a ping doesn’t guarantee indexing. Some CDNs can send the pings for you. On Cloudflare that’s Crawler Hints, which I switched on for this site.
What I can’t tell you yet
There are no before-and-after rankings or citations here, because none are measured yet. Treat this as the method, not the result.
The lesson
Engines quote pages that are easy to find, open to fetch and specific enough to answer a question. The first two are quick to fix. The third is the real work.
The rest of the website rebuild is in the frontend case study, and the Luxor page shows all of this in place.
Quick answers
- How do I get my website cited by ChatGPT or Perplexity?
- Make sure their search crawlers can reach the page, then make the page the clearest answer to one question: a specific title and summary, short direct answers, a visible date and structured data that says what the page is about.
- How do I allow ChatGPT to show my site in search answers?
- Allow OpenAI's OAI-SearchBot in robots.txt and make sure your CDN or firewall isn't blocking it. GPTBot is a separate setting that controls training, not search.
- What is llms.txt?
- llms.txt is a proposal, published by Jeremy Howard in September 2024, for a Markdown file at /llms.txt that gives language models a short summary of a site and a list of its key pages. It is a proposal, not a formal standard.
- Does FAQPage markup still get rich results in Google?
- No. Google limited FAQ rich results to government and health sites in 2023 and stopped showing them in Search in May 2026. FAQPage is still a valid schema.org type that other parsers can read.
- Why does my category page rank instead of the specific page?
- Usually because the specific page is thin or its title is vague, so the broader page looks like the fuller answer. A specific title and a page that fully answers the query give the engine a reason to pick it.