On 7 and 8 August 2026 we measured the technical state of live sites — online stores and service sites across a range of niches. No admin access, no hosting password, no involvement from the owners. Everything described below is what a site hands to anyone who opens it: response codes, time to first byte, sitemap, canonical, structured data, hreflang, certificate and the PHP version in the headers.
This is not a piece about a bad market. It is a description of the baseline every new store competes against. Most of the gaps below can be closed in one evening, which is why they read better as a list of your own opportunities than of someone else's mistakes.
How we measured, and why from the outside
Every measurement is a request to a public address and a reading of the response. Headers, the HTML of the homepage, robots.txt, the sitemap, and a product page wherever one could be opened. Three reasons to work this way.
This is how a crawler sees a site. Whatever is invisible from the outside does not exist for indexing, no matter how tidy the admin panel is inside.
It is the only way to compare yourself with a competitor. You will never get into their panel, yet from the outside you see exactly what Google sees.
It is fast. A few minutes per site. So the full technical sweep — PHP version, whether the sitemap is usable, certificate state — covers 39 sites. The per-niche breakdowns below are separate samples of four to eight sites each, taken on the same two days. They are different sets, and wherever their numbers disagree we say so.
What an external measurement cannot see gets its own section at the end. The list is longer than we would like.
Six gaps that keep repeating
No canonical on the homepage at all
The most common finding of the whole series. A canonical is the line where a page tells search «this is my real address». Without it, every variant of the same page is entitled to count as a separate one: with and without www, with a sort parameter, with a campaign tag in the link, with and without a trailing slash.
| Niche | Homepages with no canonical |
|---|---|
| building materials | 5 of 6 |
| stationery | 5 of 6 |
| optics | 5 of 7 |
| power tools | 4 of 5 |
| footwear | 4 of 5 |
| textiles | 4 of 6 |
| household chemicals | 4 of 6 |
| furniture | 3 of 4 |
| jewellery | 3 of 4 |
The homepage is the simplest page on a site and the first one anyone thinks about. If the canonical is missing there, it is certainly missing in the catalogue and the filters — and the catalogue is exactly where duplicates are born. We covered the mechanics and the checks separately: canonicalisation and removing duplicate URLs.
Breadcrumbs on the screen, nothing in the code
The «Home → Category → Product» trail is drawn almost everywhere. The markup that explains that trail to search is almost nowhere.
| Niche | Sites with no breadcrumb markup |
|---|---|
| optics | 7 of 7 |
| building materials | 6 of 6 |
| stationery | 6 of 6 |
| textiles | 6 of 6 |
| power tools | 5 of 5 |
| footwear | 5 of 5 |
| medicine | 4 of 4 |
| jewellery | 4 of 4 |
Eight niches in a row without a single exception. A result like that usually means not carelessness but that the store template never covered the topic, and nobody opened the question after installation.
The sitemap: missing, borrowed or far too large
Across the 39-site sweep, six have no usable sitemap, and on one more the sitemap points to an outside domain. Search receives a list of addresses that do not belong to that site.
The other side of the problem is size. Medians by niche:
| Niche | Median URLs in sitemap | Largest sitemap in niche |
|---|---|---|
| furniture | 93,960 | over 150,000 |
| stationery | 33,158 | over 150,000 |
| jewellery | 20,898 | 55,983 |
| power tools | 19,962 | 32,623 |
| household chemicals | 16,459 | 79,892 |
| building materials | 15,952 | 24,557 |
| footwear | 12,330 | 13,700 |
A store with a few thousand products and a sitemap of ninety thousand addresses does not have ninety thousand pages. It has the same products laid out across filters, sort orders and tags. What to do about it is in the breakdown of sitemaps and robots.txt, and you can look at your own quickly with this tool.
PHP that no longer receives security updates
Three sites in the 39-site sweep. In the building materials sample, four of six. One more turned up in each of jewellery out of four, stationery out of six, household chemicals out of six and construction firms out of eight.
The gap between «three of thirty-nine» and «four of six» is not an error but two different samples. And it is telling in itself: a market average hides the niches where an outdated build is the normal state of affairs. This is no longer a search question. An unsupported version means nobody will patch the vulnerabilities found in it.
The server's first response
Time to first byte is not site speed yet — only how long the server thinks before the first character of the answer. Images, styles and scripts come after that.
| Niche | Median first response |
|---|---|
| legal services | 772.3 ms |
| furniture | 683.9 ms |
| jewellery | 614.5 ms |
| education | 497.9 ms |
| power tools | 458.8 ms |
| building materials | 436.1 ms |
| optics | 426.4 ms |
| construction firms | 260.6 ms |
| footwear | 211.8 ms |
| travel | 185.6 ms |
The spread between niches is almost fourfold, and catalogue size does not explain it: footwear, with a median catalogue of 12,330 addresses, answers faster than legal sites with a median of 518. In furniture, the first response takes longer than a second on two sites out of four.
The product is not marked up
A product page could be opened on 30 sites. Ten of them carry Product markup, ten carry Offer price markup, four have their specifications marked up and three have a rating. Two thirds of these stores show search a page from which neither what it is nor what it costs can be read.
Separately, some sites carry no structured data at all: four of six in stationery, three of seven in optics, three of six in household chemicals. What this looks like in code is in the breakdown of structured data.
What it costs the owner
No percentages here: we measured code, not the till. But the mechanism behind each point is direct.
Duplicates eat the crawl. A crawler arrives with a limited budget of requests. If the sitemap walks it through tens of thousands of variants of one catalogue, that budget goes on copies while new products wait their turn for days.
Without Product and Offer the snippet stays plain text. Price, availability and rating in the results are read from markup. No markup, no lines — and next to you stands a competitor who has them.
Without breadcrumb markup the path is invisible in the results. Instead of a readable «section → subsection → product» the user sees a bare URL.
A slow first response multiplies into everything else. Seven hundred milliseconds to the first byte happens before the browser has downloaded a single file. Everything else is added on top of that number.
An old PHP version is no longer a marketing matter. It is an open risk to the customer and order database.
The wider list of things worth going through before a season is collected in our technical checklist for an online store.
Fifteen minutes to check your own
Every command reads your site's public response and changes nothing.
1. Canonical on the homepage.
curl -s https://your-site.com/ | grep -io '<link[^>]*canonical[^>]*>'
Empty output is your first job. A line pointing at another domain, or carrying a parameter, is a job too.
2. Whether a sitemap is declared, and what is in it.
curl -s https://your-site.com/robots.txt | grep -i sitemap
curl -s https://your-site.com/sitemap.xml | grep -c '<loc>'
Compare the second number with the count of products and categories in your admin panel. A gap of several times over means duplicates.
3. Duplicates inside the sitemap itself.
curl -s https://your-site.com/sitemap.xml \
| grep -o '<loc>[^<]*' | sort | uniq -d | wc -l
It should be zero.
4. First response.
curl -s -o /dev/null -w 'to first byte: %{time_starttransfer} s\n' \
https://your-site.com/
Three runs in a row, take the worst. There is something to compare against: the niche medians above run from 185.6 to 772.3 ms.
5. Markup on a product page.
curl -s https://your-site.com/product-url | grep -c 'application/ld+json'
Zero means there is no markup at all. One or more means you still need to look at what exactly is marked up.
Response codes across a list of addresses are easy to collect with the status code checker. If after these five points you want the full picture, we run a technical audit starting from the same checks, only across every page rather than the homepage.
Honest limits
An external measurement misses the main thing. It does not know how many visitors you have or where they come from, how many orders are lost in the cart, how your managers work, or whether analytics is configured at all. It sees the façade and draws conclusions about the façade.
A median across four to eight sites is a reference point, not statistics. It shows what counts as normal in a niche and gives no right to claim that the whole market looks that way. One site in a sample can be an outlier, and usually at least one is.
The presence of a tag does not mean it is correct. We recorded whether a canonical exists and whether structured data exists, not whether they point where they should. Correctness is checked by hand, page by page.
And the main one: not a single item on this list converts into money directly. We do not know what a missing Offer costs a particular store, and neither does anyone who quotes a percentage. What we do know is that all of it is visible from the outside, measurable in minutes, and in most cases fixed faster than it gets discussed.
We measure our own site with the same set of checks and published the result together with the commands: a site audit in one day.
