Monitoring runs against the same database it monitors, and the queries that describe a server are unusually expensive to answer. This is what that cost looked like when we went looking for it.

With the backups excluded, what is left examining large numbers of rows on this production server? Five times out of six, it is our own measurement queries.

examined      sent   per row   what it was
1,748,148        2   874,074   our index-size query
  255,164   58,296         4   Laravel schema introspection
  128,994       10    12,899   our unindexed-reads query
  128,994       10    12,899   the same query, per-table variant
  112,448        2    56,224   our task_runs count
   62,574        0    62,574   our secondary-index census

The worst of those examined 1,748,148 rows to return two. It is the query we used to measure how much disk the unused indexes occupied, part of an argument that those indexes have a cost.

Why metadata queries are so expensive

information_schema and performance_schema look like tables and are not. There is no stored relation to seek into; the server assembles the answer by walking its own metadata for every object in scope, at query time.

So a join across innodb_index_stats, statistics and tables — three metadata views — makes the server enumerate every index of every table several times over to produce six rows of summary. The optimiser cannot help, because there is nothing to index.

This is why the examined-to-returned ratios above are absurd rather than merely bad. A ratio of 874,074:1 is not a badly written query. It is the ordinary price of asking a database to describe itself, and it is charged to the server you are asking about.

The one that is not ours

Second on the list, and the only entry not written by this session:

4,368 executions   255,164 examined   58,296 returned   4 per row

That is the framework's schema introspection — reading column metadata so it can build models and validate migrations. Its ratio is fine. Four rows examined per row returned is what a well-targeted query looks like.

Its cost is entirely in the 4,368 executions. Something is asking the database to describe a table thousands of times a day, and each answer is cheap enough that nothing will ever flag it.

These two query shapes are costly in different ways, and they need opposite fixes:

bad ratio, few executions    fix by running it less, or elsewhere
good ratio, many executions  fix by caching, or by asking once

Watching for slow queries catches the first shape and misses the second entirely, while watching total time catches the second and shrugs at the first. Neither is the wrong thing to watch, but neither alone is enough.

The uncomfortable part

The measurement queries ran against production. They were read-only, they ran a handful of times, and each one examined between sixty thousand and one and three quarter million rows in order to answer a question about whether the server was examining too many rows.

On a database with a 512 MB buffer pool, walking the metadata of every table is not free. The irony is neat, but the practical point is that observability is workload. A dashboard that polls this every minute is a scheduled job doing the heaviest scans on the system, added by the person trying to reduce scanning.

Three fixes follow, from cheapest to most involved.

Reduce the frequency. Index censuses and metadata joins answer questions that change on the timescale of deploys rather than seconds, so once a day is the correct interval rather than a compromise.

Move it to a replica, with one real caveat: schema shape reads the same there, but the performance counters are per-node, so a replica cannot tell you what the primary is doing.

Stop joining metadata views. Most of the cost above is in joining three of them. Reading each separately and combining the results in an application is uglier, cheaper, and does not hold the server while it enumerates.

What to check on your own system

SELECT DIGEST_TEXT, COUNT_STAR, SUM_ROWS_EXAMINED, SUM_ROWS_SENT
FROM performance_schema.events_statements_summary_by_digest
WHERE DIGEST_TEXT NOT LIKE '%SQL_NO_CACHE%'
ORDER BY SUM_ROWS_EXAMINED DESC
LIMIT 20;

Exclude the dumps or they will fill the list. Then read the result twice: once for the ratios, and once for the execution counts, because the two columns are describing different problems and the eye naturally goes to the big number in the wrong one.

If your own monitoring appears in the top twenty, that is not a reason to stop measuring. Just know what the measurement costs, which you now do.

How this was measured, and what it does not establish

One production MySQL 8.0.46 server over a 23-hour window, read from the statement digests with dump-shaped statements excluded. Six statements are shown: the heaviest by rows examined, not a survey.

Attribution is by shape and timing. Digest text is normalised — every literal becomes ? — so the digests cannot name a client. These queries are identifiable as ours because we wrote them and ran them in this window, and that is the whole of the evidence.

A high ratio is not automatically a fault. Any aggregate examines many rows to return few, and that is the job. These queries are not badly built. The issue is that they were run against production to ask whether production was scanning too much, and that they outweigh everything the application does apart from its backups.

The framework's introspection was not traced to a caller. We know it ran 4,368 times, but not which code path is responsible. The honest next step is a trace rather than a guess. It is left open, in the same way the earlier cases were left open until somebody read the digests.