← Field guide · what went wrong

Why a rising placeholder count is a good sign

  • Every unfilled figure on this site is a marked dash, not a blank.
  • The count went up. That was progress, and it took us a while to see why.

What we published

  • A finished site around results that do not exist. Eight pages, every figure a placeholder, every verdict Pending.
  • A count of those placeholders in our own notes, tracked as a measure of how close the site was to real.
  • The assumption underneath it: the number should fall.

What was wrong with the assumption

On 7 August the count went 144, then 128, then back to 144 in a day. It looked like the site had gone backwards. It had not.

What happenedEffect on the countWas it progress?
16 decorative placeholders removed — hero counts, section tallies, a metabar−16Yes
16 real ones added — a new out-of-sample block on three test articles, and a price-data row+16Yes
Net0Both halves were
  • The two moves were opposite in sign and identical in meaning.
  • Removing sixteen was good: a dash with nothing to explain it is just a broken-looking page.
  • Adding sixteen was also good: each one is a slot a real test now has to fill, because we made the method stricter.
  • A count that treats those two as opposites is measuring the wrong thing.

How it was found, and the correction

  • By re-running the count instead of quoting it. The remembered figure of 144 did not reproduce — the documented command returned 119 lines and 180 occurrences.
  • Whatever produced 144 was a different count, and nobody could say which.
  • So the number in our notes was not measuring what its label claimed, and had not been for some time.

A rising placeholder count means new slots a real test has to fill. A falling one is only good news when the numbers actually arrived.

  • The rule now: a placeholder may only appear in a slot a real test will fill, next to a Pending badge that explains it.
  • Decorative dashes are a bug, and they were all removed.
  • And the count is taken by running the command, never by repeating the last figure.

Why this is on the site at all

  • Because the dashes are the most visible thing about this site right now, and a reader deserves to know what they are.
  • They are not an unfinished page. They are a promise with the shape of the answer already cut out.
  • The alternative was filling them with plausible numbers, which is the one thing this site exists not to do.

Common questions

When do the placeholders get filled?

When a test publishes a result. Two problems in our own data are holding that up, and both are recorded rather than worked around.

Why publish a site with no results on it?

Because the method is the product as much as the numbers are. The Field Guide states no results at all and is finished, which is why it is the part open to search.

Is a placeholder ever just a formatting gap?

Not any more. That was the correction: a dash now only appears where a real test will put a number, and something on the page has to say so.

How do you count them?

By running the command, not by remembering. That is the part that went wrong.

What changed

A placeholder may only appear in a slot a real test will fill, and something visible must explain it.

The count is taken by running the documented command every time, because a remembered figure once failed to reproduce and nobody noticed for weeks.