A dataset built by AI agents: an audit found 7 codes that silently became “allowed”

On 21 July 2026 at 11:04 UTC, a new feature appeared in the repository of our service kvednace.com.ua: for every KVED code, the service started showing whether that activity can be carried out under the single tax and in which group. At 12:01 the same day, a second commit appeared in the same repository, with fixes based on the results of an independent data audit. 57 minutes passed between them.

This article is about what the audit found, why it went unnoticed during compilation, and how to check your own model-built lookup table before people start using it. There is no tax advice here: which group to choose is something you decide together with your accountant.

How the dataset was built and what was written about it

The dataset is a table of verdicts. Each row holds a KVED-2010 code, the activity name, a level (“allowed”, “Group 3 only”, “partially”, “prohibited”), the basis in the Tax Code, a flag for whether it is a direct provision of the law or an interpretation, and a confidence level.

It was compiled by AI agents. The file header reads: “compiled and adversarially verified by a research pipeline (18 agents…)”. The same place carries a disclaimer that also appears on the service page: a prohibition in the law applies to the actual type of activity, and the code is only a guide.

When we compared the description with the file itself, the first discrepancy turned up right away. The first commit message promises “87 verdicts for KVED-2010 codes”, while the file from that commit has 70 rows. The service did not break because of it. But the discrepancy showed that the description an agent writes about its own work is not a verification of that work. Neither are the words “adversarially verified” in the header.

What the audit found

The audit was run separately from compilation. According to the commit message with the fixes, it was done by four auditors, each on their own category, with the instruction “don’t trust, verify”, and each finding was then separately checked against the code and the State Tax Service (DPS) clarifications.

The commit title says “8 fixes”. Counting by the list in the same message, there are nine changes:

• seven codes that were not in the dataset at all: 45.40 (sale of motorcycles), 46.39 (non-specialized wholesale of food, beverages and tobacco), 29.20 (bodies and trailers), 30.91 (manufacture of motorcycles), 90.04 (concert halls), 82.99 (among other things, independent auctioneers), 01.15 (growing of tobacco);
• a clarified note for 71.20 (technical testing): the prohibition in the code applies only to individual entrepreneurs (FOP), while a legal entity in Group 3 may perform such work;
• a fixed function that decides whether to show Group 4.

The last two changes are connected. Code 01.15 is one of the agricultural codes for which the service offers Group 4. Once tobacco growing got “prohibited”, without the logic fix the card would have shown both a prohibition under the single tax and an available Group 4 at the same time.

We rejected one audit finding. That is 47.19, “Other retail sale in non-specialized stores”. The audit proposed marking the code as restricted, but the prohibition here depends on the actual transaction, not on the code, so a flag on the whole class would be excessive. There is still no row for 47.19, so the service shows “allowed”.

Why nobody noticed the gaps

A gap looked exactly like a verified answer.

The service logic works like this: look for a row for the code; if there is none, look for a row for the division (the first two digits); if there is none of that either, return “allowed”. For most codes this is the correct answer, because the single tax allows most types of activity and there is no point in writing out hundreds of identical rows.

The problem is elsewhere. “No row because everything is allowed” and “no row because it was forgotten” did not differ on screen. It was even worse: a row marked as an interpretation rather than a direct provision gets the label “tentative” from the service, while a missing row gets no such label. A forgotten code looked more confident than an honestly flagged disputed one. The only visible difference: a card without a row has no “Basis” item.

We will not reconstruct exactly how the agents divided the work: there is no log of their steps in the repository. Our explanation is this. Searching for prohibitions produces rows, and checking rows sees only what has already been written. A gap can only be found when someone goes from the rule to the codes, not from the codes to the rule.

How much this affected

We calculated the figures below on 16 September 2026 with a script that replicates the service logic, on the dataset versions before and after the fixes. This is the effect on the service’s answers, not on anyone’s decisions.

Seven new rows changed the verdict on 27 pages of new codes, because for a new code the service takes the strictest verdict among the old codes that map into it. 82.99 contributed the most: in the service’s correspondence table it splits into 24 new codes.

We did not check whether the unfixed version reached the live site during those 57 minutes, so we do not claim that anyone saw a wrong verdict. As of 16 September 2026, the page for code 45.40 on kvednace.com.ua shows “Partially allowed” with a basis from the code.

How to check your own lookup table

This works for any table a model filled in: verdicts, parts compatibility, size charts, delivery rules by region. The first two steps are a quick check that gives the most. The rest are changes to the product and to how you set the audit task.

1. Find the default value. Search the code for ??, || , .get(key, default) or an else branch after the table lookup. Write down what a key that is not in the table gets.
2. Count how many keys fall into that branch. Take the full list (classifier, catalog, list of cities) and run every key through the same lookup function the site uses. For us, that is 523 codes of 615. The number itself does not matter; what matters is knowing it and understanding whether all those keys really should get the default answer.
3. Make absence visible. At least in an internal mode, show “not in the lookup table” as a separate state, not as the same “yes” as for a verified row.
4. Compare the description with the file. How many rows the commit message, README or agent report promises, and how many there actually are. A grep -c on the key field is enough. We got 87 versus 70.
5. Give the auditor a different task from the compiler. Not “check these rows”, but “here is the rule, find all keys it applies to and tell me which of them are missing from the table”.
6. Record rejected findings together with the reason. Our 47.19 sits in the commit message with an explanation, and the next audit will not raise it again without new arguments.

What we changed and what that did not fix

Seven codes added, the note for 71.20 clarified, Group 4 no longer shown for prohibited activity. All of it sits in one commit in the repository history.

The design itself remained: 523 codes still get “allowed” without a row, and on screen this still does not differ from a verified permission. The audit closed the gaps it found, but did not prove there are no others. That is why the single tax page carries a reminder: the prohibition applies to the actual type of activity, not to the code itself.

A business owner should take one thing from this story: a lookup by code does not protect you from a prohibition based on what you actually do. If the service shows a code as allowed, you still cannot trade under it in goods that the Tax Code prohibits under the single tax.

For a technical reader, something else: data from AI has to be checked not only for what is written in it, but also for what is missing from it. We wrote about a similar trap in code, where the result looks correct and only an external check catches the error, in our breakdown of development assistants. And reference services where each page answers one question are something we build as knowledge bases.