The evidence pack: what your auditor receives

A full evidence pack for a fictional bank’s credit agent, produced by the same software as a real one. Its test plan is cut to twelve obligations so that it stays readable. SPECIMEN is printed on every page, and it is signed with a specimen key, never our production key.

Its test run has one failed test and one that needs review. The run summary lists both, each with its rate and one reply.

The bank, the agent, its replies and the people named in it are fictional. Everything else is the real product’s: the sections and their order, the law quoted word for word, and the signature scheme.

1. 01

   Cover

   The agent, the decision, the version and the content fingerprint. The fingerprint is printed again on every page.

2. 02

   Executive summary

   The AI Act classification, Article 5 and Article 50, how many tests and documents back the decision, and the decision itself. When the registration left scope open, a warning comes first, before any finding.

3. 03

   Registration

   What the agent does, the data and tools it uses, who it serves and how a person oversees it, as your team declared it.

4. 04

   Legal classification

   The AI Act classification, derived by rule from the registration, with the article behind each basis. The review priority, our own ranking of how much review attention an agent needs, is not printed.

5. 05

   Controls evaluation

   Each control with its result and any reviewer override with its note. Each regulation carries a label stating the depth at which it was checked.

6. 06

   Test run summary

   How many tests were sent and how they came out, how the agent was reached, the tests that failed or need review with their rate and one reply each, and any call to a tool the agent had not declared.

7. 07

   Per-obligation coverage

   Each applicable obligation quoted from the law and why it applies, then each test: the prompt, the expected behaviour, the finding across all attempts, and one reply with its attempt number and the judge’s reasoning. Documents appear with the passages quoted. Obligations out of scope are listed with the reason, and those that do not apply yet are kept apart.

8. 08

   Reviewer decision

   Approved, Approved with conditions or Rejected, with the notes and conditions, and whether the reviewer also registered the agent or configured its connection.

9. 09

   Audit trail

   The events behind the decision, in order, with the date and result of the last check that none was altered or removed.

10. 10

    Methodology

    The versions that produced the findings: the judge’s instructions, the test technique library, the obligation corpus and the coverage thresholds.

Every reply, not only the one printed, is in the run report and its per-attempt export.

• If your registration left scope open, page one says so before any finding.
• Every obligation left out is listed, with the reason.
• Tests that bear on an obligation only indirectly are labelled as such.

1. Compute the SHA-256 of the PDF.
2. Take the signature and the key id from the register line. For the specimen, that is specimen-signatures.json.
3. Check the Ed25519 signature over the hexadecimal digest with the public key. For the specimen, that is specimen-signing-key.json.

For a real evidence pack, the register line comes from the CSV export of your Evidence packs list, and the public keys are published at /.well-known/vidimus-signing-keys.json.

sha256sum vidimus-specimen-evidence-pack-en.pdf
printf '%s' "$(sha256sum vidimus-specimen-evidence-pack-en.pdf | cut -d' ' -f1)" > digest.txt
jq -r '.[] | select(.locale == "en") | .signature_ed25519_hex' specimen-signatures.json | xxd -r -p > sig.bin
jq -r .public_key_pem specimen-signing-key.json > key.pem
openssl pkeyutl -verify -pubin -inkey key.pem -rawin -in digest.txt -sigfile sig.bin

const { createHash, createPublicKey, verify } = require('node:crypto');
const pdf = require('node:fs').readFileSync('vidimus-specimen-evidence-pack-en.pdf');
const line = require('./specimen-signatures.json').find((e) => e.locale === 'en');
const key = createPublicKey(require('./specimen-signing-key.json').public_key_pem);
const hex = createHash('sha256').update(pdf).digest('hex');
console.log(hex === line.sha256 && verify(null, Buffer.from(hex), key, Buffer.from(line.signature_ed25519_hex, 'hex')));

No account, no call to Vidimus.

Classification record

The AI Act classification, whether you are the provider or the deployer, Article 5, Article 50 and any Article 6(3) exception, in one versioned document with its SHA-256 printed on page one.

PDF

Article 50 attestation

Page one of the classification record and its Article 50 section: which paragraphs apply, who holds each duty, and the test evidence for each paragraph.

PDF

Run report

Every test of one run with its outcome, and every reply with the attempts that gave it. Also as a spreadsheet, with one row per test or one row per attempt.

PDF, CSV

DORA register of information

The rows for templates RT.05.01 and RT.07.01, per agent in scope. Contract fields stay empty, for your contract system to fill.

CSV

AI literacy record (Article 4)

The answers to the AI literacy questions of the registration, per agent.

CSV