Writing the standard

What the delivered work is checked against, published before you buy:

1. Every populated field carries a page and section cite to the source document
2. Field-level agreement with your labelled sample is at or above 95 percent
3. Every field you named appears for every document, blank where the source is silent
4. The file is valid UTF-8 CSV with your field list as the header row and one row per document

The full unit · Order it

What the delivered work is checked against, published before you buy:

1. Every case in your test file has a recorded pass or fail in the delivered run log
2. The workflow runs end to end on your data without manual intervention between steps
3. Any case that cannot pass is named with the reason, and the price is reduced pro rata
4. The delivered run log names the account and credential the workflow executed under

The full unit · Order it

For an extraction run, the field list is the header row of the file you get back. Name each field, say what counts as its value, and say what an absent value looks like. The published check is that every field you named appears for every document, blank where the source is silent, so a field you cannot define is a field that will come back blank and count against the agreement score.

• Field commencement_date - the date the term begins, as written in the document, ISO 8601. Blank if the document states only a trigger event.
• Field base_rent_monthly_usd - the first full monthly base rent, digits only, no currency symbol. Blank if rent is stated annually and no monthly figure appears.
• Sample Ten documents you have already extracted by hand, with those same field names. This is what the 95 percent agreement is measured against, so it decides whether the run passed. Pick ten that are typical, not ten that are easy.

For a workflow run, a case is an input plus the condition that decides whether the output was right. The published check is that every case has a recorded pass or fail in the delivered run log, so a case with no stated pass condition cannot be recorded either way, and a case that cannot pass gets named with its reason and the price drops pro rata.

• Case Given an invoice PDF with a purchase-order number in the header, the workflow files it under that PO and posts the total to the matching open PO line. Passes when the PO line total increases by exactly the invoice total.
• Case Given an invoice with no PO number, the workflow files it to the review queue and does not post anything. Passes when the review queue gains one item and no ledger line changes.
• Case Given an invoice whose PO is already closed, the workflow does not post and names the reason. Passes when nothing posts and the run log states the PO was closed.

Three cases is a real test. The failure cases are the ones worth writing, because a workflow that only handles the good path is the one that costs you money quietly.

1. A standard that arrives after the work does. It grades the work against itself.
2. A case with no stated pass condition. It cannot be recorded as passed or failed, so it cannot go in the run log.
3. A field with no definition. It will come back blank and count against your own agreement score.
4. An accuracy sample chosen because it was easy. It measures the wrong thing and you are the one who pays for that.