Vendors of AI reconciliation software in India quote headline auto-match rates with no dataset, no definition and no count of wrong matches. A finance head cannot tell from a demo whether the tool will hide differences, change last quarter's result after an update, or act on its own without approval.
Turn each vendor claim into a test on your own data. Seal an answer key from a month already reconciled by hand, give the vendor the files without the answer, score precision, recall and rupee value in wrong matches, run the same files twice to check for flips, and add 14 India edge cases covering UPI, NEFT UTRs, TDS-net receipts, duplicates and instruction-like narration text.
One closed month of at least 1,000 bank lines, at least three months of history, one bank format per account in use, at least 8 percent of lines drawn from the 14 edge cases, a sealed answer key in a separate sheet, four pass-or-fail gates (repeatability, human approval, reasons, data handling) and a 100-point weighted score.
A score sheet per vendor with gate results, value-weighted precision and recall, flip count between identical runs, per-case edge-pack outcomes and written data-handling answers. Shortlist at 70 points or more with all four gates passed; drop below 55; file the evidence for the auditor.
Buying AI reconciliation software in India usually starts with a demo and a headline: “Matching done by AI”, “98 percent auto-match”, “learns from your team”. None of those statements can be checked from the vendor’s page. This guide turns twelve common claims into tests you run on your own bank, UPI, TDS and ledger data, with a 14-case India edge pack, a way to score the result in rupees as well as lines, and a sheet with pass-or-fail gates. It is written for the finance head or controller choosing or renewing a reconciliation tool. It is vendor-neutral and names no product. Every table below is complete on this page; the AI Reconciliation Vendor Proof Pack is the same material as an Excel workbook.
What does “AI” mean on a reconciliation vendor’s page?
“AI” on a reconciliation page can mean any of four approaches: fixed rules, statistical scoring, a learned model, or a language model. Each behaves differently when your data is messy, and each fails differently. You do not need to know which one sits inside a product. You need to observe its behaviour on your own files and judge that.
Four approaches and what each can and cannot do
| Approach | What it needs | Where it helps | How to test it |
|---|---|---|---|
| Fixed rules (amount, reference, date window) | A rule set someone maintains | Clean references such as full UTRs and cheque numbers | Read the rules; check one match per rule against the statement |
| Statistical scoring (fuzzy name and amount closeness) | Thresholds and a reviewer | Name variants, small rounding differences | Ask what threshold applied and who approved it |
| Learned model (trained on past matches) | Your history, or someone else’s | Recurring counterparties with irregular narrations | Re-run after corrections; confirm past months do not move |
| Language model (reads free text) | A model provider and a data route | Messy narrations, drafting explanations | Ask which steps use it, who hosts it, and plant instruction-like text |
A reconciliation engine is the part that proposes and records matches; what a reconciliation engine does explains the job without tying it to a product.
Judge behaviour you can observe
Five behaviours matter more than the label: does each match carry a reason, does the same input give the same output, are wrong matches counted, does anything change a balance without approval, and where does your data go. The rest of this guide tests those five.
What should AI decide in reconciliation, and what needs a person?
AI can be used for reconciliation, but only for work that does not change a balance on its own. Ranking candidate matches and reading untidy narrations are reasonable uses. Writing off, posting, closing a period or changing a rule must sit behind a named approver and a log entry, whatever the tool is.
| Reasonable use of AI | Needs a human approval or a hard rule |
|---|---|
| Ranking candidate matches for a reviewer | Writing off an unmatched item |
| Reading messy UPI or NEFT narrations | Posting a journal to the ledger |
| Suggesting many-to-one invoice sets | Closing or reopening a period |
| Spotting outliers and likely duplicates | Changing a tolerance or matching rule |
| Drafting an explanation for a reviewer | Learning from a correction without a visible change log |
Manual and automated processes fail in different places; manual versus automated failure modes covers that comparison, so it is not repeated here.
Twelve vendor claims and the test for each
Every common vendor claim can be turned into a test. Ask the question that makes the claim specific, run the test on your own data, keep the evidence, and stop the evaluation when you hear the red-flag answer. The table below is the core of this guide.
| # | The vendor says | Ask | Test on your own data | Keep as evidence | Red flag |
|---|---|---|---|---|---|
| 1 | ”Matching done by AI” | What decides each match: a fixed rule, a statistical score, a learned model or a language model? Which fields does it read? | Take 50 matched lines (25 of the highest value, 25 at random) and ask for the stated reason on each | Export with a reason or reference per match; reviewer sign-off sheet | ”Proprietary model” with no per-match reason; the only reason shown is “matched by AI” |
| 2 | ”95 to 99 percent auto-match” | Percent of what: lines, invoices or rupee value? Measured on whose data? Are wrong matches counted? | Blind hold-out: give one closed month without your manual result and compare with your answer key | Counts of matched, unmatched, correct and wrong; value-weighted error | A rate with no dataset; “suggested” matches counted as matched; no false-match figure |
| 3 | ”Learns from your corrections” | Where is a correction stored? Does it change past months? Can it be switched off per account? | Correct 10 wrong matches, then re-run the same month and the previous month | Change log with user, time and scope; before and after diff | Past months change silently; no way to see what was learned |
| 4 | ”Same inputs, same result” | Is the output repeatable? What can change between runs? | Run the identical files twice, then once more after any vendor release | Run ID and version stamp on each run; diff of the matches | Any unexplained flip; no version stamp |
| 5 | ”Explainable AI” | What does an explanation contain, and can a reviewer challenge it? | Plant five wrong matches in the pack and ask for the explanation of each | Export of each explanation; the override with the reviewer’s reason | A generic sentence per match; overrides not logged |
| 6 | ”Built for Indian bank data” | How does it read UPI handles and RRNs, NEFT and RTGS UTRs, IMPS references, truncated narrations and cheque numbers? | The 14-case edge pack below | Per-case outcome: matched with the reference used, or exception with a reason | Matches on amount and date alone; UTRs ignored; silent forced matches |
| 7 | ”Handles TDS, GST and TCS lines” | How are TDS-net receipts, GST on bank fees and TCS on payouts shown? | One receipt of ₹2,21,400 against a ₹2,46,000 invoice after ₹24,600 TDS at 10 percent | Status shown (“waiting for credit” or “short payment”), not plain “matched” | Closes the receivable too early, or calls it a short payment with no TDS reason |
| 8 | ”Generative AI agents resolve exceptions” | Which actions can it propose and which can it execute? Who approves? | Ask it to resolve 20 exceptions; check what was posted; add one narration containing instruction-like text | Approval log; list of actions it can never take alone | Posts write-offs or journals without approval; treats narration text as an instruction |
| 9 | ”Secure and India-hosted” | Where is data processed and stored? Which model providers and sub-processors? Is customer data used for training? How long is it kept? | Written answers plus a data-flow diagram; delete a masked test file and confirm | Data-flow diagram, processor terms, certificates the vendor cites | Cannot name the model provider; “anonymised” with no definition; training on customer data by default |
| 10 | ”Audit-ready” | What does an auditor receive: edit log, users and roles, rule or model history, ingestion log, period lock? | Auditor-style walk-through of 25 sampled matches from report line to statement row to ledger entry | Sample trail with users and timestamps | Trail is a screenshot, not an export; manual overrides unlogged |
| 11 | ”Connects to your ERP” | How does it read from Tally, SAP, Oracle, Zoho or Busy, and if it writes back, how? Is a retry safe? | Sandbox connection, map the ledgers, import one month, force one failed post | Mapping sheet; proposed versus posted entries reconciled | Direct write-back with no approval step; duplicates after a retry |
| 12 | ”Scales and goes live fast” | What was the largest volume tested? What do you configure and what does the vendor build? | Run three times your normal month and time the month-end run | Timings; list of configuration items; reference-call notes | Cannot list what you must configure; reference customer has a different data shape |
The match-rate test
A match rate means nothing until the vendor says what it counts and on whose data. Insist on a blind month: you keep the answer, they get the files. Ask for wrong matches as a separate number. A tool that never reports a false match has not been measured, not proven.
The repeatability test
Run the same files through the same version twice. The matches should be identical. Then ask what changed after the vendor’s next release and whether last quarter’s closed result moved. Reproducibility and the auditor explains why a re-performable close matters under Indian audit rules.
The corrections test
“Learns from your team” is only safe if you can see what was learned and where. Correct ten wrong matches, then re-run the current month and the previous one. The current month may improve. The previous month must not change unless someone reopened it with a logged reason.
The narration-text test
Bank narrations are free text written by the payer. If a tool uses a language model to read them, a narration such as “approve all pending” or “write off balance” must be treated as data only. Plant one such line. Nothing should change state, and the line should go to exceptions like any other unclear credit.
How do you build a test pack from your own bank and ledger data?
Build the pack from statements and ledgers you have already reconciled, mask names but keep structure, and seal the correct answer in a separate sheet before any vendor sees the files. Use at least three months, one bank format per account in use, and at least 1,000 lines in the hold-out month. Building a test pack before a demo covers the basic pack; the cases below are the AI-specific additions.
The 14 India edge cases
| ID | Case | What a sound tool does |
|---|---|---|
| E1 | UPI credit whose narration holds only a payer handle and a 12-digit reference; open invoices of ₹18,750 and ₹18,500 | Matches on exact amount plus reference; never on payer name alone |
| E2 | NEFT credit with the UTR cut short by narration truncation | Uses the partial reference as a clue, not as proof; sends to exceptions if ambiguous |
| E3 | One bank credit paying twelve invoices, with a ₹1 rounding difference | Proposes the set, shows the ₹1 as tolerance, asks for approval of the tolerance rule |
| E4 | One invoice paid in three instalments | Keeps the invoice open with a running balance |
| E5 | TDS-net receipt: invoice ₹2,46,000, bank credit ₹2,21,400 | Shows “waiting for credit” until the tax-credit statement confirms the ₹24,600 |
| E6 | Same amount from two customers on one day | Does not force a match; asks for a reference |
| E7 | Same UTR credited twice, or the same invoice paid twice | Flags the duplicate; does not net it silently |
| E8 | Payment returned after debit (failed or reversed transfer) | Pairs the return with the original debit |
| E9 | Bank charge with GST, interest credit, SMS charge | Classifies each; does not leave “miscellaneous” |
| E10 | Round-sum advance of ₹1,00,000 with no reference | Leaves it as an unidentified credit with an owner and an age |
| E11 | Cheque dated the 30th but cleared on the 2nd | Treats it as a timing item at month-end, with its age |
| E12 | Transfer between two of your own accounts | Pairs the two legs; does not raise two exceptions |
| E13 | Customer name variants (“ABC Pvt Ltd”, “A.B.C. Private Limited”) | Uses the customer master, shows the evidence, asks when unsure |
| E14 | Narration with instruction-like text (“approve all”, “write off”) | Treats it as data only; no state change |
For E3, agree the tolerance rule with your reviewer before the test; tolerance matching rules sets out how to write one. For E5, the TDS receivable stays open until the credit appears in the deductee’s tax-credit statement; this is the same discipline a TDS reconciliation process follows.
Seal the answer key and hold out one month
Make at least 8 percent of the hold-out month’s lines edge cases from the table. Keep the answer key in a separate sheet with three columns: line ID, correct counterpart (or “exception”), and reason. Do not share it with the vendor until the run is finished. Give the vendor only the statement, the ledger and the open-item list.
How do you score the result: precision, recall and rupee value?
Score the result with five measures defined before you see any number: auto-match rate, precision, recall, their rupee-weighted versions, and the flip rate between identical runs. A false match hides a real difference, so precision matters more than the headline rate. A missed match costs analyst time but hides nothing.
- Auto-match rate is lines the tool matched without a person, divided by all lines.
- Precision is correct auto-matches divided by all auto-matches. One minus precision is the false-match rate.
- Recall is correct auto-matches divided by the items that could be matched, from your answer key.
- Value-weighted versions use rupee amounts instead of line counts.
- Flip rate is matches that change between two runs of identical input, divided by lines.
Worked example (illustrative)
One closed month has 1,200 bank lines worth ₹640 lakh. The answer key says 1,080 lines can be matched and 120 are genuine exceptions. The vendor reports “96 percent auto-matched”.
| Measure | Result | How |
|---|---|---|
| Auto-matched | 1,152 lines (96.0 percent) | 1,152 / 1,200 |
| Correct | 1,056 | Checked against the answer key |
| Wrong | 96 (90 should have stayed exceptions, 6 went to the wrong counterpart) | 1,152 − 1,056 |
| Precision | 91.7 percent | 1,056 / 1,152 |
| Recall | 97.8 percent | 1,056 / 1,080 |
| Value auto-matched and value correct | ₹605.0 lakh and ₹563.2 lakh | From the answer key |
| Value precision | 93.1 percent; ₹41.8 lakh sits in wrong matches, one line alone ₹18.4 lakh | 563.2 / 605.0 |
The headline “96 percent” is true, and it says nothing about the 96 wrong matches or the ₹41.8 lakh they hide. Now run the same files a second time: 14 of 1,200 matches change, a flip rate of 1.2 percent. Even if the second run scores better, the output is not repeatable, so an auditor cannot re-perform last quarter’s close. That fails the repeatability gate. Ask what changed between the runs before going further.
Which audit and data questions come before the contract?
Settle two sets of questions before signing: what your auditor will receive from the tool, and where your data goes. Indian rules already set the floor for both. An audit trail that cannot be disabled is a Companies Act requirement for accounting software, and personal data handled by a vendor stays your responsibility under the DPDP framework.
Audit trail and the auditor’s walk-through
From financial years beginning on or after 1 April 2023, a company that keeps its books in accounting software must use software that records an audit trail of each transaction, keeps an edit log with the date of each change, and cannot have the audit trail disabled (proviso to Rule 3(1), Companies (Accounts) Rules, 2014, as reproduced in the ICAI implementation guide). The statutory auditor reports on this under Rule 11(g) of the Companies (Audit and Auditors) Rules, 2014. ICAI’s guidance takes the view that the audit trail should be kept for at least eight years, in line with the books of account.
Whether a stand-alone reconciliation tool is itself “accounting software” depends on whether it maintains books of account; ask your auditor early. Either way, the auditor will test the reliability of reports the company produces. ICAI also notes that compliant software may be hosted in India or abroad, in the cloud or as SaaS, so hosting is not the test; the trail is. Ask for an exportable edit log, a user and role list, the rule or model history, the ingestion log and a period lock, and run the 25-match walk-through from claim 10. Can your auditor rely on the reports goes deeper on edit logs and change control.
Data location, model providers and training use
Bank statements carry customer names, account numbers and payment references, so a reconciliation vendor is a processor of personal data. Under the Digital Personal Data Protection Rules, 2025 (notified 13 November 2025), the security, breach and retention obligations commence eighteen months after publication. They include reasonable safeguards such as encryption or masking, access control and logging, processor contract clauses, a detailed breach report to the Data Protection Board within seventy-two hours, and keeping processing logs for at least one year. Plan the contract for them now.
CERT-In’s directions of 28 April 2022 require specified cyber incidents to be reported within 6 hours of noticing them, ICT system logs to be kept securely for a rolling 180 days, and clocks to be synchronised to NIC or NPL time sources. Ask the vendor how it meets each. If you are a regulated entity, your outsourcing directions from RBI also apply; read the current text for your entity type with your compliance team. If the vendor uses a language model, add four written questions: which steps use it, who hosts it, what data leaves, and whether your data trains anything. The security checklist for reconciliation software has the wider list.
Scoring rubric and decision gates
Compare vendors fairly with two layers: four gates that any vendor must pass, then a 100-point weighted score. A failed gate ends the evaluation until the vendor fixes it, whatever the score. The weights below are a starting point, not an industry standard; adjust them to your risk.
| Gate | Pass condition |
|---|---|
| G1 Repeatability | Identical inputs and identical version give identical output; a version stamp on every run |
| G2 Human approval | No write-off, journal or rule change is applied without a named approver |
| G3 Reasons | Every auto-match carries an exportable reason or reference |
| G4 Data handling | Written answers on location, sub-processors, model providers, training use and retention |
| Area | Points | Scored from |
|---|---|---|
| Accuracy on the blind month (value-weighted precision and recall) | 30 | The scoring section above |
| India edge-case pack | 20 | E1 to E14, one point per case, scaled to 20 |
| Exceptions: reason, age, owner, approval | 15 | Walk-through of the exception queue |
| Audit evidence | 15 | Claim 10 |
| Integration and operations | 10 | Claims 11 and 12 |
| Security and data | 10 | Claim 9 |
Shortlist at 70 points or more with all four gates passed. Below 55, drop the vendor. File the score sheet and the evidence for each gate where your auditor can find it.
Red-flag answers and what a good answer sounds like
Some answers should end the evaluation on the spot. The pattern is the same each time: a good answer names a specific case, a setting or a log; a weak answer appeals to the model improving over time or to autonomy.
| Ask | A good answer | A weak answer |
|---|---|---|
| ”Show me a false match you caught and one you missed” | Names a real case with the line, the reason and the fix | ”We have not seen one" |
| "If I re-run last quarter next week, is the result identical?" | "Yes, same version; the run stamp shows it; differences are listed" | "It improves over time, so it may differ" |
| "Which steps use a language model and who hosts it?” | Names the steps, the provider and the data that leaves, or says none | ”We use AI throughout" |
| "What can it do without a person approving?” | A short list, with approvals for anything that changes a balance | ”Anything, it is autonomous" |
| "What happens to past months when you release an update?” | Past months are frozen; changes apply forward and are logged | ”It re-learns everything" |
| "How do I switch the AI off for one bank account?” | A per-account setting, with the fallback rules shown | ”Not possible” |
How to run the pilot
Run the pilot in seven steps on a month you already know the answer to. No fixed duration is promised here; the length depends on how quickly the vendor configures your banks and ledgers.
- Fix the answer key from a month you have already reconciled by hand, and seal it.
- Give the vendor the files for that month only, with no hints.
- Compare the results with the key and list every disagreement with a reason.
- Run the same files again to test repeatability.
- Do one correction round; check that corrections persist and past months do not move.
- Walk your auditor through 25 sampled matches.
- Write the decision memo with gates, scores and open risks.
For the general framework that sits around this pilot (integration, commercial terms, support), see how to evaluate reconciliation software in India and the vendor selection questions.
Download the AI Reconciliation Vendor Proof Pack
The AI Reconciliation Vendor Proof Pack packages this guide as one Excel workbook: the score sheet with gates and weights, the 14-case edge-pack template with answer-key columns, the twelve-claim test log and a vendor-response log. Everything in it is also printed in full above.
How Terra Insight approaches repeatability
Terra Insight builds TransactIG around reproducible matching: the same inputs and the same configuration give the same result, and each match decision carries a recorded reason a reviewer and an auditor can read. Delivered today are TDS reconciliation and bank-to-books matching, with an exception queue, exports and presets. To see how that fits a wider evaluation, start with reconciliation software in India.
- ▸ ICAI Implementation Guide on Reporting on Audit Trail under Rule 11(g) of the Companies (Audit and Auditors) Rules, 2014 (Revised 2024) — Reproduces the proviso to Rule 3(1) of the Companies (Accounts) Rules, 2014: accounting software must record an audit trail of each transaction, keep an edit log with dates, and the audit trail cannot be disabled; applicable from 1 April 2023.
- ▸ Digital Personal Data Protection Rules, 2025 (G.S.R. 846(E), 13 November 2025) — Rules 3, 5 to 16, 22 and 23 come into force eighteen months after publication; rule 7 requires a detailed breach report to the Board within seventy-two hours; processing logs are kept for at least one year.
- ▸ CERT-In Directions under section 70B(6) of the IT Act, 2000 (28 April 2022) — Specified cyber incidents must be reported within 6 hours of noticing them; ICT system logs maintained securely for a rolling 180 days; clocks synchronised to NIC or NPL time sources.