Do your IDP do diligence

Vendor Name: Date:

Base Score

Weighted

Criterion	Weighted Multiplier	Details	Score 0–2	Notes / Evidence
Fit to our documents		Can it handle our most used documents?
Accuracy on key fields		Is it accurate on what we care about? Can we see the exceptions?
Unstructured Capabilities		Is it understanding our most complicated documents?
Speed & capacity		Can it handle large volumes at speed?
Machine Learning		Does the solution exhibit continuous learning based on past corrections?
Human-in-the-loop		Can a human easily review mistakes and make changes to the model training?
Cost clarity		Is it outcome-based? Token-based? Usage-based? Are we okay with that?
Deployment		Can we choose where it runs based on our security and compliance needs?
Auditability		Can we control how the model is trained, monitor changes, and see how our data is used?
External users		Can external users submit documents and are they notified when inputs are missing?
Integration		Does it support the neccessary output formats to integrate results in our apps and databases?
Vendor focus		Does the vendor have a clear vision for the future with a product roadmap?