Similar documents, different owners or versions
Similar meaning is not identity. Ask whether issuer and version were used, or only the words.
How to compare
Ask the same six questions about every product
Including RAGForge. Remove options that fail a hard requirement. Test the remaining options on your documents. Compare full cost only among the ones that pass.
| Question | What to check |
|---|---|
| Will it use the correct source and version? | Similar documents, document identity, effective dates, and historical questions. |
| Will it refuse when the evidence is too weak? | When the check runs, what it measures, and whether the reason for refusing is recorded. |
| Can you test it on your documents? | Representative questions, unanswerable questions, and known correct sources, before you choose. |
| Can you investigate a wrong answer? | Whether it stores search results, the evidence decision, refusals, quality results, and timing. |
| Where will the data be processed and stored? | Documents, prompts, indexes, logs, external model calls, regions, and what it takes to leave. |
| What will it cost to run and maintain? | Service fees, licences, models, infrastructure, engineering, support, and operations. Not the cloud bill alone. |
Cases to put on the test
Test these before treating a demo as evidence
A question about an older version
If the source does not keep history, no product can reconstruct it. The honest result is a refusal.
Material spread across restricted systems
Connectors, permissions, and data rules may mean the answer you want cannot be assembled. Find that before a rollout.
Three delivery paths
Compare categories. Do not treat a feature table as a decision.
| Framework | Managed platform | RAGForge | |
|---|---|---|---|
| Decision to answer | The application team sets and tests the refusal rule. | Depends on the service and how it is configured. | Checks evidence before the final answer and records a reason when it does not answer. |
| Records | Tracing tools can be combined. The team chooses the fields and the store. | Platform logs and traces. What can be exported depends on the service. | Request, retrieval, and evaluation records in Postgres you host. |
| Where it runs | Wherever the team deploys it. | Supported regions, inside the provider's service boundary. | Your VPC or data centre. A fully offline setup only after every endpoint has been tested locally. |
| What you operate | Most of the system. | Less of the stack. Service limits and the provider's processing still apply. | The infrastructure and the operations. You license the application. There is no per-request fee to RAGForge. |
This table does not name vendors. A named comparison goes stale, and it should be checked against current documentation before it is shared. It also does not say which option is cheaper. Cost changes with region, volume, retention, support, and staffing. Calculate it for the actual use.
Decision rule
Remove what does not fit. Test the rest. Then compare cost.
Hard constraints first
Deployment rules, third-party processing, the systems that must be connected, available staff, and contract limits. A feature does not overcome a rule you cannot break.
Good cases and bad cases
Wrong source, wrong version, an answer that should have been refused, a refusal that should have been an answer, citations, time, and cost. On your documents.
Only among options that passed
Service charges, licences, infrastructure, engineering, operations, support, and what it costs to leave. The test may still say: improve what you have, or stop.