You have two, three or four programs on a shortlist. Every demonstration went well. Every salesperson answered yes. Now you have to choose one, justify the choice to a partner or a board, and live with it for years. What you want is a way to compare them that rests on evidence rather than on which presenter was more convincing. This article gives you that method, step by step, with a scoring sheet you can copy.
It assumes you have already decided to buy something ready-made. If you are still deciding between a package and a system built for you, start with how to decide between Odoo and a custom system and come back.
Step one: write down what the software has to do, before anyone shows you anything
The most common reason comparisons go wrong is that the buyer's requirements are formed by the demos. The first vendor shows a feature, it becomes a requirement, and the next vendor is judged on it. By the fourth demo the list describes the vendors, not your business.
So write the list first, from how your business actually works.
- Walk through your real processes. Order to delivery, purchase to payment, production from work order to packing, month-end. Note who does what and where it goes wrong today.
- Turn them into scenarios. Ten to twenty concrete cases, written as stories: "A customer orders 300 pieces in three sizes, pays half in advance, and changes the colour of one size after cutting has started." Include your awkward cases, not only the normal ones.
- Sort every requirement. A simple method is to label each one must have, should have, could have, or will not have this time. Be strict with the first label: a must-have is something without which you would not buy at all.
Step two: turn the scenarios into a demonstration script
Send the same scenarios to every vendor, in advance, and ask them to demonstrate those — in that order, with data that looks like yours — rather than their standard presentation. A vendor who cannot or will not do this has told you something important.
During each demonstration, one person follows the script and records, for each scenario, one of four outcomes:
- Works as standard.
- Works with configuration — settings, no programming.
- Needs development — someone has to write code. Note who, how long, and who maintains it through upgrades.
- Cannot be done.
The difference between the second and third outcomes is where most budgets go wrong, so press on it. "We can do that" means nothing until you know which of those two it is.
Step three: score them on a weighted sheet
Agree the weights before the demonstrations, not after, so the scores cannot be tuned to a favourite. A sheet like this is enough.
| Criterion | Weight | Program A | Program B | Program C |
|---|---|---|---|---|
| Fit to your must-have scenarios | 30% | |||
| Ease of use for the people who will use it daily | 15% | |||
| Arabic, right-to-left and local requirements (tax, e-invoicing) | 10% | |||
| Integration and data export | 10% | |||
| Reliability, security and backups | 10% | |||
| Support: hours, language, response times | 10% | |||
| Five-year total cost | 15% |
Score each cell from one to five, multiply by the weight, and add. The weights above are an example; yours should reflect what hurts most in your business today. If an option fails any must-have, it is out, whatever its total.
For the quality rows, the international reference is the ISO/IEC 25010 model of software product quality. Its current version breaks quality into characteristics such as functional suitability, performance efficiency, compatibility, interaction capability (what earlier versions called usability), reliability, security, maintainability, flexibility and safety. You do not need to use the standard formally, but running down its list is a good way to catch a criterion you forgot.
For the local row, check the details rather than accepting a tick. If you sell in Egypt, the system has to meet the tax authority's e-invoicing and e-receipt requirements; what an accounting system actually has to do for e-invoicing in Egypt lists what to verify. Check that printed documents — invoices, delivery notes, labels — come out correctly in Arabic, not only the screens.
For the cost row, use a five-year figure, not the first-year price. What business software really costs over five years shows how to build it so the programs are comparable.
Step four: let the real users try it
A demonstration is performed by an expert on prepared data. The people who will use the system every day are not experts and your data is not prepared.
Ask each finalist for a trial environment, load a small amount of your own data, and give three or four daily users a short list of tasks: create an order, record a delivery, find last month's invoices for one customer, correct a mistake. Watch without helping. Note where they get stuck and how long each task takes. Twenty minutes of this is worth more than two hours of presentation, and it is the only way to score ease of use honestly.
Step five: talk to customers the vendor did not choose for you
Every vendor has two or three happy references. Ask for them, and also ask for a customer similar to you in size and sector who went live in the last year. Then ask that customer:
- How long did it take to go live, compared with what was promised?
- What did you have to change in how you work?
- What did the vendor say could be done that later turned out to need development?
- How quickly does support answer, and in which language?
- What happened at the first upgrade?
- Would you choose it again?
The last answer matters less than the hesitation before it.
The traps
- Comparing feature lists. Every program claims every feature. A feature list tells you what exists somewhere in the product, not whether it handles your scenario.
- Choosing on first-year price. The cheapest subscription can be the most expensive system once implementation, customisation and renewals are counted.
- Demos on the vendor's data. Anything looks capable on clean sample data.
- Promises on the roadmap. Score what exists today. A feature "coming next quarter" is worth nothing in the comparison.
- One person deciding alone. Include the people who will use it and the person who will pay for it; they notice different problems.
- No exit. Before signing, know how your data leaves the system. What to settle before buying ready-made business software covers the contract terms.
The next step
Write your ten to twenty scenarios this week, and send them to every vendor on your list with a request to demonstrate them. The vendors' reactions to that request will narrow the list before you score anything. If none of the programs on your shortlist handles the scenarios that matter most, that is the case where a system built for you is worth pricing; Softwiro can look at the scenarios with you and tell you honestly whether a package would serve you better.
Sources
- ISO/IEC 25010: the product quality model — ISO 25000 portal. The quality characteristics used as a checklist above.
- Digital Outcomes and Specialists buyers' guide — UK Government. Assessing suppliers by scenario, case study and reference, with one scoring scheme and agreed weightings.
- Evaluation stage — UK Government Commercial Function. Evaluating responses against criteria and recording the reason for each score.