Software Evaluation Methodology

Software Evaluation Methodology Table of Contents Table of Contents.......................................................................................................1 Purpose.....................................................................................................................1 Summary..................................................................................................................1 Requirements............................................................................................................1 Software Candidates...................................................................................................1 Weighting..................................................................................................................2 Scoring.....................................................................................................................2 Total Cost of Ownership..............................................................................................2 Report......................................................................................................................3

Purpose Establish a common methodology when comparing, evaluating and selecting new software.

Summary Below 1. 2. 3. 4. 5. 6.

is a summary of the steps involved in selecting new software: Gather requirements Select software candidates Establish weighting Score/evaluate candidates Determine Total Cost of Ownership (TCO) Publish recommendation report

Requirements This step results in a set of requirements that will be used to evaluate and score the software candidates. The requirements will be gathered through interviews with each of the stakeholders (end users, managers, senior managers, etc). As the requirements are gathered they should be documented in a wiki to allow stakeholders the opportunity to review and edit in a collaborative fashion. Sign off must be received by all stakeholders before completing this step.

Software Candidates The goal of this step is to identify software candidates for evaluation. The criteria below must result in a minimum of three candidates. If the number of candidates is less than three then candidates will be selected based on the discretion of the project team. The final list of software candidates must be agreed upon by a majority of the project team members. •

At least one software application currently in use or purchased by the city (if one exists)

Page 1 of 3

Software Evaluation Methodology • •

The Gartner magic quadrant leader or another commercial market research source (if one exists) At least one open source software application (if one exists)

A conference call will be held for the survey participants (see below) to review requirements and software candidates as well as address any questions or concerns.

Weighting Once the requirements have been finalized, a survey will be created that allows participants to score each requirement on a scale of 1 (nice to have) to 5 (highly desired). The survey must allow participants the opportunity to add new requirements. The survey will be open for no less than 5 business days and no more than 10 business days. Once a survey is live the requirements cannot be edited. Only completed surveys will be used to calculate the weighting for each requirement by taking the average score (rounded to the tenth decimal). For example a requirement would have a weight of 4.2 if the following scores were provided: 3, 4, 5, 4, 5. Survey participants should include the following groups of users and reach as many individuals as possible: • Project team members • IT managers • Potential business users

Scoring Each of the software candidates will be rated with the assistance of the vendor. The list of requirements with weighting will be provided to the vendor at least five days in advance of a demo. The demo will be conducted on-site or via a web conference. Invitations will be sent to all survey participants to rate the software candidates five days in advance of the demo. A scoring sheet will be provided to all participants who must sign and date the form. Scoring participants will rate software candidates based on its ability to meet each requirement on a scale of 1 (poor) to 5 (excellent). In addition to internal scoring, the vendor will rate their application providing supporting narrative and any other artifacts such as screen captures to support their conclusion. A final score will be calculated for each requirement by taking the average of all ratings multiplied by the requirement weight.

Total Cost of Ownership To provide the best possible value for our constituents, an accurate understanding of total cost of ownership will be determined. The standard total cost of ownership worksheet will be provided to each vendor to complete. The vendor will have no more than 10 business days to complete the worksheet (they must submit costs using this form). Any assumptions, requirements or restrictions issued must be applied to all software candidates. For example, number of users or an integration with an external system.

Page 2 of 3

Software Evaluation Methodology Report The recommendation report will have the following sections: • Introduction • Executive Summary • Recommendation Summary • Evaluation Methodology • Software Candidate Descriptions • Conclusion • Appendix A - Total Cost Of Ownership Worksheets • Appendix B - Survey Results (all participants) • Appendix C - Scoring Results (all participants) The report will be distributed to all survey participants via email. A conference call will be at least five business days after distribution of the report to address any questions or concerns. Click here for a completed Recommendation Report Example. Click here for TCO worksheet Template and completed Example.

Page 3 of 3