Tender Evaluation Process Notes
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Tender Evaluation Process Notes as PDF for free.
More details
- Words: 3,648
- Pages: 18
Tender Evaluation Process Notes
May 2009
Tender Evaluation Process Notes
Contents 1. INTRODUCTION, PURPOSE AND SCOPE........................................................... 3 2. TENDER RESPONSE EVALUATION AND SELECTION.......................................... 4 2.1 OVERVIEW ...................................................................................................... 4 2.2 EVALUATION PROCESS ........................................................................................ 4 2.3 EVALUATION TEAM .......................................................................................... 13 2.4 EVALUATION CRITERIA ...................................................................................... 16 2.5 EVALUATION AND SCORING PROCEDURE ................................................................. 17 2.6 COMMUNICATION WITH RESPONDENTS .................................................................. 18
Page 2 of 18
Tender Evaluation Process Notes
1. Introduction, Purpose and Scope This document contains some notes on a possible approach to tender evaluation.
Page 3 of 18
Tender Evaluation Process Notes
2. Tender Response Evaluation and Selection 2.1 Overview The objectives of the evaluation process are twofold: • •
To select the most appropriate proposal to meet requirements To create an evaluation audit trail and set of supporting documentation that will support the evaluation process and final decision.
The output from the process will be a Vendor and Solution Evaluation and Selection (VSES) report. The document describes the evaluation criteria that were applied to responses to the tender. Any evaluation process is liable to subjectivity. The evaluation process described below can be subjective in: • •
The selection of the evaluation factors and the relative weights assigned The scores assigned to each vendor
The purposes of a formal evaluation process are: • • •
To eliminate subjectivity as much as possible from the evaluation process To provide transparency in the evaluation process To ensure that all proposals are evaluated in the same way
2.2 Evaluation Process The generalised solution evaluation process can be customised to meet the exact needs. The VSES report will have the following general structure: Part A – Overview 1. Executive Summary Part B – Short List Selection 2. List of Vendors Contacted 3. Product Decision Model 4. RFP Findings 5. Summary of Short List Recommendation Part C – Finalist Selection 6. Reference Checks and Site Visits 7. Functional Demonstration Findings
Page 4 of 18
Tender Evaluation Process Notes
8. Technical Review Findings 9. Solution Demonstration Planning Lab Findings 10. Summary of Finalist Recommendation Part D – Appendices Containing Supporting Material (optional) Schematically, the high-level view of the evaluation process that is proposed to be used during the evaluation and selection of a reporting solution is:
Page 5 of 18
Tender Evaluation Process Notes
In summary the evaluation process consists of the following elements:
Page 6 of 18
Tender Evaluation Process Notes
•
Evaluation Team - the evaluation team consists of individuals with appropriate skills to be able to evaluate responses received from vendors
•
Evaluation Process - the evaluation process is the set of steps to be followed by the evaluation team to conduct an evaluation of the responses
•
Evaluation Criteria - this is a set of criteria by which the responses will be assessed
•
Evaluation Score - the score is an objective assessment of the merits of a response based on the application of the evaluation criteria
•
Evaluation Report - this is a document describing the evaluation process and its results
In more detail, the steps in the process are: Step Start VSES Process
Description The amount of effort involved in starting the Vendor and Solution Evaluation and Selection (VSES) step depends on:
How recently you made any VSES scope and direction decisions Whether the team has already been selected How familiar each team member is with the functional area within which the application is to be deployed, and the systems currently in use in BoI to support Monthly Financial Performance Reporting • How experienced the team is with vendor and solution evaluation and selection. Confirm Scope Review previous decisions made about the scope of the project. and Approach Discuss any scope changes and adjust the work plan to reflect approved changes. • • •
Consider the urgency of need, and determine how it impacts the selection strategy. Determine the timing and general strategy for contract negotiation.
Establish and Train the Evaluation Team
Determine what parties to involve in defining procurement procedures, handling legal questions, and negotiating the contract so that the right people can be involved at appropriate points. Assemble and organise a solution evaluation and selection team. Observe the following guidelines: • •
Include technical personnel who understand the systems management and support processes that are being implemented Include business personnel who understand the operational objectives, business processes and needs of Executive
Page 7 of 18
Tender Evaluation Process Notes
Step
Description stakeholders and sponsors of the project. • Include subject matter experts in Best Business practice in Business Performance reporting as needed. • Include experts who understand the market context within which the vendors operate so as to ensure an understanding of: o The capability and capacity of service providers on the local market where resources will be required o The extent to which the vendors solutions will need to be customised to meet the requirements identified in the RFP o The market for services based activities (consulting/training/application development etc) • Provide for a workshop facilitator with a solid background in the subject area if you plan to use facilitated workshops as part of the evaluation and selection process. • Keep the core group small but representative of the functional areas the product most affects. • Include as support members anyone involved in defining procurement procedures, handling legal questions, and negotiating the contract. • Include management personnel responsible for approving the selection Once you have established the evaluation and selection team, designate team members to be the primary and secondary vendor contacts. The secondary vendor contact works with the vendor when the primary contact is unavailable, such as during vacation or illness. These individuals represent the interests of the client and the systems integrator, answer questions about the solution evaluation and selection process, strategise with the vendor, arrange vendor contact with other members of the team, and are empowered to make informal commitments. Next, designate a team librarian to administer the vendor scheduling, mail all vendor requests, record all vendor contact, obtain and distribute vendor materials, and maintain a filing system for vendor materials. Walk through the evaluation and selection activities with team members to familiarise them with the general process. If the team is relatively inexperienced with solution evaluation and selection, consider using a team-building exercise to preview the process. Scale down and simulate evaluation and selection activities. Save detailed discussions for just-in-time training sessions. Discuss the objectives and principles of the evaluation and selection process with the team to ensure that everyone has a consistent understanding.
Page 8 of 18
Tender Evaluation Process Notes
Step
Description Compile Compiling criteria when selecting a solution involves: Function • Developing or gathering criteria appropriate to the evaluation al method Criteria • Avoiding excess precision • Keeping the focus on business priorities • Designing a score sheet that facilitates getting and analysing vendor responses • Incorporating the criteria into the score sheet. These criteria will be taken from the tender. In this activity, define the functional criteria - the requirements as seen from the business user’s point of view. These requirements represent the business processes the solution needs to accommodate. For more information on developing application selection criteria and designing a score sheet, see the Product Evaluation technique. Design functional criteria and group them as follows: • • • •
Compile Technical Criteria
Business processes Data External system interfaces Business performance measurement.
Assemble the functional criteria into one document, the Functional Criteria List. Define technical criteria to represent the solution design and technical infrastructure aspects of the solution. The technical requirements concern the solutions operation and infrastructure. To enhance readability and minimise oversights, develop categories of technical criteria in the areas of: • • • •
Systems management and support process Operational interface Technical infrastructure Volume and technical performance.
Assemble the technical criteria into one document, the Technical Criteria List. These criteria will be taken from the tender. Compile General Criteria
The buyer of a reporting solution is buying more than a service. In most cases he or she is also developing a long-term relationship with the vendor, expecting some, if not all, of the following:
Page 9 of 18
Tender Evaluation Process Notes
Step
Description • Initial transition, configuration and implementation services • Custom services • Consulting services • Future changes • Customer support services, such as a hot line and a user group. General criteria include “vendor” and miscellaneous criteria. These criteria are factors that influence the solution purchase decision but are not functional or technical in nature. General criteria require that you answer them through an iterative process, which requires different tactics: Concentrate on criteria that would eliminate vendors from the short list. Consult with legal and purchasing personnel to extract general criteria that can quickly narrow the solution selection. Add criteria to ensure the team understands the vendor’s decisionmaking and delivery processes, schedules, and organisation before making a final solution selection and developing an implementation plan. With functional and technical criteria, you are usually looking for a specific response. With general criteria questions, you are looking for potential risks and information needed for deployment. Collect responses to these criteria by asking the vendor directly, by asking references who use the software, and by doing your own research. Develop categories to enhance readability and minimise oversights. Assemble the categories of general criteria into one document, the General Criteria List.
Determine Scoring Approach
Conduct RFP Response Evaluation
These criteria will be taken from the tender as well as other more specific requirements that extend the material in the tender. To narrow solutions options to a reasonable number, design an overall package selection framework to communicate and summarise findings. Use weighting to reflect the relative importance of criteria; use scoring to indicate how well the vendor’s solution meets the criteria. Weighting and scoring are often represented numerically, giving the illusion of objectivity. Keep in mind that these numbers are really just a quick means to communicate subjective judgments. When making the package decision, check that the bottom-line numbers are consistent with your overall judgment. The objective of this activity is to obtain and evaluate comprehensive vendor proposals and presentations that explain how their products meet the specific criteria, and at what estimated cost. The RFP has information instructing vendors to respond formally, in writing, and possibly in a presentation. Vendors are
Page 10 of 18
Tender Evaluation Process Notes
Step
Description compared uniformly and strictly as to how well they meet the criteria, comply with the instructions, and present their solution.
The responses to the RFP will be evaluated with respect to the criteria developed above. Develop Script During the solution meetings with vendors, the team needs to Cases compare solutions fairly and evenly, even though different solutions are being proposed and discussed. This is hard for the team to do if each vendor is allowed to determine the meeting’s sequence, contents, and emphasis. By prescribing that all vendors follow a planned sequence of requirements, or scripts, you: Ensure consistency of content coverage across vendors Ensure the vendor is presenting those features most important to you, not just those the vendor puts forward as strong points • Can more easily spot features and functions that do not quite match expectations • Can more easily distinguish how the features and functions differ among solutions. Select Vendor Conduct a workshop or a work session to obtain team consensus on whether or not to proceed with solution evaluation and to reduce Short List the list of vendors under consideration to three or fewer. In general, use a workshop for large solutions or solutions with a major impact such as the service being requested. Check Vendor Check vendor references as another way to confirm that the References vendors’ solution and services could meet the requirements. Develop a questionnaire to record reference responses. Include questions covering relevant functional, technical, and general criteria. Involve members of the team in reference checking. This distributes the work, counteracts individual bias, and provides experience useful in subsequent VSES activities. Have the short list vendors conduct functional and operation walkConduct Functional and throughs using the script cases provided, so you can: Operational • Confirm that the solution, can process operate to your Review Meetings satisfaction • Determine, based on your own observations, which solution provides the best answer to needs Conduct a technical review of the solution with the vendor’s Perform technical staff. The vendor’s technical staff members are usually Technical Review involved in the provision of the solution and its underlying technical components and are not as oriented as the sales staff toward selling the solution. They usually paint a more realistic picture of the solution and the vendor’s ability to support it. They can also confirm technical infrastructure details about operating the solution. • •
The technical review typically has several components:
Page 11 of 18
Tender Evaluation Process Notes
Step
Description • Demonstration of technical facilities and elements • Review of remaining questions on technical and general criteria responses • Support and operational model comparison • Performance investigation • Review of solution architecture • Conceptual technical infrastructure design • Review of services delivery and quality • Software, support and Operations costing • Review the software release and patching strategy the vendor employs The importance and, hence, the amount of effort spent preparing for and conducting the various components of the technical review vary based on several factors.
Visit Reference Sites Perform Financial Analysis Select Finalist
Complete Evaluation
During the technical review, each team member updates the relevant criteria scores on a copy of the current Technical and General Criteria Lists. The team members take notes justifying the scores and identify other risk areas, strengths, and weaknesses. At the end of the technical review, the team meets to reach consensus on the technical review findings. Decide whether to visit vendor reference sites. Plan the approach, visit the sites, then consolidate the findings. This activity is a critical juncture for confirming the vendor’s financial estimates. Help make sure the vendor’s cost estimates are as correct as possible by confirming the preliminary cost estimates. The team is now prepared to select a finalist. Review the decision to confirm that the reporting approach should still be pursued. Determine contract negotiation timing and approach. Conduct a workshop or a work session to select the solution finalist. In general, use a workshop for a large solution such as that being requested. Complete the evaluation process. Complete the project records. Finalise the evaluation report. Notify the losing vendors. Prepare for any vendor loss meetings. Conduct any vendor loss meetings Assess satisfaction with the evaluation project. Summarise the project results and the lessons learned.
Page 12 of 18
Tender Evaluation Process Notes
Step
Description Review and recognise the team performance. Close out the project records
This is a high-level view of the evaluation process that will be used. The following additional collateral will be created and maintained during the evaluation and selection process • •
•
• •
•
•
Evaluation Matrix. Create an Evaluation Matrix to record evaluations of vendor responses to an RFS or RFP. Solution Decision Model. Create a Solution Decision Model reflecting how you measure and evaluate the different evaluation components, such as RFP responses, demonstrations, and site visits. Vendor Demonstration Support Materials. Create a Script Case Plan, Process Scenario Worksheet, Script Case Summary Score sheets, and EBP Summary Score sheets to plan vendor demonstrations, record observations, and summarise team evaluations of the demonstrations. Solution Comparison Matrix. Update the Solution Comparison Matrix to keep track of significant vendor and solution evaluation factors. Vendor Reference Questionnaire. Create a Vendor Reference Questionnaire to record questions and answers for checking a vendor’s references. Negotiated Vendor Agreements. Support the creation of Negotiated Vendor Agreements. Create the agreements to bind the vendor and company in a clearly defined and enforceable way regarding a solution the vendor will supply. Strengths and Weaknesses List. Create Strengths and Weaknesses Lists to record individual and team assessments of vendor and solution strengths and weaknesses.
2.3 Evaluation Team The evaluation team may be structured as follows:
Page 13 of 18
Tender Evaluation Process Notes
The team will be lead by a team manager. The team may be divided into a number of groups: •
Functional - this group will be responsible for evaluating the functional aspects of the responses
•
Technical - this group will be responsible for evaluating the technical aspects of the responses
•
General - this group will be responsible for evaluating general factors such as the responding companies and the finances
No members of the evaluation team will be connected with any of the respondents. The team manager may be assisted by a number of specialists in particular aspects of the development and operation of a reporting system.
The responsibilities of the Evaluation Team Manager will be: •
Managing the evaluation process
Page 14 of 18
Tender Evaluation Process Notes
•
Consulting with specialists as required
•
Documenting the evaluation process
•
Assign weights to the evaluation factors in conjunction with the Group Leaders
•
Conducting meetings of the evaluation group leaders and of the whole team
•
Preparing the final report on the evaluation
•
Reviewing and understanding responses from respondents
•
Acting as the single point of contact between respondents and other team members
•
Allocating and scheduling work for the evaluation groups
•
Reviewing the outputs from the evaluation groups and seeking clarification
The responsibilities of the evaluation group leaders will be: •
Conducting meetings of the evaluation group
•
Preparing evaluation task lists and other material to assist evaluation
•
Reviewing and understanding responses from respondents
•
Co-ordinating any questions to and responses from respondents from group members
•
Documenting the work of the group
•
Reporting to the team manager as required
•
Allocating and scheduling work between the members of the group
•
Reviewing and discussing the work of the group as it takes place
•
Summarising the results of the evaluation group
The responsibilities of the members of the individual group members will be: •
Reviewing and understanding responses from respondents
•
Understanding the evaluation process
•
Documenting their work
•
Performing their assigned work in a timely and efficient manner
Page 15 of 18
Tender Evaluation Process Notes
•
Seeking clarification where required
•
Responding to queries and requests for updates from the group leader and team manager
2.4 Evaluation Criteria The evaluation criteria will be taken from the RFP. They will be grouped. Each criterion will be assigned a weight to reflect its relative importance in the required solution. The following table contains an illustrative list of evaluation criteria: Category
Factor
Response Document Vendor
Response document
Methodology
System Design
Implementation Software Tools
Security
Hardware
System Operation
Respondent stability Respondent references Respondent knowledge and experience Respondent personnel Respondent certifications Project delivery ability Methodology Risk Management Standards System design System performance and throughput System usability Growth and expansion Timescale Team Development language(s) Database management system Other software components Connection to End User Company user directory Flexibilility Server infrastructure Client infrastructure Network infrastructure User reporting
Weight % Total Category Weight % 3.00% 3.00% 4.00% 4.00% 4.00% 3.00% 1.00% 4.00% 2.00% 2.00% 1.00% 3.00% 2.00% 3.00% 2.00% 4.00% 3.00% 2.00% 2.00% 2.00% 2.00% 2.00% 1.50% 1.50% 1.00% 4.00%
20.00%
5.00%
10.00% 7.00%
6.00%
4.00%
4.00%
Page 16 of 18
Tender Evaluation Process Notes
Financial
Future
User analysis System administration and management Office interface Mail interface System development Software licenses Support and maintenance Hardware costs Upgrade costs Overall costs Growth in data volumes Growth in users Additional reports
TOTAL
4.00% 3.00% 2.00% 2.00% 3.00% 3.00% 2.00% 1.00% 1.00% 10.00% 2.00% 2.00% 2.00% 100.00%
15.00%
20.00%
6.00% 100.00%
2.5 Evaluation and Scoring Procedure The evaluation process involves the following steps: •
Weighting or classifying criteria based on importance to the overall project
•
Scoring vendors ability to match criteria
•
Multiplying score by the weight to derive final notional score Importa nce Weightin g (1-5)
Total Possibl e Score
Responde .. Responde nt 1 nt M Score Score (Unweigh (Unweigh ted) ted)
Respond .. Respond ent 1 ent M Score Score (Weighte (Weighte d) d)
Category 1 Factor 1 ... Factor N ... Category P Factor 1 ... Factor N TOTAL Deriving a scoring for the cost of the response could involve some formal comparison of all the costs.
Page 17 of 18
Tender Evaluation Process Notes
Respondents could be either assigned a simple score based on their rating in a table of scores. However, this does not take into account large variations in costs. The cheapest response could be assigned the maximum number of points. The differences between each response and the lowest response could be calculated. The amount of the total difference each respondent is responsible for can be used to calculate the number of points to be assigned to the vendor. In the following example, the cheapest response is 75 and the largest in 100. The total of all the differences from the cheapest is 110. So a respondent who proposed a cost of 100 is responsible for nearly 24% of the total variation. The respondent's score is the maximum score reduced by this percentage. Cost 75 80 85 85 90 95 100 100
Difference From Lowest 0 5 10 10 15 20 25 25 110
Percentage of Total Variation 0.00% 4.55% 9.09% 9.09% 13.64% 18.18% 22.73% 22.73%
Points Assigned 100 95 91 91 86 82 77 77
2.6 Communication With Respondents During the evaluation process, it may be necessary to communicate with respondents. All communication will be in writing, by letter, FAX or e-mail. Responses from respondents must be in writing. These responses will be made available to members of the Evaluation Team. The communications will take the form of a list of issues requiring clarification. The communications will not disclose any information on the perceived strength or weakness of the respondent’s proposal. All respondent communications will be via a single member of the Evaluation Team.
Page 18 of 18
May 2009
Tender Evaluation Process Notes
Contents 1. INTRODUCTION, PURPOSE AND SCOPE........................................................... 3 2. TENDER RESPONSE EVALUATION AND SELECTION.......................................... 4 2.1 OVERVIEW ...................................................................................................... 4 2.2 EVALUATION PROCESS ........................................................................................ 4 2.3 EVALUATION TEAM .......................................................................................... 13 2.4 EVALUATION CRITERIA ...................................................................................... 16 2.5 EVALUATION AND SCORING PROCEDURE ................................................................. 17 2.6 COMMUNICATION WITH RESPONDENTS .................................................................. 18
Page 2 of 18
Tender Evaluation Process Notes
1. Introduction, Purpose and Scope This document contains some notes on a possible approach to tender evaluation.
Page 3 of 18
Tender Evaluation Process Notes
2. Tender Response Evaluation and Selection 2.1 Overview The objectives of the evaluation process are twofold: • •
To select the most appropriate proposal to meet requirements To create an evaluation audit trail and set of supporting documentation that will support the evaluation process and final decision.
The output from the process will be a Vendor and Solution Evaluation and Selection (VSES) report. The document describes the evaluation criteria that were applied to responses to the tender. Any evaluation process is liable to subjectivity. The evaluation process described below can be subjective in: • •
The selection of the evaluation factors and the relative weights assigned The scores assigned to each vendor
The purposes of a formal evaluation process are: • • •
To eliminate subjectivity as much as possible from the evaluation process To provide transparency in the evaluation process To ensure that all proposals are evaluated in the same way
2.2 Evaluation Process The generalised solution evaluation process can be customised to meet the exact needs. The VSES report will have the following general structure: Part A – Overview 1. Executive Summary Part B – Short List Selection 2. List of Vendors Contacted 3. Product Decision Model 4. RFP Findings 5. Summary of Short List Recommendation Part C – Finalist Selection 6. Reference Checks and Site Visits 7. Functional Demonstration Findings
Page 4 of 18
Tender Evaluation Process Notes
8. Technical Review Findings 9. Solution Demonstration Planning Lab Findings 10. Summary of Finalist Recommendation Part D – Appendices Containing Supporting Material (optional) Schematically, the high-level view of the evaluation process that is proposed to be used during the evaluation and selection of a reporting solution is:
Page 5 of 18
Tender Evaluation Process Notes
In summary the evaluation process consists of the following elements:
Page 6 of 18
Tender Evaluation Process Notes
•
Evaluation Team - the evaluation team consists of individuals with appropriate skills to be able to evaluate responses received from vendors
•
Evaluation Process - the evaluation process is the set of steps to be followed by the evaluation team to conduct an evaluation of the responses
•
Evaluation Criteria - this is a set of criteria by which the responses will be assessed
•
Evaluation Score - the score is an objective assessment of the merits of a response based on the application of the evaluation criteria
•
Evaluation Report - this is a document describing the evaluation process and its results
In more detail, the steps in the process are: Step Start VSES Process
Description The amount of effort involved in starting the Vendor and Solution Evaluation and Selection (VSES) step depends on:
How recently you made any VSES scope and direction decisions Whether the team has already been selected How familiar each team member is with the functional area within which the application is to be deployed, and the systems currently in use in BoI to support Monthly Financial Performance Reporting • How experienced the team is with vendor and solution evaluation and selection. Confirm Scope Review previous decisions made about the scope of the project. and Approach Discuss any scope changes and adjust the work plan to reflect approved changes. • • •
Consider the urgency of need, and determine how it impacts the selection strategy. Determine the timing and general strategy for contract negotiation.
Establish and Train the Evaluation Team
Determine what parties to involve in defining procurement procedures, handling legal questions, and negotiating the contract so that the right people can be involved at appropriate points. Assemble and organise a solution evaluation and selection team. Observe the following guidelines: • •
Include technical personnel who understand the systems management and support processes that are being implemented Include business personnel who understand the operational objectives, business processes and needs of Executive
Page 7 of 18
Tender Evaluation Process Notes
Step
Description stakeholders and sponsors of the project. • Include subject matter experts in Best Business practice in Business Performance reporting as needed. • Include experts who understand the market context within which the vendors operate so as to ensure an understanding of: o The capability and capacity of service providers on the local market where resources will be required o The extent to which the vendors solutions will need to be customised to meet the requirements identified in the RFP o The market for services based activities (consulting/training/application development etc) • Provide for a workshop facilitator with a solid background in the subject area if you plan to use facilitated workshops as part of the evaluation and selection process. • Keep the core group small but representative of the functional areas the product most affects. • Include as support members anyone involved in defining procurement procedures, handling legal questions, and negotiating the contract. • Include management personnel responsible for approving the selection Once you have established the evaluation and selection team, designate team members to be the primary and secondary vendor contacts. The secondary vendor contact works with the vendor when the primary contact is unavailable, such as during vacation or illness. These individuals represent the interests of the client and the systems integrator, answer questions about the solution evaluation and selection process, strategise with the vendor, arrange vendor contact with other members of the team, and are empowered to make informal commitments. Next, designate a team librarian to administer the vendor scheduling, mail all vendor requests, record all vendor contact, obtain and distribute vendor materials, and maintain a filing system for vendor materials. Walk through the evaluation and selection activities with team members to familiarise them with the general process. If the team is relatively inexperienced with solution evaluation and selection, consider using a team-building exercise to preview the process. Scale down and simulate evaluation and selection activities. Save detailed discussions for just-in-time training sessions. Discuss the objectives and principles of the evaluation and selection process with the team to ensure that everyone has a consistent understanding.
Page 8 of 18
Tender Evaluation Process Notes
Step
Description Compile Compiling criteria when selecting a solution involves: Function • Developing or gathering criteria appropriate to the evaluation al method Criteria • Avoiding excess precision • Keeping the focus on business priorities • Designing a score sheet that facilitates getting and analysing vendor responses • Incorporating the criteria into the score sheet. These criteria will be taken from the tender. In this activity, define the functional criteria - the requirements as seen from the business user’s point of view. These requirements represent the business processes the solution needs to accommodate. For more information on developing application selection criteria and designing a score sheet, see the Product Evaluation technique. Design functional criteria and group them as follows: • • • •
Compile Technical Criteria
Business processes Data External system interfaces Business performance measurement.
Assemble the functional criteria into one document, the Functional Criteria List. Define technical criteria to represent the solution design and technical infrastructure aspects of the solution. The technical requirements concern the solutions operation and infrastructure. To enhance readability and minimise oversights, develop categories of technical criteria in the areas of: • • • •
Systems management and support process Operational interface Technical infrastructure Volume and technical performance.
Assemble the technical criteria into one document, the Technical Criteria List. These criteria will be taken from the tender. Compile General Criteria
The buyer of a reporting solution is buying more than a service. In most cases he or she is also developing a long-term relationship with the vendor, expecting some, if not all, of the following:
Page 9 of 18
Tender Evaluation Process Notes
Step
Description • Initial transition, configuration and implementation services • Custom services • Consulting services • Future changes • Customer support services, such as a hot line and a user group. General criteria include “vendor” and miscellaneous criteria. These criteria are factors that influence the solution purchase decision but are not functional or technical in nature. General criteria require that you answer them through an iterative process, which requires different tactics: Concentrate on criteria that would eliminate vendors from the short list. Consult with legal and purchasing personnel to extract general criteria that can quickly narrow the solution selection. Add criteria to ensure the team understands the vendor’s decisionmaking and delivery processes, schedules, and organisation before making a final solution selection and developing an implementation plan. With functional and technical criteria, you are usually looking for a specific response. With general criteria questions, you are looking for potential risks and information needed for deployment. Collect responses to these criteria by asking the vendor directly, by asking references who use the software, and by doing your own research. Develop categories to enhance readability and minimise oversights. Assemble the categories of general criteria into one document, the General Criteria List.
Determine Scoring Approach
Conduct RFP Response Evaluation
These criteria will be taken from the tender as well as other more specific requirements that extend the material in the tender. To narrow solutions options to a reasonable number, design an overall package selection framework to communicate and summarise findings. Use weighting to reflect the relative importance of criteria; use scoring to indicate how well the vendor’s solution meets the criteria. Weighting and scoring are often represented numerically, giving the illusion of objectivity. Keep in mind that these numbers are really just a quick means to communicate subjective judgments. When making the package decision, check that the bottom-line numbers are consistent with your overall judgment. The objective of this activity is to obtain and evaluate comprehensive vendor proposals and presentations that explain how their products meet the specific criteria, and at what estimated cost. The RFP has information instructing vendors to respond formally, in writing, and possibly in a presentation. Vendors are
Page 10 of 18
Tender Evaluation Process Notes
Step
Description compared uniformly and strictly as to how well they meet the criteria, comply with the instructions, and present their solution.
The responses to the RFP will be evaluated with respect to the criteria developed above. Develop Script During the solution meetings with vendors, the team needs to Cases compare solutions fairly and evenly, even though different solutions are being proposed and discussed. This is hard for the team to do if each vendor is allowed to determine the meeting’s sequence, contents, and emphasis. By prescribing that all vendors follow a planned sequence of requirements, or scripts, you: Ensure consistency of content coverage across vendors Ensure the vendor is presenting those features most important to you, not just those the vendor puts forward as strong points • Can more easily spot features and functions that do not quite match expectations • Can more easily distinguish how the features and functions differ among solutions. Select Vendor Conduct a workshop or a work session to obtain team consensus on whether or not to proceed with solution evaluation and to reduce Short List the list of vendors under consideration to three or fewer. In general, use a workshop for large solutions or solutions with a major impact such as the service being requested. Check Vendor Check vendor references as another way to confirm that the References vendors’ solution and services could meet the requirements. Develop a questionnaire to record reference responses. Include questions covering relevant functional, technical, and general criteria. Involve members of the team in reference checking. This distributes the work, counteracts individual bias, and provides experience useful in subsequent VSES activities. Have the short list vendors conduct functional and operation walkConduct Functional and throughs using the script cases provided, so you can: Operational • Confirm that the solution, can process operate to your Review Meetings satisfaction • Determine, based on your own observations, which solution provides the best answer to needs Conduct a technical review of the solution with the vendor’s Perform technical staff. The vendor’s technical staff members are usually Technical Review involved in the provision of the solution and its underlying technical components and are not as oriented as the sales staff toward selling the solution. They usually paint a more realistic picture of the solution and the vendor’s ability to support it. They can also confirm technical infrastructure details about operating the solution. • •
The technical review typically has several components:
Page 11 of 18
Tender Evaluation Process Notes
Step
Description • Demonstration of technical facilities and elements • Review of remaining questions on technical and general criteria responses • Support and operational model comparison • Performance investigation • Review of solution architecture • Conceptual technical infrastructure design • Review of services delivery and quality • Software, support and Operations costing • Review the software release and patching strategy the vendor employs The importance and, hence, the amount of effort spent preparing for and conducting the various components of the technical review vary based on several factors.
Visit Reference Sites Perform Financial Analysis Select Finalist
Complete Evaluation
During the technical review, each team member updates the relevant criteria scores on a copy of the current Technical and General Criteria Lists. The team members take notes justifying the scores and identify other risk areas, strengths, and weaknesses. At the end of the technical review, the team meets to reach consensus on the technical review findings. Decide whether to visit vendor reference sites. Plan the approach, visit the sites, then consolidate the findings. This activity is a critical juncture for confirming the vendor’s financial estimates. Help make sure the vendor’s cost estimates are as correct as possible by confirming the preliminary cost estimates. The team is now prepared to select a finalist. Review the decision to confirm that the reporting approach should still be pursued. Determine contract negotiation timing and approach. Conduct a workshop or a work session to select the solution finalist. In general, use a workshop for a large solution such as that being requested. Complete the evaluation process. Complete the project records. Finalise the evaluation report. Notify the losing vendors. Prepare for any vendor loss meetings. Conduct any vendor loss meetings Assess satisfaction with the evaluation project. Summarise the project results and the lessons learned.
Page 12 of 18
Tender Evaluation Process Notes
Step
Description Review and recognise the team performance. Close out the project records
This is a high-level view of the evaluation process that will be used. The following additional collateral will be created and maintained during the evaluation and selection process • •
•
• •
•
•
Evaluation Matrix. Create an Evaluation Matrix to record evaluations of vendor responses to an RFS or RFP. Solution Decision Model. Create a Solution Decision Model reflecting how you measure and evaluate the different evaluation components, such as RFP responses, demonstrations, and site visits. Vendor Demonstration Support Materials. Create a Script Case Plan, Process Scenario Worksheet, Script Case Summary Score sheets, and EBP Summary Score sheets to plan vendor demonstrations, record observations, and summarise team evaluations of the demonstrations. Solution Comparison Matrix. Update the Solution Comparison Matrix to keep track of significant vendor and solution evaluation factors. Vendor Reference Questionnaire. Create a Vendor Reference Questionnaire to record questions and answers for checking a vendor’s references. Negotiated Vendor Agreements. Support the creation of Negotiated Vendor Agreements. Create the agreements to bind the vendor and company in a clearly defined and enforceable way regarding a solution the vendor will supply. Strengths and Weaknesses List. Create Strengths and Weaknesses Lists to record individual and team assessments of vendor and solution strengths and weaknesses.
2.3 Evaluation Team The evaluation team may be structured as follows:
Page 13 of 18
Tender Evaluation Process Notes
The team will be lead by a team manager. The team may be divided into a number of groups: •
Functional - this group will be responsible for evaluating the functional aspects of the responses
•
Technical - this group will be responsible for evaluating the technical aspects of the responses
•
General - this group will be responsible for evaluating general factors such as the responding companies and the finances
No members of the evaluation team will be connected with any of the respondents. The team manager may be assisted by a number of specialists in particular aspects of the development and operation of a reporting system.
The responsibilities of the Evaluation Team Manager will be: •
Managing the evaluation process
Page 14 of 18
Tender Evaluation Process Notes
•
Consulting with specialists as required
•
Documenting the evaluation process
•
Assign weights to the evaluation factors in conjunction with the Group Leaders
•
Conducting meetings of the evaluation group leaders and of the whole team
•
Preparing the final report on the evaluation
•
Reviewing and understanding responses from respondents
•
Acting as the single point of contact between respondents and other team members
•
Allocating and scheduling work for the evaluation groups
•
Reviewing the outputs from the evaluation groups and seeking clarification
The responsibilities of the evaluation group leaders will be: •
Conducting meetings of the evaluation group
•
Preparing evaluation task lists and other material to assist evaluation
•
Reviewing and understanding responses from respondents
•
Co-ordinating any questions to and responses from respondents from group members
•
Documenting the work of the group
•
Reporting to the team manager as required
•
Allocating and scheduling work between the members of the group
•
Reviewing and discussing the work of the group as it takes place
•
Summarising the results of the evaluation group
The responsibilities of the members of the individual group members will be: •
Reviewing and understanding responses from respondents
•
Understanding the evaluation process
•
Documenting their work
•
Performing their assigned work in a timely and efficient manner
Page 15 of 18
Tender Evaluation Process Notes
•
Seeking clarification where required
•
Responding to queries and requests for updates from the group leader and team manager
2.4 Evaluation Criteria The evaluation criteria will be taken from the RFP. They will be grouped. Each criterion will be assigned a weight to reflect its relative importance in the required solution. The following table contains an illustrative list of evaluation criteria: Category
Factor
Response Document Vendor
Response document
Methodology
System Design
Implementation Software Tools
Security
Hardware
System Operation
Respondent stability Respondent references Respondent knowledge and experience Respondent personnel Respondent certifications Project delivery ability Methodology Risk Management Standards System design System performance and throughput System usability Growth and expansion Timescale Team Development language(s) Database management system Other software components Connection to End User Company user directory Flexibilility Server infrastructure Client infrastructure Network infrastructure User reporting
Weight % Total Category Weight % 3.00% 3.00% 4.00% 4.00% 4.00% 3.00% 1.00% 4.00% 2.00% 2.00% 1.00% 3.00% 2.00% 3.00% 2.00% 4.00% 3.00% 2.00% 2.00% 2.00% 2.00% 2.00% 1.50% 1.50% 1.00% 4.00%
20.00%
5.00%
10.00% 7.00%
6.00%
4.00%
4.00%
Page 16 of 18
Tender Evaluation Process Notes
Financial
Future
User analysis System administration and management Office interface Mail interface System development Software licenses Support and maintenance Hardware costs Upgrade costs Overall costs Growth in data volumes Growth in users Additional reports
TOTAL
4.00% 3.00% 2.00% 2.00% 3.00% 3.00% 2.00% 1.00% 1.00% 10.00% 2.00% 2.00% 2.00% 100.00%
15.00%
20.00%
6.00% 100.00%
2.5 Evaluation and Scoring Procedure The evaluation process involves the following steps: •
Weighting or classifying criteria based on importance to the overall project
•
Scoring vendors ability to match criteria
•
Multiplying score by the weight to derive final notional score Importa nce Weightin g (1-5)
Total Possibl e Score
Responde .. Responde nt 1 nt M Score Score (Unweigh (Unweigh ted) ted)
Respond .. Respond ent 1 ent M Score Score (Weighte (Weighte d) d)
Category 1 Factor 1 ... Factor N ... Category P Factor 1 ... Factor N TOTAL Deriving a scoring for the cost of the response could involve some formal comparison of all the costs.
Page 17 of 18
Tender Evaluation Process Notes
Respondents could be either assigned a simple score based on their rating in a table of scores. However, this does not take into account large variations in costs. The cheapest response could be assigned the maximum number of points. The differences between each response and the lowest response could be calculated. The amount of the total difference each respondent is responsible for can be used to calculate the number of points to be assigned to the vendor. In the following example, the cheapest response is 75 and the largest in 100. The total of all the differences from the cheapest is 110. So a respondent who proposed a cost of 100 is responsible for nearly 24% of the total variation. The respondent's score is the maximum score reduced by this percentage. Cost 75 80 85 85 90 95 100 100
Difference From Lowest 0 5 10 10 15 20 25 25 110
Percentage of Total Variation 0.00% 4.55% 9.09% 9.09% 13.64% 18.18% 22.73% 22.73%
Points Assigned 100 95 91 91 86 82 77 77
2.6 Communication With Respondents During the evaluation process, it may be necessary to communicate with respondents. All communication will be in writing, by letter, FAX or e-mail. Responses from respondents must be in writing. These responses will be made available to members of the Evaluation Team. The communications will take the form of a list of issues requiring clarification. The communications will not disclose any information on the perceived strength or weakness of the respondent’s proposal. All respondent communications will be via a single member of the Evaluation Team.
Page 18 of 18
Related Documents
Tender Evaluation Process Notes
June 2020 3
Tender Evaluation Form
December 2019 7
Process Speech Evaluation
October 2019 13
Training Evaluation Process
November 2019 21
Liquefaction Process Evaluation
November 2019 11
Design Process And Evaluation
November 2019 20More Documents from ""
Business Cases And Benefits Management
June 2020 8
