Predicting Failure Of Ictd Projects In India

Predicting the failure of ICTD projects: the ROI approach

Abstract—To understand the high failure rates of ICTD projects, and the factors behind the rare successes, this paper examines eleven such projects, based in India, in which the author has had a direct role. These projects are examined on the following simple questions: 1) Is the goal of the project well defined and measurable? 2) Can a precise ROI (return on investment) be calculated? 3) Is the implementing agency driven primarily by the above ROI considerations? Defining measurable goals and attaching a dollar value to these goals are major barriers to the practical application of the above approach. However, even with qualitative data concrete conclusions can be arrived at: An application of the above questions to the eleven projects clearly shows that if the answer to even one is a clear NO, then the project ends in failure. We recommend that for constructive use of this approach implementors of ICTD projects and researchers in the area rope in economists and sociologists in arriving at measurable parameters for every ICTD project and make cost benefit analysis a mandatory exercise. Further, a study of many of the projects presented in this paper can provide valuable lessons for future ICTD projects.

I. I NTRODUCTION A study of the ICTD field today reveals a dark landscape brightened locally at numerous places by firebrands of pilot successes, occasional steady flames from tall towers, and littered by the dying embers of attempted large scale projects. Undaunted, more and more crusaders are optimistically working on newer technology lights to bring in the dawn of development to this bleak landscape. In this paper we attempt to understand the high failure rates of ICTD projects, and the factors behind the rare successes. The literature on ICTD in the Indian context is strangely skewed. Very few papers present comprehensive studies of successfully completed projects, or unsuccessfully abandoned projects. Such a study is critical in identifying the parameters of success or failure and in using the learning to strengthen future projects. Most papers about Indian ICTD projects fall into three broad categories. First and most prominent are those that use the standard format: Use a development related problem, a specific technology solution, a pilot implementation, qualitative surveys, anecdotal evidence and some human interest story, to present the potential for the use of the specific technology for development. [1] and [2] are illustrative examples. The second category of papers starts with some area of development, like health care or agricultural information, shifts the attention to specific technology issues and after describing the technology innovation, concludes with optimistic projections about the impact of widespread application of the presented technology. Examples are [3] and [4]. The third category of papers is most relevant to the present discussion. The papers in this category attempt to study ICTD projects with the aim of learning from the successes, to enable replication, and from failures, in order not to repeat them. Kenniston [5] writing specifically about India, laments “Unfortunately, such hopes are built almost entirely on an empirical vacuum. Almost nothing is known about factors that make for effectiveness or ineffectiveness of grassroots ICT projects in developing nations”. A systematic study by Sreekumar [6] of the much lauded Gyandoot project in the Dhar district of Madhya Pradesh concludes with the sobering observation:

“The idea that ICT is inherently a liberating technology and hence e-governance is a new way of transcending inept and inefficient bureaucratic systems which empowers end users appears to be completely inaccurate in the rural societal setting.” [6] Pal [7] studies the Akshaya telecenter project with the focus on the effectiveness of the telecenters for e-literacy and concludes (emphasis added) ”Finally, if it is agreed that e-literacy qualifies as a public good, the data from Akshaya presents strong evidence to uncouple it from telecenters.” The research by Heeks and his group targets e-governance projects in the developing world in general [8]. In particular, [9] cites references to show that most e-governance projects around many developing countries have ‘failed’ and then presents theoretical frameworks to help future projects apply the learnings from the failures. The major shortcoming of this work is the lack of focus on measurable economic benefits of development projects. This is especially glaring since there is clear recognition that failure of such projects incurs more than the direct financial costs. For example, six categories of costs incurred by such projects are listed: direct and indirect financial costs, opportunity costs, beneficiary costs, future costs, and political costs. However, there is no corresponding detail on the benefit side of the projects. The failure or success of such projects is determined based on fuzzy goals (like, taking government to the citizens, empowering the citizens, minimising bureaucracy, etc) and by surveying participants in the projects and using their subjective conclusions about whether the stated goals of the project have been met. Vasudevan [10] studies the impact of the STAR project for land record computerisation in Tamil Nadu. Benefits to the stake holders are evaluated using detailed surveys and personal interviews. These benefits include reduced time for transactions, increased reliability and efficiency of services, and digitisation of transaction records. The costs of the project are not analysed in detail except to note the important fact that since the users of the system pay an additional charge per transaction, the project is self sustaining without the need for diverting any government resources. The work by Patra [11] analyses computers in schools from a cost benefit perspective. Various deployment models (single ownership, single user per shared device and multiple user per shared device) of computers in schools to cover all the school children in India are considered and costs computed for each model. However, the focus is entirely on the cost of the devices and their replacement once in five years. The benefits of computers in schools, accepted by the authors as a very controversial and unfinished debating agenda, are studied using a simplistic word recognition application and conclusions are drawn based on qualitative interviews. However, this is among the few papers that even consider economic analysis of ICTD projects. Thus there is a clear paucity of work in the ICTD literature that focuses on cost benefit analysis of ICT projects. A model for such work, worth emulating, is the one by Jensen [12], who systematically studied over a five-year period the economic impact of the use of

mobile phones by the fishermen of Kerala, and concluded that there is clear economic benefits directly attributable to the use of mobile phones. However, the technology domain in this work is that of mobile phones acquired by individuals rather than a traditional ICTD project, the focus of the current paper. Jensen summarises this aspect very nicely in the concluding section of his excellent paper: “Also unlike most development projects, the service is selfsustaining; mobile phone companies provide the service because it is profitable to do so, and fishermen are willing to pay for mobile phones because of the increased profits they receive. This point is also relevant for reconciling our results with anecdotal evidence that government or NGO projects setting up internet kiosks or other information services for farmers in other developing countries often do not meet similar success.”[12] In contrast to research reported in ICTD literature, most other developmental projects run by global organisations like the World Health organisation or the UN are obsessively focused on cost benefit analysis. A pre-requisite to this is impact assessment (for example, health impact assessment [13]) that plays a key role at every stage of a development project. A striking example is the work by Horton [14] where there is painstaking level of detail of the cost benefit analysis, cost effectiveness and target goals. For instance, “Additional investments in iron fortification and salt iodization would cost $286 million, with only small DALY gains but large returns in dollar terms. Although the resource cost is $286 million, the cost to governments could be considerably smaller, if consumers absorbed the modest per-person cost of the fortificant. Deworming preschoolers would cost $26.5 million, yielding $159 million in benefits.” [14] Definitions of terms like DALY (disability adjusted life year) used widely in global health related literature and the fact that a dollar value can be attached to DALY gains are lessons that ICTD community can emulate. Of course there are ongoing debates about the validity and appropriateness of measures like DALY, but the key factor of interest to us is that detailed cost benefit analysis is a critical component of most Development projects around the world. Thus ICTD projects currently appear to be the exceptions to the rule that cost benefit analysis is central to Development projects. This is obviously an indication of the immaturity of the field as much as that of the technologies involved. Another factor is the lack of coherence in the ICTD literature itself. An excellent critique of ICTD literature can be found in [15]. ICTD literature at present lacks depth in any of the component disciplines because it is a newly created inter-disciplinary field. The difference between the quality of analysis of ICTD projects found in the ICTD literature and those in well established literature is starkly brought out by two papers. Jensen [12], is published in the Economic quarterly. Reuben [16], published in ITID Journal, studies the same scenario (impact of mobile phones on the same fishing community in Kerala), comes to similar conclusions, but is based on very fuzzy data, subjective interviews and minimal analysis. It is all the more surprising since Jensen thanks Reuben for the latter’s valuable comments on his paper. The objective of the current paper is to underline the need for rigorous cost benefit analysis of ICTD projects. This is attempted using the following plan: present eleven different ICTD projects with which the author was directly inovlved. Study these projects using a simplistic cost benefit analysis. Demonstrate that even with such a simplistic analysis actual projects can be evaluated for their potential

success or failure. Highlight the difficulties involved in defining costs and benefits in development projects using the example projects. Finally conclude with a plea that implementors of ICTD projects and researchers in the area rope in economists and sociologists in arriving at measurable parameters for every ICTD project and make cost benefit analysis a mandatory exercise. It is also suggested that systematic study of many of the unstudied projects presented in this paper can provide valuable lessons for future ICTD projects. II.

COST- BENEFIT QUESTIONS

We propose a set of three simple questions as the basis for studying ICTD projects. 1) Is the goal of the project well defined and measurable? 2) Can a precise ROI (return on investment) be calculated? 3) Is the implementing agency driven primarily by the above ROI considerations? Clearly all these questions require only qualitative answers. And it is also evident that the questions themselves are fuzzy and subject to multiple interpretations. However, as we proceed in the next section to apply these questions to real projects, it will be clear that many projects get implemented without attempting to answer even such fuzzy questions. There are many difficulties in defining specific terms like ’goals’, ’measurable parameters’, and ROI. For instance, ROI is calculable only when we have dollar figures for the investment and dollar figures for the returns. These difficulties will be addressed in the concluding sections of this paper. III. E LEVEN PROJECTS The projects described in this section have some common threads. First, the author has had direct involvement in some way with almost all of the projects. Second, all projects are located in India and third, with a few exceptions, these projects have not been described in the ICTD literature and the only source of reference is usually media report archives on the web. Hence there are no authoritative sources for some of the data and inferences presented and hence these should be investigated further before final conclusions can be drawn. Many of the details in this paper relating to government projects are based on the author’s involvement in some of the bidding processes or based on private communication with individuals who executed some of the projects. Where available, relevant URL’s have been listed as reference. Fortunately, the Right To Information Act (RTI) passed by the government of India, a major facilitator of transparency in Government, provides fairly well defined processes for accessing details of any of the projects from the concerned government officials, and is a valuable tool for researchers. In what follows, we briefly describe each project, apply the above questions and tabulate the answers in Table 1, along with the author’s conclusion about the state of the project. A. The Simputers in Education Project (2002-2003) A grant of Rs 50 lakhs (about US$ 100,000, in 2001 exchange rates) was given by the South Asia Foundation to PicoPeta Simputers to deploy Simputers for primary education of tribal children. This was meant to be an exploration of the potential benefit of the Simputer platform for education and on successful completion of the pilot, funding to scale up the project was to be made available by the same agency. The project was taken up and implemented with missionary zeal since this was the first project of any scale involving the Simputer. Detailed interaction with teachers of two schools in Chattisgarh (which themselves were identified after a long drawn process) over

periods extending to several months, development of suitable software and content creation tools, training of the teachers and the students in the technologies and iterative improvement of these were all carried out over a period of nine months. 100 Simputers were shared and used by over 600 students for a period of four months after the preparatory work. The feedback from the teachers and the students were heartwarming, and so were the personal experiences of the project staff. However, at the end of the pilot project, which was delayed by more than a year due to many reasons, a review was done by the SAF founder to understand the implications of the project. In hindsight, it was not surprising that he did not find the results worth replicating or scaling up. As Table I indicates, the project had no clear measurable goals, no measure of ROI, nor the concurrence of the project funding agency on an agreed set of measurable goals and performance parameters. The project was approved based on excitement of the potential of a new technology and the funding was given based on a one page proposal that simply stated that Simputers will be used to aid education in tribal schools. The fact that education and the measurement of learning outcomes and the economic value of such learning are among the most slippery of activities is not an excuse for not even attempting to discuss end goals of such a large project. So making an entry for this project in the table is very simple: the answers are clear NOs for every question and the project was a clear failure. B. Electricity Bill metering (2002 onwards) Plugging leakages and iImproving efficiencies of government services like electricity generation and distribution, water (both for irrigation and consumption), public transport, road infrastructure, etc., in developing countries will directly impact the resources available for poverty alleviation and employment generation. Hence the use of ICTs for such projects is directly relevant for Development. Till 2001, all electricity meters were read manually by agents physically visiting each and every location of the meters. The current reading will be noted by the agent, the information taken back to the office and after about two to three weeks, a bill for that month’s consumption would be sent either by post or by another agent individually dropping it off in each metered location. In addition to the inherent errors in such manual reading and transcribing, this process resulted in the time gap between the reading of the meter and the actual paying of the bill by the consumer to be on an average six to eight weeks. There were frequent complaints of wrong reading by the consumer, as well as loss of revenue due to collusion between the consumer and the reading agent in consistently under reporting the consumption. This was among many other reasons why the state electricity agencies were perpetually bankrupt and had to be shored up by government subsidy. The Simputer along with a printer was used by a small scale entrepreneur to provide a computerised solution for meter reading, billing and back end database integration. Based on the widespread interest generated by the initial pilot and realisation of the benefits by the distribution agencies, over the next three years, computerisation and use of handheld devices for meter reading and billing have become widespread across Karnataka, Andhra Pradesh and Maharashtra, three progressive states in India and many other states are moving to catch up. The initial focus on technology was quickly replaced by a comprehensive return on investment based approach: the Distribution Company sought competitive bidding for complete billing solution and awarded contracts based on a single number:

the lowest cost bid for generating and issuing one bill to one end customer. For example the project may be awarded to a vendor for servicing hundred thousand households for a period of one year at the rate of Rs. 2.00 (about 4 cents) per bill delivered to the customer’s premises. The actual technologies used by the vendor are of no interest to the Agency. The use of handheld devices combined with back-end computerisation have resulted in the entire billing cycle, from meter reading to payment of the bill by the customer, to be completed within a month, which implies the Distribution Company is able to collect a month’s revenues within the next month. The Table entry for this project is a clear YES for all three questions and is an ongoing successful project. C. Bus ticketing system (2003 onwards) Public buses are the primary mode of transport used by a large segment of rural population as well as the majority of urban population that cannot afford their own private vehicles. In a move similar to that of the Electricity agencies, the Karnataka state road transport corporation embraced technology solutions for improving its operational efficiencies. The result after a few years of experimentation has been phenomenal: more than twenty thousand buses are now equipped with ticket issue machines. The machines themselves are very simple: microcontroller based devices with a 4 line display and a printer attached, with sufficient batteries to last a day of operation. This project is one where every stakeholder can actually show a measurable return on the investment: the passengers get their tickets very quickly, their currency transactions with the conductor are smooth, the conductors have a much reduced workload, much reduced harassment from inspectors who scrutinise the activities of the conductors, and at the end of the day, the settlement of the ticket sales and cash by conductors is reduced from about 45 minutes to a few minutes. The cost savings to the corporation purely in terms of savings on printing and managing multicolored tickets used in the manual mode is about US$200,000 every year. All of this information is anecdotal from one of the solution providers [17]. There has been no systematic study of this project to understand the key parameters and learnings from such an obviously successful project. Even the ’success’ of the project is only being inferred from the fact that the number of buses provided with the technology enabled solution has been rapidly increasing over the past three years and continues to be replicated in many other States, including Tamil Nadu and Andhra Pradesh. So the entry for this project is a clear Yes for all columns. D. NSSO (2004-2005) The National Statistics and Survey Organisation is the principal agency in India entrusted with the collection of socio-economic data. For example, prices of all commodities that determine the weekly inflation rate, household income and expenditure, number of households that have access to credit, and similar data that is used by the Central and State governments to take short and long term measures for the welfare of the population, are all collected by NSSO. All of this data is collected across the country, both in urban and rural areas and across all sample demographics. NSSO is thus a large data gathering, collating and report generating machine of the Central government with a organisational structure that reaches out to the farthest corners of the country. Data collection and collating is still a manual process, with different paper forms used for different purposes. Each of these forms is fairly complex and requires skilled survey takers (called Investigators) to collect the data. These paper forms are then verified

by an Assistant Superindentant and then a Superindent to ensure that the form has been filled properly and completely. Then the paper forms are sent to a Data Processing Center, where data entry operators transcribe and enter the handwritten forms into machines. After verification and validation steps of this process, aggregation and report generation create the reports that reach decision makers. The lag between the collection of data and the dissemination of useful information to decision makers can be anywhere between several months to a few years. A simputer-based solution for speeding up the process was proposed by PicoPeta Simputers Pvt Ltd. With full support and directions from the Minister in charge of NSSO, and close interaction with the NSSO officials of the Karnataka unit of the NSSO, the complete solution that included the conversion of a key set of forms (schedules 0.0, 1.0 and 2.2, called the National sample survey) into an efficient client application of the Simputer was implemented by Picopeta. The completed solution was also field tested by Investigators and checked out by NSSO officials. In a large meeting involving all stakeholders, including Ministry officials, NSSO bureaucrats and field officers, it was agreed that the Simputer-based solution meets the requirements of efficient survey operations and it was suggested that a proposal for a pilot project that involves about 25 Investigators be sent in by PicoPeta for approval and funding by NSSO. A proposal was sent with the total cost of a pilot project of approximately US$80,000. This included the complete software to be owned by NSSO for use on any number of Simputers in the full-scale project, 25 devices, training for field personnel, debugging and field support for the duration of the project. No decision was taken on this proposal despite all efforts. It was learnt that part of the hesitation was due to the fact that a year previously, NSSO had spent close to US$400,000 on buying handheld devices from a multinational vendor, and the devices have remained unused because the purchase of the hardware was not part of a planned complete solution. Hence the reluctance to go after a new solution that involved different handhelds. With reference to the Table, the key missing link is the value that can be attached to efficient collection of data and its rapid aggregation and availability. Given that most Developmental projects depend on NSSO data it is ironic that the author could not find any study on the economic impact of dependence on the NSSO data that is usually out of touch with reality by at least a year. E. TB treatment monitoring (2003-2004) A WHO-funded Tuberculosis Institute project, for health workers to monitor the medication for TB patients spread out in Bangalore rural district was, implemented. The driving agency was the National Tuberculosis Institute (Directorate General of Health Services) with funding from the World health Organisation, as part of the Health InterNetwork (HIN) Project, with the broad aim of bringing together the Government, private sector, non-governmental organizations and resource institutions to ensure equitable access to health information. This project involved design, development and implementation of Simputer based mobile application as part of the HIN Mobile Data Management System. After a year of interactions with the end users a Simputer-based solution was created to help in the better supervision and data collection, for the Senior Treatment Supervisor (STS) and Senior Tuberculosis Laboratory Supervisor. The project goals were to equip the STSs with handheld devices that contain complete patient information. The patients are required to take medication for periods upto a year. The STS’s task is to periodically visit the patients in her area, ensure regular intake of the medications, refill as needed and pass on the information to the TB

institute for overall monitoring of TB treatment in the project area. There was no analysis done to compare the existing method with any technology assisted alternates, in terms of effectiveness, cost, or achievable objectives. Hence there was no question of ROI calculation and hence any commitment to meet the goals. In a process of typical of ICTD projects in India, a public event was held in which the Simputers were handed over to 16 STSs, and speeches were made by dignitaries on how ICT will improve healthcare [18]. After a year of the pilot project, the agency approached the solution provider (Bharat Electronic Limited) to port the solution to the newer version of the Simputer. This task was successfully completed. As far as the author is aware, the technology involved in the solution met the operational requirements of the project. The entry in Table 1 for this project has negative answers to the first three questions, and at the time of writing of this paper, the project appears to have been abandoned. F. India Health Care Project in Andhra Pradesh (2002-2004) Basic health care delivery in rural India is minimal and patchy. Where present, the service is delivered through health workers from the local communities, called Auxiliary Nurse and Midwives (ANMs). These are usually women who work with dedication and are the sole deliverers of health care and information for up to a few thousand individuals. The ANMs take care of prenatal and postnatal care, provide general immunisation, treat and collect data about endemic and epidemic diseases among the population that she is responsible for and perform deliveries as well. These are some of the most hardworking and underpaid government servants in India. In order to decrease the burden of work for the ANMs (including the sheer physical burden of having to carry two or three large registers that contain the data and past medical history of everyone in her care while walking or cycling from village to village), a pilot project, funded by InfoDev of World Bank, and overseen by the Government of Andhra Pradesh was executed by Computer Maintenance Corporation (CMC), a public sector company at that time, since acquired by the Tata Group. The project consisted of about 225 Compaq Ipaq PDAs with client software developed by CMC that managed all the data from all the registers of the ANM, allowed access to specific data as well as the addition of new data. This software was developed with considerable difficulty since the data managed by the ANM was large and complex, the screen size of the PDA was small and the interface capability offered by Windows for PDA’s during 2002-2003 (the period of the project implementation) had several serious limitations. The total hardware cost per device was close to Rs 35,000 (inlcuding the cost of a 32MB memory upgrade). the software development cost must have been significant. 225 ANMs were trained on the use of this software (which was initially done in English since local language support in Windows on PDAs was non existent at that time, with a plan of converting it into Telugu subsequently). A pilot phase of the ANMs using the PDAs was conducted with accompanying media coverage about PDAs bringing healthcare to rural Andhra Pradesh. The author has not been able to locate a report on the results of the pilot project. However, post the pilot project, it appears that the project is no longer on the field. There is brief outline of the software solution in CMC’s website [19] promoting it as a solution offering from CMC. There were no measurable goals, but just the acceptable assumption that digitisation of the data from the multiple notebooks of each ANM and the provision of random access and update capabilities will reduce the workload and improve the delivery of helathcare.

Hence there were no ROI measures specifed. The costs of the scaled up version as well as the returns from the scaled up version were never studied or presented. The implementing agency, namely, the Government of AP was more focused on projecting itself as a leader in IT rather than on specific ROIs from this project. Predictably, the project has been abandoned after the pilot phase with no learnings to show from the huge amounts of donor development money, time and resources spent on the project. G. Tamil Nadu Sugars (2005 onwards) Another project going forward beyond the pilot stage is the Simputer project with Tamil Nadu Sugars. Here Simputers are used a s part of the sugarcane farming information systems tied intimately with the sugar mill that will buy the harvested sugarcane. Each Simputer handles the information of about 50 farmers and their sugarcane fields. Data is collected at every stage of the sugarcane cycle and this data is periodically updated to the data center of the sugar mill where the data from all the Simputers is consolidated. The mills then take decisions about the agricultural support to be given to the farmer as well as to schedule the harvest from different holdings in line with the requirement of the mill. This project was driven by the government agency that runs the sugar mills and with the support of the large farming community that is dependent on the effective functioning of the sugarcane growerharvester-sugar mill relationship. After a year of effort, the pilot project with about 200 Simputers has been declared a success and a public function was held to publcise the initiative with representatives from other Sugar mills in the State invited to study the project and to replicate in their procurement areas. The followup of scaling up the project in the same location as well as to spread similar solution to other sugarcane growing areas in Tamil nadu has been taken up. H. ITC-echoupal This is one of the most successful kiosk projects in India and has been written about extensively. Essentially, Indian Tobacco Corporation, a large corporation set up VSAT-connected computer kiosks in representative farmers’ houses in villages in the procurement areas of the company. Farmers bid with ITC to sell their produce without the involvement of traditional middlemen. It has been a tremendous success as measured by the fact of its continued replication targeting 10 million farmers using 20,000 echoupals in 10 years [20]. The author’s interaction is with ITC during the early stages of the e-choupal project when a Simputer-based solution was considered as an alternate to a full fledged PC. It was clear from the interaction that ITC had very clear goals for what each kiosk is to accomplish, the location of the kiosk, the business model for the farmer hosting the kiosk, and the procurement target to be achieved. It was also clear that they were completely neutral about what technology to be used, so long as the technology met their goals. There was no obsession about ’low cost’. Thus all the entries in Table 1 for this project are clear affirmatives and the project is meeting the predicted success. Incidentally, the Simputer-based solution though it might be less expensive was not pursued since the overall cost of introducing a new system was not worth the per-kiosk savings achieved. The benefit of mobility afforded by the Simputer was weighed against the need for a second operational model and the concomitant training and management overheads. The wise decision of sticking to a single setup for the kiosk and keeping the focus on scaling up the model has enabled this project to succeed. Many lessons can be learnt from this project, key being that commercial agencies can do good while doing well.

I. Bhoomi-Suggi Project (2003-2005) The Bhoomi project, operational for the past few years in Karnataka, is a land record computerisation project that has received several e-governance awards. As an extension of the Bhoomi project, it was proposed that handheld computers be given to Village Accountants, the grass root government functionaries. The idea is to use the backend infrastructure and digitised data created for the Bhoomi project and to extend its utility and reach. Multiple applications are possible. But to start with harvest data collection by the village accountants was chosen as the end task. A complete solution (called the Bhoomi-Suggi, Suggi meaning harvest in Kannada) was created and deployed in 2003. The following are the components of the solution: • • •

• • •

Hand-held devices (Simputers) with the end users (Village Accountants) Client application on the Simputer in Kannada (data collection with smartcard access control for security), Application on Taluk Windows PCs for data upload and download (integration of the data with back-end Microsoft platform database) Training of over 600 Village accountants to date Hardware Field support over an extended geography comprising of several districts of Karnataka Software upgrades, bug fixing and improvements over three years.

The total cost for all of the above paid by the government of Karnataka was US$75,000, inclusive of everything. Obviously very low total cost. And on every technical, operational, training, and support parameters, the solution was validated over three years where a sequence of pilots were done in different districts of Karnataka. This project did not scale up to cover the 9000 village accountants as originally envisioned. The reason is obvious if we evaluate the project as in Table 1. There are clear measurable goals for the project: the productivity of VAs, the expedient access to data, and given the infrastructure of handhelds, connectivity, back-end, a trained enduser group (the village accountants in this case), the potential for additional services that can be provided to the village communities. However, none of these goals were part of the tendering process. Without such measurable benefits being part of the debate, the discussion about this project was sidetracked into which is the right handheld to use, and how to bring down the cost of the device so that 9000 units can be bought cheaply. The project has not scaled up beyond the pilot. J. SKS Microfinance Access to credit has been recognised as one of the major tools for poverty reduction and hence organisations around the world are striving to create means of reaching credit to the rural poor. In India, there is a thriving SHG model in many parts of the country, most modeled on the Grameen bank and some others on local models. SKS Microfinance, a for-profit company is one of the most successful MFIs in the country. It clearly does not fit the ICTD project category. However, what is of interest here is that in the early years of SKS, considerable focus was put on technology: smart cards for members and handheld devices for the agent were tried out to improve the efficiencies of the process. A Simputer-based solution was also considered as a possibility. However, very soon, the focus shifted to the real goals of SKS: offering credit to the rural poor, ensuring that the disbursement and collection process was made most efficient, increasing the number of customers, and ensuring that the overall

business was profitable. So all the entries corresponding to SKS are very strong affirmatives and hence it is no surprise that SKS today is a thriving business with over 2.5 million clients, 1200 branches and more than RS 15 billion in loans. K. Simputers and BPL benefits project in Chhattisgarh (2005 onwards) The definition of poverty line, by The Government of India (GoI), defined in terms of per capita per month earning [21] is Rs 356 for rural and Rs 539 for urban areas, essentially about $0.40 per day. The World Bank’s definition of the poverty line, for under developed countries, like India, is US$ 1/day/person or US $365 per year. It is especially important to keep these numbers in mind when dealing with ICT projects for poverty alleviation. The GoI has numerous schemes to benefit Below Poverty Line (BPL) families. However, there are three major problems with these schemes; cost of delivery of the benefits, given the extremely low values for daily earnings, second the delivery of the benefits to all the persons entitled to the benefits and third, to prevent pilferage of the benefits by persons not entitled to them. There is widespread belief that the proper use of ICT can help overcome these problems. ChIPS (Chhattisgarh Information technology promotion Society) is a government agency charged with all aspects ICT deployment in the State of Chhattisgarh. After extensive evaluation of the Simputer, starting from the time of the education projects with the Simputer in the state discussed above, ChIPS eventually procured in 2004, 900 Simputers for the implementation of a statewide project called e-Gram Suvraj for providing decision making support for village sarpanchs (secretaries) [22]. The objective of this project is to ensure that many benefit schemes from the GoI and the state government reach the intended beneficiaries through the Village Sarpanchs. However, there are no measurable goals, nor a clear statement on RoI for this project. Hence one would expect this project to close down after the ongoing large pilot phase. This is another candidate for a future study. L. Other projects There are many other projects that have been excluded due to space constraints. We present a few pointers to two major domains in which massive investments in ICT are imminent. Bhamasha project: The Government of Rajasthan has taken major steps in bringing the 5 Million BPL families in the state into banking and health insurance services. As a first step, the registration of all the 5 Million families including the details of all the family members and issue of a unique identity number to each family has been completed, itself a massive IT supported initiative. The second major step is to issue smartcards for each of the families containing biometric identification of all the family members, photographs and other information and will be tied to the BPL family database with the unique identity numbers created in the first phase. These cards will be used as basis for providing no-frills bank accounts. The Government of Rajasthan has earmarked Rs 750 crores using which each BPL family’s bank account will be deposited with Rs 1500. The cards will also allow access to health insurance through partner insurance companies. Over 13,000 business correspondents equipped with mobile transaction terminals will provide the banking services at the doorsteps of these families. Such terminals will also be available at health centers to manage access to health insurance services. The RFP for the project [23] is very instructive. This is another project that is a very good candidate for detailed study.

Financial Inclusion: Recognising that access to banking services is critical for Development, the Reserve Bank of India (RBI) has recently mandated that all banks and financial institutions in the country (both public and private) extend their services to reach more than 400 Million people who are currently beyond the reach of such services. In a recent address, the RBI Deputy Governor outlines the potential for ICT to make a difference: “The use of IT solutions for providing banking facilities at doorstep holds the potential for scalability of the FI initiatives. Pilot projects have been initiated using smart cards for opening bank accounts with bio metric identification. Link to mobile or hand held connectivity devices ensure that the transactions are recorded in the banks books on real time basis. Some State Governments are routing social security payments as also payments under the National Rural Employment Guarantee Scheme through such smart cards. The same delivery channel can be used to provide other financial services like low cost remittances and insurance.“ [26] Hence, this is a project domain that holds huge potential for development for the underservered population of India and at the same time offers fertile ground for study of the impact of ICT. IV. D ISCUSSION AND POINTERS TO FUTURE RESEARCH The projects presented contain a wealth of unexplored lessons. Looking at the Table, we see that projects 2, 3, 7 and 10 are outstanding successes: We attribute the success to the fact that there were clear ROI considerations driving all stakeholders in these projects. And many of the goals of the project were measurable and monitor-able. Projects 1, 4, 5, 6 and 9 have been failures even after successful demonstration of the technology during pilots. Again, it is clearly due to the lack of well defined and articulated ROI objectives. Projects 8 and 11 are in successful pilot phase and need to be followed up to see if they scale up. There are two issues here. First, whether there are goals that are measurable. For instance, in the education project, the goals themselves are fuzzy, with no universally accepted measurable goals Second, even when there are measurable goals, as in the case of the NSSO project, where one can use the time for data collection, accuracy of the data, time for aggregation and report generation as measurable goals, there is no accepted way of converting these improved efficiencies into tangible monetary values. For instance, there are no known ways by which statements like “reducing time of NSSO household income survey process from 9 months to 1 month will result in saving of X crores of rupees”. Given this gap, the focus of government agencies is directed only on cost reduction of the technology solution rather than on the potential benefits. A related issue is the decoupling of the Development goals from the project implementation goals. In this context we can add one more to the list of gaps gaps that cause e-governance projects to fail, listed by Heeks [9]. This is the ROI gap as perceived by the initiating government agency and the project implementing Contractor. So many ICTD projects continue to get funded in spite of the widely acknowledge failures because the ROI from the project for the government is not the same as or even linked to the ROI from the project for the Contractor. The project implementor, usually tendered and contracted private sector (in India primarily on the basis of lowest cost) views the project from clear commercial terms: what are the parameters on which my delivery is measured? how can I deliver these at the lowest cost?

How much margin can I make after deducting the costs from the money I receive from the government. By this measure, most ICTD projects are roaring successes for large IT companies that implement them. The right way to proceed is to tie the returns of the contractor to the benefits generated by the project. The bus ticket solution and the electricity bill metering solution are clear examples where such an approach has succeeded. This is the most important conclusion of this paper. A. Some general observations The study of these projects brings out three major negative consequences of the excessive focus on technology: first, the real goals of development are shortchanged and precious resources squandered due to the excess attention paid to glamorous technology, an example of which is the focus on PDA for healthcare project. Second, many technologies that can accelerate development are prematurely discarded because of incorrect understanding of the non-technology causes of failure. An example of this is the Simputer in education as well as the Bhoomi-Suggi project were it is easier to conclude that the Simputer is the cause for the project failure and for other technologists to then spend energies on building a better device. Third, many projects that have successfully used ICT for development have not received the kind of attention they deserve since they have used unremarkable technologies. Both the utility billing projects fall in this category. Many of the ICTD projects that have garnered very large global media and literature attention are, as per the ROI test proposed in this paper, unlikely to succeed. A short list of some of the key projects that fall under this category follows. •

•

Technology for school education: This is one of the major areas were billions of dollars are spent worldwide without clear goals. Even in the developed world there is no consensus on the benefits of ICT in school education [24]. Projects that start with a candidate device and proceed to address education are clear instances of the technology tail wagging the education dog. The $100 laptop/OLPC, the Classmate PC, and other similar projects all fall in this category. Telecenters: LINCOS, MSSRF Knowledge Centers, Gyandoot, Warana and very many other kiosk projects around the world with the goal of ’bringing internet to remote village communities’ all are technology windmills. Countless Quixotes tilting at the kiosk windmill will be humorous but for the tragic waste of billions of dollars of developmental monies that are being squandered on these projects. Successful projects like the ITC e-choupal clearly demonstrate that ROI goals must take the dominant role and technology just another enabler among many other factors.

Paucity of local research: There is a general paucity of social scientists involved in researching ICTD projects in India. This gap is slowly being filled by multinational R&D labs establishing presence in India and encouraging quality social sciences research. The general lack of systematic study and learning from ICTD projects based on solid econometric analysis is adversely affecting our ability to harness the real benefits of ICT for development. V. C ONCLUSION Kapur et al. [25] provide details of developmental expenditure by the Government of India and the inefficiencies in the process: an astounding Rs 79,000 crores (US$ 17.5 billion) is allocated in the 2008-09 budget for central schemes.

“... in each case only a small fraction of overall resources reaches the poor due to, in varying degrees, targeting inefficiency (inability to reach the poor), leakages (to the non-poor), participation costs (foregone earnings that are especially consequential in employment programmes) and large administrative costs. ...” “... in 2005, the Planning Com- mission estimated that the government spends Rs 3.65 to transfer Re 1 worth of food, suggesting leakage of about 70 percent.” [25] ICTs have the potential to improve efficiencies in all the development projects. However, there is a real danger that ICTs add another major cost layer to the process without removing the underlying inefficiencies or reducing leakages. Every rupee spent on an ICTD project is a rupee that could have been directly given to an intended beneficiary (the direct cash subsidies approach advocated by [25]). Hence it is critical for everyone involved in ICTD projects to put the ROI of any project as the overriding parameter and use rigorous econometric analysis as an integral part of all stages of ICTD projects, from conception to implementation and evaluation.

Table 1: Summary of Projects Serial No.

ICTD Project Name

Did the project have measurable goals?

Is there some ROI measure?

Is the ROI a key factor driving the project?

Status of the project

1

Chhattisgarh Education

No. Just “use Simputers for rural educations”

No.

No. Just the excitement of supporting a new technlogy for education

Pilot !successfully" completed. No scale up. Closed.

2

Electricity Bill metering

Yes: read meters and distribute bills within a fortnight every month

Yes; Cost per bill fixed. Increased revenue collection.

Yes.

Successful pilots. Scaled to cover many States. Expanding to other utilities billing.

3

Bus ticket vending

Yes. Simplify conductors workload. reduce pilferage. Improve revenue collection.

Yes: Cost per ticket issued is basis of tender. Increased revenue collection.

Yes.

Successful pilots. Scaled to cover many States. Expanding rapidly to cover all buses, local and long distance.

4

NSSO

yes: Reduce the time for data collection, aggregation and report generation

NO: the cost of hardware and solution known. No attempt to evaluate the economic benefit of goals.

NO; since there was no clear idea of returns.

Did not go to even the pilot stage in spite of successful demonstration of technology.

5

TB Monitoring

Yes. Improve the efficiency of Field Inspectors and to have immediate access to data

NO: No clear enunciation of value of the goals.

No

Successful pilot. Abandoned after WHO funded project was completed.

6

ANM PDAs

Yes. Reduce workload, improve health care delivery.

NO: Unclear as to how the improvements translate to measurable benefits.

NO

Project abandoned after pilot phase.

7

e-choupal

Yes

Yes

Yes.

Remarkable success. Expanding.

Table 1: Summary of Projects Serial No.

ICTD Project Name

Did the project have measurable goals?

Is there some ROI measure?

Is the ROI a key factor driving the project?

Status of the project

8

TN Sugars

Yes: Improve sugarcane procurement by continuous interaction with farmers.

Possible: Cost of solution known. But unknown if benefits have been quantified.

Funded by government to study the possible benefits.

Pilot successfully completed. Work in progress to scale. Needs to be studied.

9

Bhoomi-Suggi

Yes: Decrease workload of VAs. Reduce data collection and aggregation time

No: No clear enunciation of value of the goals.

No.

Successful pilot running for 8 harvests in many districts. Not been scaled up so far.

10

SKS Microfinance

yes. Number of customers, amount of loans to be disbursed

Yes. Profitable operation of the loan business.

Yes: A forprofit company

Very successful

11

Gram Suvraj Chhattisgarh

No. Fuzzy goals about access to Government schemes

No.

No

Ongoing large pilot. Requires to be studied.

References 1. Gandhi, R. eta al., Digital Green: Participatory Video for Agricultural Extension, In Proceedings of the ICTD 2007 Conference, Bangalore India, pp. 21-30. 2. Heffernan, C. and Nielsen, L., The LivestockGuru: The Design and Testing of a Tool for Knowledge Transfer among the Poor, in Information Technology for Internataional Development, MIT Press, Volume 4, Number 1, Fall 2007, pp. 113-121 3. Plauche, M. and Nallasamy, U., Speech Interfaces for Equitable Access to Information Technology, in Information Technology for Internataional Development, MIT Press, Volume 4, Number 1, Fall 2007, pp. 6986 4. S. Guo et al., Design and Implementation of the KioskNet System, In Proceedings of the ICTD 2007 Conference, Bangalore India, pp. 300-309. 5. Kenniston, K., Grassroots ICT Projects in India: Some Preliminary Hypotheses, ASCI JOURNAL OF MANAGEMENT 31(1&2), 2002. Also at http://www.mit.edu/people/kken/PAPERS/ASCI_Journal_Intro_ _ASCI_version_.html 6. Sreekumar, T.T., Decrypting e-governance: narratives, power play and participation in the Gyandoot intranet, The Electronic Journal on Information Systems in Developing Countries (http://www.ejisdc.org), Vol. 32, No.4, pp. 1-24. 7. Pal, J., Examining e-literacy Using Telecenters as Public Spending: The Case of Akshaya, In Proceedings of the ICTD 2007 Conference, Bangalore India, pp. 59-67. 8. Institute for Development policy and Management, http://www.sed.manchester.ac.uk/idpm/ 9. Heeks, R., Most eGovernment-for-Development Projects Fail: How Can Risks be Reduced? 2003. http://idpm.man.ac.uk/publications/wp/igov/index.shtml 10. Vasudevan, R., Changed Governance or Computerized Governance? Computerized Property Transfer Processes in Tamil Nadu, India, in Information Technologies and Internataional Development, MIT Press, Volume 4, Number 1, Fall 2007, pp. 101-112. 11. Patra, R., et al., Usage Models of Classroom Computing in Developing Regions, In Proceedings of the ICTD 2007 Conference, Bangalore India pp. 158-167. 12. Jensen, R., The digital provide: Information (technology), market performance, and welfare in the South Indian fisheries sector, in The Quarterly Journal Of Economics, Vol. CXXII, Iss. 3, August 2007, pp. 879-924. 1

13. WHO health Impact Assesment. http://www.who.int/hia/en/ 14. Horton, S. et al., Copenhagen Consensus 2008 Challenge Paper: Hunger and Malnutrition, http://www.copenhagenconsensus.com/Default.aspx?ID=1149 15. Raiti, G. C., The Lost Sheep of ICT4D Research, in Information Technologies and International Development, MIT Press, Volume 3, Number 4, Summer 2006, pp.1-7. 16. Reuben, R. Mobile Phones and Economic Development: Evidence From the Fishing Industry in India, in Information Technologies and Internataional Development, MIT Press, Volume 4, Number 1, Fall 2007, pp. 5-17. 17. Quantum Aeon Pvt. Ltd., Bangalore India. (http://www.quantumaeon. com/btm.htm) 18. National Tuberculosis Institute Bulletin, Feb 2005. http://medind.nic.in/nac/t05/i1/nact05i1p41.pdf 19. Computer maintanence corporation Ltd., Web report. http://www.cmcltd.com/industry_practices/e-Governance/healthcare/ ihc.htm 20. ITC e-choupal, corporate website. http://www.itcportal.com/rural-development/echoupal.htm 21. Below Poverty Line definion of Planning Commission, India. http://www.planningcommission.gov.in/news/prmar07.pdf 22. Chhattisgarh informatics promotion Society (ChIPS) e-Gram Suvaraj project. http://chips.gov.in/content/projects.htm 23. RFP for Bhamassha project, Government of Rajasthan. http://finance.rajasthan.gov.in/bhamashah.shtm 24. Oppenheimer, T., The Flickering Mind: Saving Education from the False Promise of Technology, Random House Trade Paperbacks, 2004. 25. Kapur, D., Mukhopadhyay, P., Arvind Subramanian. A., The Case for Direct Cash Transfers to the poor, Economic and Political Weekly, April 12, 2008, pp. 37-43. 26. Speech at the UN by Usha Thorat, Deputy Governor RBI. http://www. iibf.org.in/financeq/docs/Ushathorat\_Fin\_Inclusion.pdf.

2