The research design used for the 2015 A2F Survey is summarized in the following figure:
Like in 2008, the 2015 A2F survey is a comprehensive national demand-side survey conducted with individual respondents. The survey design is based on the methodology that was used in 2008 developed by FinMark Trust (South Africa) and the World Bank.
During the preparation phase, it was found that the results of focus group discussions made in 2008 were sufficient guides for the design of the questionnaire and they were not repeated in 2015. Nonetheless, with the aim of obtaining a more in depth understanding of the respondents motives and strategies, questions were added to the questionnaire. It was also decided that, as for the 2008 survey, the 2015 A2F survey would focus on individuals, taking into consideration the predominantly individual nature of financial decisions discovered during the 2007/2008 focus group discussions. . Like in 2008, only people above 18 years old have been interviewed.
As the objective was not to simply replicate the 2008 A2F survey but to also obtain specific results as per the survey's term of reference (see list of attendants in appendix), research questions were formulated and translated into a basic structure for the questionnaire and presented to the stakeholders during two workshops in Karachi and Islamabad attended by members of the industry and of the Pakistani and foreign public agency involved in the design of policies and of programs for the development of the financial sector (see list of attendants in appendix). These workshops enabled the research team (Horus and Gallup Pakistan)to include actual questions relevant for the stakeholders and to check on the present range of financial products and services available in the market.
Whereas the structure of the 2008 questionnaire was kept, additional questions were introduced to better understand the financial strategies of the Pakistani population and to take into account the introduction in the market of services that were absent in 2008, specifically mobile accounts and mobile money services. Hence, the obtained questionnaire needed to be tested anew. The first tests were aimed at verifying that questions on difficult concepts were properly understood by the respondents. Great care was applied to get an Urdu version of the questionnaire employing a language that every person in Pakistan can understand without compromising the required precision.
Well trained interviewers are crucial for the implementation of questionnaires on difficult matters using semi-open questions; the interviewers must understand enough about the underlying subjects to be able to provide explanations when required without creating bias. HORUS provided a trainer who worked with the Gallup Pakistan team on guidelines for the interviewers trainers and supervisors. Classroom training was conducted using Power Point presentations presenting an overview of the survey's objectives and research questions, followed by teaching on financial terms and explanations on specific questions in the questionnaire. Then, the interviewers were first asked to do mock interviews in pairs and trainers monitored their performance; detailed feedback was shared. The training went on with teaching on GPS-coordinated takings. Finally, interviewers with trainers went on doing pre-tests in the field and providing feedback on their performances. The duration of the training in each of the regions was 4 days with full day sessions.
The pilot was conducted with a sample of 360 households. This sample was built using the already sampled List of Locations using Stratified Random Sampling Approach. 15 interviews per Enumeration Area were conducted using Random Walk method. A total of 24 Enumeration Areas were visited by Gallup Pakistan field work team. For reasons of practicality and keeping the aim of pilot exercise, pilot activity was conducted in 3 provinces namely: Punjab, Sindh and KP.
Baluchistan, which represents only 5% of Pakistan's population, was left out. The purpose of the pilot was to carry out a number of tests on the questionnaire's robustness and reliability; the productivity of the interviewers in the field; the data entry procedures; the tabulation instructions and the comparability of results with the 2008 results. These tests were found positive.
Comparability of the obtained access strand categories was tested again after 1,250 interviews were completed: the 2015 results were found plausible when compared to those of 2008 with due consideration of the known evolutions on the supply side. These preliminary results were presented to the stakeholders in May 2015.
The main survey was then completed in the field until mid of June 2015.