Review of the Contract for the In-Country Management of the New Zealand Development Scholarship and Short Term Training Award schemes in
Papua New Guinea
Report Summary | Eva0719
Print-friendly version  Report Summary | Eva0719sum | PDF 36kb |
|
...........................................................................................................................................................................
Date | March 2007
Team | Elena Procuta, Scholarships Unit, NZAID
Background
Under New Zealand Agency for International Development's (NZAID's) bilateral development assistance programme for Papua New Guinea (PNG), up to 15 New Zealand Development Scholarships in the Public category (NZDS-Public) are offered annually to eligible PNG citizens. The purpose of the NZDS scheme is to provide selected individuals with knowledge and skills to contribute to the sustainable development of key sectors in their home country, with the aim of reducing poverty, promoting good governance, and contributing to the human resource base of targeted countries. The scholarships are offered for full-time, tertiary level study in New Zealand . The fields of study for these scholarships are linked to PNG's Medium Term Development Strategy, National Strategic Development Plan and Human Resources Planning Framework, and/or NZAID's priority development sectors under the PNG Programme. In addition, up to NZ$250,000 [sic] is allocated annually to fund PNG citizens to undertake short-term, vocational training in New Zealand through the Short Term Training Award (STTA) scheme, linked to human resource development needs identified by the Government of PNG (GoPNG) and NZAID.
NZAID engaged the Port Moresby-based contractor GRM International Pty Ltd for three years from April 2006 to manage the in-country, pre-award processes of these two NZAID scholarships schemes in PNG. The contract aimed to improve the efficiency and effectiveness of the management of pre-award scholarships processes in PNG, and thereby improve the quality of the applications and of the short-listed scholarship nominees. This was to be achieved by better targeting of scholarships to priority sectors and key agencies aligned to NZAID's PNG Programme, more transparent and inclusive selection processes agreed between GoPNG and NZAID and improved communication. Prior to the contract, the pre-award scholarship processes were undertaken by NZAID staff in Port Moresby.
Purpose and Objectives
The primary purpose of the Review was to assess GRM's first-year performance against the outcomes under the contracted scope of services, and to determine whether any of the agreed tasks needed refining to further improve scholarship processes prior to the commencement of the 2007 selection rounds. The Review also considered whether to outsource the management of the pre-award processes for the Open category of the NZDS scheme to GRM.
The timing of the Review was originally planned to take place in November 2006 to coincide with the outcome of AusAID's re-tender for in-country management of Australian Government scholarships for PNG. NZAID and AusAID had initiated discussion in late 2005 around harmonising pre-award processes of the Australian Development Scholarship (ADS) scheme and the NZDS scheme.
The third objective of the Review was to identify whether there was any scope to harmonise donor processes for the remaining term of the contract with the aim of streamlining the pre-award management of AusAID and NZAID scholarships.
Main Findings
GRM's management of the pre-award processes for the NZDS-Public and STTA schemes for PNG was well-organised and timely, efficient and transparent application and selection procedures were instigated. Communication between NZAID's PNG Post ( Port Moresby ) and the two GoPNG nominating authorities1 around scholarships improved considerably, resulting in greater GoPNG involvement in scholarships selection and cooperation among the primary stakeholders. The nomination authorities expressed satisfaction with the arrangements and content of the joint-selection meetings and their involvement. The lack of support from NZAID Scholarship Unit during the hand-over period resulted in the NZAID PNG Post dedicating more time than anticipated to clarification of processes.
The process was more transparent than previously, although there was not clarity around advertising and the distribution of forms. Comprehensive applicant data will be used by NZAID for monitoring and evaluation purposes, including training demand in relation to the PNG programme. For the 2006 intake, 33 percent of NZDS expressions of interest forms were from the public sector, and 33 percent were from female applicants. Just over 20 percent of applicants came from key targeted agencies, with nearly 70 percent of applicants identified with priority sectors. Direct promotion of NZDS to priority agencies for future intakes may help better align the applicants with NZAID/GoPNG priorities. The data can also be used by Posts to analyse recurrent training demands and addressing in-country training needs. NZAID can use this data for monitoring and evaluation purposes.
Insufficient funding of promotional activities, outside newspaper advertising, limited the extent to which specific groups – including priority agencies and provinces – could be targeted. Agreed GoPNG and NZAID selection criteria ensure that scholarships continue to be awarded for study and training linked directly to NZAID's development assistance focus in PNG. The results from International English Language Testing System (IELTS) testing suggest that PNG scholarship recipients experience difficulties with academic English language proficiency.
Although there are advantages to the PNG Programme to allocate a specific number of NZDS-Open under the programme (rather than being available through the contestable Pacific regional allocation), further consultation needs to take place internally before the PNG programme can determine whether to proceed with this approach.
AusAID has not yet finalised the shape or management of the Australian Government scholarships offered to PNG. It is unlikely, therefore, that pre-award scholarships processes will be harmonised with AusAID processes in the lifetime of GRM's current contract.
Although the NZDS and STTA scholarships offered to PNG are aligned to the PNG Programme, and GoPNG's involvement in the selection of awardees has increased, the lack of a NZAID human resource development strategy to support in-country training and capacity building with New Zealand-based scholarships and training means that the impact of NZAID scholarships will remain difficult to measure.
The report makes many recommendations at the operational level to improve performance and outcomes:
- promotional activities (targeting, funding)
- timing of selection meetings
- further definition of priority sub-sectors.
- IELTS testing for all shortlisted applicants regardless of professional background. (PNG Programme to agree and budget for 6 months New Zealand-based English language training for candidates that fail proficiency criteria but score better than 5.5 across all IELTS bands)
- timing of submission of nominations to the NZAID Scholarships Unit and for the selection rounds
- outsourcing of NZDS placement role (by Scholarships Unit)
- scheduling an integrated mid year interview, IELTs testing and information session in Port Moresby . GRM to submit revised budget to include costs associated with IELTS testing of up to 30 applicants (travel, per diems etc).
Post to maintain responsibility for financial management of the contract, inclusding confirming to NZAID (DPM) that GRM has met contracted outputs. GRM to submit 6-monthly reports with actual recoverable costs.
The feasibility of the outsourcing the NZDS-Open scholarships to GRM is raised in the Review, as is the discussion between the PNG Programme and the Scholarships Consultative Group of the allocation of these scholarships.
Key Issues and Learning for NZAID
Gender Equality
The Review reported a gender imbalance in the 2006 intake of the NZDS; however without multi-year data for PNG and the comparison with all NZAID-funded scholarship programmes it is not possible to determine trends. This raises the issue of whether NZAID should devote resources to redress imbalances at the level of contractors who manage individual scholarships programmes or at a higher level.
NZ verse In-country Based Short Term Training
The Review raised questions about the content, quality and relevancy of NZ verse in-country based STTA that could be addressed through a deeper analysis of the complementarities between these .
IELTS Testing as a Factor Contributing to Academic Success
Through the GRM contract mandatory IELTS testing of all short-listed candidates was instituted. An analysis of the data revealed that assumptions of PNG English language proficiency, on the basis of English as the language of instruction in school, are not correct. This means that lower levels of English language proficiency are a barrier to learning at the tertiary level. This finding has implications for other NZAID bilateral scholarships programmes that do not require IELTS testing.
Integrated Approach to Human Resource Development
The Review observed that, although the NZDS and STTA scholarships offered to PNG are aligned to the PNG Programme's development assistance focus and the GOPNGs involvement in the selection of awardees has increased substantially, the lack of an NZAID human resource development strategy to support in-country training and capacity building with New Zealand-based scholarships and training means that the impact of NZAID scholarships will remain difficult to measure. Consideration could be given to the application of this across NZAID country programmes.
Challenges of Managing and Coordinating Multiple Actors
Clear role definition is a key to successful management of the in-country and pre-award processes of NZDS and STTA in PNG which involve two Management Services Contractors, the Post and the Scholarships Unit.
Standardised Approach for Consistency and Cost-savings
There are a range of practices discussed in this Review which could be useful in all NZAID scholarship programmes: examples of these include targeting and budgeting of promotional activities (section 3.1), integrated approach to pre-departure briefing (section 3.8.3), targeted STTA rounds (section 3.4.2) and STTA management in country (section 3.2.3).
Follow-up
Following submission of this Report to NZAID's Evaluation and Research Committee, the PNG Country Programme team requested revision of several of the recommendations. NZAID is working with the contract provider to implement all of the recommendations.1National training Council (NTC) and Department of Personnel Management (DPM) Back
To request a copy of the full report email evaluation@nzaid.govt.nz and quote the reference number - Eva0719.
return to | main Reports Page