City and Islington College is one of the largest providers of further       education in England, with over 15,000 students studying in five centres across Islington. Listening to the learner’s voice is at the heart of the College’s strategy to continually improve the quality of their teaching and students’ learning experience. Surveys are used to gain feedback from learners not only on teaching but also on services and facilities offered by the College.

Until 2009 the College ran mostly online surveys. Paper based surveys were only used for learners with learning difficulties and some learners where English was their second language, but these questionnaires needed to be manually keyed into the survey system.

When the College’s Management Information System (MIS) technology changed, the in-house built online system for surveys could no longer be used effectively, as the database that it had been linked to was no longer in use. As a temporary solution, an online survey was designed using SharePoint. This approach had its limitations that resulted in additional manual steps and the decision was made to identify a new solution to meet the complex needs of the organisation.

The MIS team used this opportunity to select a suitable system that would also further improve survey administration in terms of efficiency and quality. “EvaSys was the only product we found that had the facilities to manage both paper and online surveys; advanced reporting and was compatible with our ICT and MIS infrastructure”, explains Kirsty Webb-Wood, Director of Learner Engagement,      Marketing & Admissions.

Since the start of the academic year 2009/10, City and Islington College has used EvaSys Survey Automation Suite to manage their processes and conducts two college-wide surveys annually with the system. The first is the start of year survey in October covering the student inductions to the College and their initial impressions of their courses and the second is the end of year survey in April which covers all courses and the students' experience of their course and the College. The results of the   surveys are used as part of the College's quality assurance cycle. In addition, the system is used for smaller survey projects, such as an open days visitor’s survey.

Previously, flexibility in terms of topic areas covered by surveys was limited because the Learning and Skills Council (LSC) ran a compulsory online survey which served as a benchmark. Thus, questions had to be comparable with other organisations. With the LSC now dissolved, the College decided to widen the previously limited range of questions. Now they cover a broad range of topics, both related to  internal quality and to policy issues, such as questions on diversity or Every Child Matters. Survey    results on these areas also serve as additional evidence in self-assessment reports for Ofsted.

“We use the surveys as an instrument to check we get things right”, says Kirsty Webb-Wood, “we ask for feedback on initiatives that have been implemented through quality enhancement outcomes and new services such as our student portal my.candi”. As part of the learner’s voice agenda, EvaSys survey results are also used by the quality team in a mixed-method approach to identify possible topics for the 60 focus groups they run to complement the quantitative surveys. Key issues that become apparent in the surveys, such as learning being disrupted by students arriving late, are discussed by students in a qualitative focus group session and resolutions are identified.

Paper based:

Two four-page questionnaires have been designed with the inbuilt designer in EvaSys, which creates an optimal layout for scanning. Once designed, questionnaires can be changed or amended easily. For example, they could easily adapt the survey for learners with English as a second language (ESOL) and learners with learning disabilities.

In addition to the questionnaires, a coversheet is generated for each course. The cover sheet is based on the course details previously imported into EvaSys by CSV. The cover sheet includes a barcode that automatically links the questionnaires with the specific course and instructor during the scanning  process.

Online:

In order to create efficiency savings in administration, online surveys are also used where it makes sense and where the learners have access to computer labs to complete the surveys in class. For the online administration, the same two four-page questionnaires designed in EvaSys are used to generate the template along with unique passwords and time controls to set reminders.

Moving from online to hybrid:

The move from online to hybrid paper and online surveys has been very successful as responses increased more than threefold. While 1,500 students had responded to the previous online start of year survey, 5,000 learners filled in the paper-based survey at the start of year 2009/10, which equals a response rate of 72%. “It is phenomenal that we are actually getting feedback from so many more learners.” says Kirsty Web-Wood. The broad response means that surveys yield more information in terms of open comments and are more representative of all students. This is also an important aspect when survey results are used to complement the self-assessment provided for future Ofsted audits.

“The huge response that we gained from the first survey off sets the extra time that was necessary to initially set up the survey processes in EvaSys for the first time”, explains Kirsty Webb-Wood. As they use the system more effectively over time, City & Islington expects the process to become more cost effective.

Staff reaction has been very positive to EvaSys. One reason for this is that paper-based surveys are much easier to use for them than online surveys for which they need to book computer labs for the students. However, staff in some centres such as the Sixth Form College are also keen to use the online technology and this is planned for the future.

Another reason why EvaSys has been well received by staff is the reporting. The EvaSys reports are much more user friendly than the previous report format. Graphs visualise numeric results for each question. In addition, profile lines present the results for all scaled questions at one glance, making results easy to compare. They can also be used to compare results against previous years. In order to make the most of this reporting feature, City and Islington College has changed survey questions so that they are all using more informative scale questions rather than simple “yes/no” questions.

Survey reports are generated automatically by EvaSys as soon as the scanning of questionnaires is finished. The results are then published on the College intranet SharePoint. Reports are made available for each of the five College centres, service areas, learning centres, facilities, marketing and by             curriculum area. Reports are also made available without comments to be viewed by all staff. Reports  by course have restricted access and curriculum managers access them via protected folders.

The possibility of VLE (virtual learning environment) integration was an important aspect in the selection process for EvaSys. “Right now we’re looking with our surveys for top level feedback, but at course level when we look for more specific feedback an integration of surveys into our VLE system might help”, explains Kirsty Webb-Wood. As they are currently in the process of moving VLE products, they will consider this as a next step once they have completed the move to Moodle.

Paper or Online – Both worlds in one system

Having the flexibility to conduct paper and online surveys with the same system and collating the results of both is one of the major benefits of using EvaSys for the college, states Kirsty Webb-Wood. However, increased response rates and the enhanced reporting with year-on-year comparisons are all positive effects of implementing the new survey automation system. “We have found it very easy to work with Electric Paper. They have coached us all along the way addressing our needs and specific questions as they arose.”

back: References