SLS-6b1 Critical Success Factors

In 2010 PolyU made a bold decision to make service-learning credit-bearing and compulsory.  Financially it has committed a tremendous amount of funds to finance the offering of 60+ academic subjects to 4,000+ students each year.  It is proper to ask whether the effort is yielding the desired outcomes.  We have been monitoring the results since the beginning, and the results have been very encouraging.  Through pre- and post-surveys, the results indicate that the students are achieving the four major learning outcomes as expected: academic/intellectual, social, civic and personal.  The students’ performances as measured by the subject grades have been more than satisfactory.  In fact, average grades for SL subjects tend to be a bit higher than other academic subjects.  Students satisfaction as reflected through scores on the “Subject Feedback Questionnaires” filled in by the students at the end of each semester are also positive.  We have conducted interviews with students, teachers, and partner NGOs and the results are good.  Hence, overall, PolyU can be satisfied that the Service-Learning Requirement is achieving the expected positive results.  In other words, service-learning works for PolyU.

But we wish to understand better how the learning outcomes are achieved.  For example, specifically which factors are critical for achieving which of the intended learning outcomes?  Partly it is out of intellectual curiosity. When we know something works, we want to find out how it works. Partly, it is also to identify ways to achieve even better results.  In the 2014/15 academic year, we conducted a study to identify the key pedagogical elements influencing the student learning outcomes.  Target participants were 2,880 students enrolled in 55 credit-bearing SL subjects offered in the year.  A total of 2,214 valid returns were received, with a response rate of 76.9%.  Participants were invited, at the end of the subject, to complete a post-experience questionnaire that required them to: (a) indicate the quality of different aspects of their learning experience, and (b) rate their attainment of the intended intellectual, social, civic and personal learning outcomes as well as their overall learning gains as a result of attending the SL subject. Pearson’s product-moment correlation and multiple regressions were performed to identify and compare the relative contribution of the various factors that influence students’ learning experience and their attainment of specific learning outcomes of SL. 

Results show that students’ self-reported gain in intellectual development was most strongly influenced by “preparing students adequately for service”, “challenging and meaningful tasks” and “perceived benefits of service to the community/service recipients” On the other hand, students’ self-reported gain in social development was most strongly associated with “relationship with teammates” and “motivated and supportive teammates”.  Students’ self-reported civic development was found to relate most strongly to “challenging and meaningful tasks’, “perceived benefits of service to the community/service recipients”, “appreciation by collaborative agency/recipients” and “preparing students adequately for service”, whereas their reported gain in personal development were mostly influenced by “student effort in planning, preparing and delivering the service”, “challenge to try things never done before”, “challenge and meaningful tasks” and “perceived benefits to the community/service recipients.”  

Based on the analysis, we found that students’ learning outcomes from SL are influenced by a multitude of pedagogical elements, some more influential than others.  The relative impacts of the different pedagogical elements are not uniform across different outcomes.  14 pedagogical elements are significantly associated with at least one of the 4 SL outcomes.  

Based on these results we make these recommendations to the teachers of SL subjects:

1. Involve students in challenging tasks.
1. require students to apply knowledge & skills to deal with complex problems in the service setting.
2. challenge students to move outside their comfort zone & try new things. 
2. Design meaningful services that meet genuine community needs.
1. Students will work harder and learn better when they think their service is meaningful and 
2. the service is valued and appreciated. 
3. Prepare students well for the service.
1. Better understanding of the clients & community
2. Knowledge & skills for planning and implementing the SL project.
4. Engage students in direct interaction with clients
1. reinforce their understanding of social issues & problems
2. develop empathy for the needy
3. provide direct feedback on the service
5. Motivate students to invest more time and effort in service
1. long service duration & deep enough experience have positive impact on student learning 
6. Provide diverse SL subjects & projects, and allow student choices
7. Engage students in critical deep reflections through structured reflection tasks with clear instruction. 

One of the most encouraging results come from the observation that almost all of these factors are generally under our control.  One of the beliefs on SL is that students come with prior perceptions and attitudes.  If they are interested in the subject and are motivated that the fortunate teacher can help them achieve good outcomes. If they don’t then there is not much that the unfortunate teacher can do to change that.  The study tells us that this belief may not be completely true.  There is indeed much that a teacher can do to provide the student with a good learning experience, and achieve good learning outcomes.  We can even achieve that despite having some students with an unhelpful attitude.  More on that in the next section. 

The results of this study was published in the paper “A Multivariate Study of Factors Influencing Students’ Learning Outcomes from Service-Learning (SL), presented at the International Association for Research on Service-Learning and Social Engagement (IARSLCE) 2016 Conference, 26-28 September, New Orleans, Louisiana, USA.