UFHRD 2014: 4 June, parallel session on Assessment, Measurement and Evaluation
We’ve now moved to the parallel sessions and I’ll be taking short notes on these.
We’re starting with Lynne Booth, Michelle Blackburn and Simon Warwick (Sheffield Hallam) on the effective use and assessment of web-based collaborative learning. The learning intervention was in the context of a UG programme on business and human resources and the development of a first year course. Also positioned in the context of a institutional emphasis on employability and the student experience to develop tangible evidence of the HR knowledge of students. So the course was based on the production of HR websites by the student groups using Google Sites so outside the VLE but able to use Sites templates.
While group work was popular but not for assessment due to social loafing (and emphasised by employers) so generated a way of differential marking. Groups were self-selected by should be mixed gender and mixed culture with an aim for authentic learning to enhance employability.
The websites were tone developed as a response to a development need identified by a fictional HR manager by email.
Students also had to create a academic reflection on the sites produced.
Students tended not to react to formative feedback but did appear to have a positive effect on placement rates but was difficult to mark. Little evidence of social loafing.
There remains an issue on scalability and support from learning technologists for staff and students. Also, all other modules use more traditional assessments.
Placing the Transfer of Learning at the Heart of HRD Practice with Vivienne Griggs, Dianne McLaren, Barbara Nixon, Joanna Smith of Leeds Met but the research involves both Synaptic Change Ltd and Connecting Housing on testing a transfer of learning model. The research on the model is framed by issues of alignment between L&D and business strategy as well as the use of big data in evaluation of L&D. The model was based on the importance of the line manager in L&D in embedding training and development outcomes in BAU. Also sought to embed evaluation as a process in L&D interventions.
A key focus was to define success criteria for the individual, the trainer, the line managers and the organisation which can be a challenge.
The training was on having difficult conversations which was followed by stakeholder focus groups and interviews on how effective the model was seen for transferring and embedding training and learning. A stakeholder approach including the trainees in defining success criteria was important in the overall success of the intervention. However, there was little preparation by line managers for the transfer of training. Line managers seemed disengaged from the use of the training but peer support was effective. A trainee comments that they would want their manager involved!
There is an issue of scaling the process to larger organisations. Further development to involve sustainability of impacts and supporting peer-support.
Return on Investment: Contrary to Popular Belief, MOOC’s Are Not Free with Marie A. Valentin, Fred Nafukho, Celestino Valentin Jr., Detra Johnson, John LeCounte (Texas A&M) started with a introduction to MOOCs and the research questions on the true costs of MOOCs and the direct and indirect ROE based on Human Capital Theory. The research seemed to be
MOOC providers making revenue from credits; certification; but also recruitment services pay a fee for data on users, text book sales by linking the course to the text book; selling data to third parties and claims for HEI recruitment to mainstream programmes. This was a work-in-progress and was very much orientated to xMOOCs and the VC-backed US MOOC providers.