Professor Joel Smith is Distinguished Career Teaching Professor in philosophy at Carnegie Mellon University and is currently the co-principal investigator of a research project on barriers and affordances to implementation of instructional innovations titled is currently the co-principal investigator of a research project on barriers and affordances to implementation of instructional innovations titled “Understanding and Overcoming Institutional Roadblocks to the Adoption and Use of Technology-Enhanced Learning Resources in Higher Education“.
[This is the first time in a while that I’ve attempted to live blog a seminar so here goes.]
Sian Bayne is introducing Professor Joel Smith.
Joel starts with stating his perspective is as an administrator of departments that have supported faculty in implementing TEL but returned to teaching in 2013. But he is currently the co-principal investigator of a research project on barriers and affordances to implementation of instructional innovations titled “Understanding and Overcoming Institutional Roadblocks to the Adoption and Use of Technology-Enhanced Learning Resources in Higher Education” with Lauren Herckis.
The study is at CMU so is situated and contingent to that local situation and a point of interest is whether these results resonate with us here and to what extent they are generalisable to other contexts.
CMU has long history of research and development around teaching and technology especially from the work of Herbert Simon and his emphasis on researching and understand what works in technology enhanced learning (TEL).
The effectiveness of TEL is based on:
- Improving LOs (ie, pre-test/ post-test)
- Implementation of innovations.
Lots of research on the first and on whether TEL improves the achievement of learning outcomes. For example, the Open Learning Initiative statistical reasoning course study that found strong evidence of increased efficiency of learning using TEL. Further studies using randomised trials of the course in use across seven universities in 2010 – 2012 showed increased effectiveness for hybrid/ blended courses against traditional delivery modes. Yet, the statistical reasoning course is rarely used at CMU or other universities despite it being both free and open source.
So this leads to the question of why are TEL innovations not more widely adopted? What are the barriers and facilitators of implementation of TEL innovations. Joel noticed that in related studies, key barriers were identified around institutional structure and cultures and so selected a anthropology-based mixed method approach to the study.
This mixed methods approach used ethnography, artefacts, surveys and interviews across faculty, students and administrators. The study focused on four projects that were either starting or looking to transfer to new programmes.
- Introduction to Computer Science
- Discrete Maths
- Writing across Curriculum
- Statistical Reasoning.
The study uses extensive digital ethnography, document analysis and institutional analysis to identify variable of roadblocks and affordances of implementation for the survey. The interviews focus on ideas of ‘good teaching’ and ‘being a good professor’ and identifying factors considered in making instructional choices.
The main themes of the findings of the study are:
Collaboration: miscommunication and lack of information transfer at handover. Most TEL innovations are collaborations involving technologists, subject experts and cognitive scientists. Miscommunication through misunderstanding and disagreements along with variations in priorities undermining coordination. Collaborations require a guiding champion to avoid these failures.
Institution structure and processes: un-synced cycles of innovation support and conflicting ideas of ‘good teaching’. Cycles of complex system that are out-of-sync as a university has distinct cycles that shape faculty and administration abilities to engage in innovations. These include career cycles (reseach and tenure); institutional support (learning and teaching centres) and support for faculty; technology infrastructure decisions (is the technology in place) and global changes in technology. These cycles interact and are frequently out of sync with one another.
Concepts of ‘good teaching’: conflicting models of ‘good teaching’ and skepticism about evidence from educational research. Very often views of good teaching are highly subjective drawing on early learning experiences. This may be modelling an ‘inspiring’ professor or from their parents (!). So educational research struggles to displace these mental models with evidence-based alternatives especially where the alternatives challenge the existing mental models.
Faculty identifies as teachers: threats to autonomy and threats to student approval of teaching so faculty are unwilling to take risks in teaching as it make affect student satisfaction. Student feedback reinforces a focus on satisfaction not learning effectiveness: “why would I even take the chance?” Also innovations driven by management are perceived as threats to faculty autonomy.
An innovation that matches mental models of teaching and key aspects of self-identity of faculty is more likely to become more widely adopted.
So how can these roadblocks be addressed in policies and practices that is sensitive to situated contingencies and complexities?
Classification of mental models of ‘good teaching’ as: (a) relational – the bond with students; (b) content based and clarity of delivery; (c) measurability and using evidence-based practice; (d) practical that focuses on the right problems for student to address. So TEL implementations should involve stakeholder discussions on what constitutes good teaching is important and may lead to a conclusion that the collaboration simply would not work or needs to be significantly reshaped. Strategies for implementation should resonate with the instructional values of the participants. But these discussions tend to be very rare in higher education.
The seminar finished with Joel raising the question of whether education should look to health services for models of an ‘implementation science’ supporting the wider use of innovations in complex systems [one example could be the UK NHS Improvement service].