The experience of developing evaluation criteria made me realize that there are key elements to what makes learning resources effective. But in applying criteria to diverse learning resources, it can be challenging due to the pedagogy built into the resource and the learning theory it is based in.

Since all the resources for LRNT 527 were developed through the design thinking process (Stanford University Institute of Design, 2016) , criteria that followed from this empathetic design process seemed appropriate. Therefore, our group determined criteria based on the design thinking process (Stanford University Institute of Design, 2016) that aligned with criteria within the Learning Object Overview Instrument (LORI) from Leacock & Nesbitt (2007). The steps in the design thinking process that the class had completed for the resource included empathize, define, ideate and prototype. However, we determined our focus would be on the empathy and define phases for the criteria. This was based on time spent on these phases in the class and their overall importance we felt that they contributed to the design of the resources.

In discussing which criteria from LORI best fit or could be adapted to fit these two phases, we determined design, usability, user awareness and problem assessment. These criteria were also applicable to all of the resources evaluated and we determined this through ensuring each member of the team had a thorough understanding of one resource when we discussed the criteria. Overall, having the LORI framework was very useful to have a consistent and peer-reviewed list of criteria to choose from. It was also interesting that we had the opportunity to develop and apply criteria to resources that had all been developed through the same design process.

Here is a detailed look at our team’s criteria.

References

Leacock, T. L., & Nesbit, J. C. (2007). A framework for evaluating the quality of multimedia learning resources. Journal of Educational Technology & Society, 10(2).

Stanford University Institute of Design. (2016). Bootcamp Bootleg. Retrieved from http://dschool-old.stanford.edu/wp-content/uploads/2013/10/METHODCARDS-v3-slim.pdf