The d.School design thinking process, a systematic approach for developing user-centered, real-world educational products (d.School, 2018), was used to develop a prior learning assessment framework to advise pedagogical provision and adaptation in digitally-mediated critical thinking-based courses (Carpenter & MacKay, 2021). The framework, entitled “The Empathetic Informed Design for Digital Environments” (2021), was subjected to peer review, revealing both strengths and weaknesses in the prototype’s design. This paper will reflect on the peer review feedback and how it informs the revision process; also, it will reflect on the d.School design thinking process and its implications on my professional practice as a higher education instructor.
The d.School design thinking process proved to be a challenging but rewarding experience, and accordingly, our prototype design returned mixed peer reviews. The framework is criticized for its lack of direction, with peers indicating uncertainty about implementing each stage of the framework (Dougan, 2021; Piechnik, 2021). We aimed to create a framework that is adaptable and supportive of various educational contexts; however, over-specializing the framework risked limiting its adaptability (Schwartz et al., 2009), a challenge that ultimately may have weakened the design of the initial prototype. What good is a prototype if nobody knows how to use it? The work of Schwartz et al. (2009) suggests adaptable design lay somewhere between being “flexibly adaptive” and “prescriptive” (p.188) to be deemed effective, so perhaps specializing certain aspects of the framework will help rectify some of the inconsistencies in the design.
Conversely, peers also viewed the framework as a valuable tool to empathize with learner needs, which, according to Goldman et al. (2012) and Mattelmäki (2014), is essential to effective instructional design (ID). Piechnick (2021) explained that he often struggles to understand his student’s learning needs in DLE’s and that the framework’s questionnaire may generate critical information to help shape the pedagogical approach. Similarly, Koval (2021) explained how a lack of empathy is “the biggest flaw of online education,” suggesting the framework to be a useful working solution to understand online learners. By accommodating various learning conditions, educators’ pedagogical adaptations accurately conform to specific learning and teaching gaps, resource issues, and students’ motivational needs, thus enhancing the overall learning experience for all parties involved (Dousay, 2017). I credit human-centered design thinking (Goldman et al., 2012), a “mind shift [that is] characterized by a central focus on empathy for others” (p.16), for enabling the creation of an assessment framework that is perceivably well-suited for learners, educators, and pedagogical adaptation. The learner-centeredness of the framework appears to be a win in all accounts.
There are two key takeaways from the peer review feedback that may lead to an improved prototype. First, creating a detailed timeline with instructions would help inform users of deployment strategies (Dougan, 2021; Piechnick, 2021). For example, instructions could notify users of the following: (1) the design of the questionnaire is to collect prior-learning assessment data at or near the beginning of the course, (2) the critical challenge is designed to assess pre-existing CT skills in learners at the onset of the course, immediately following the questionnaire phase, and (3) the reflection stage can be deployed anytime throughout a course but is designed to immediately follow the critical challenge stage to indicate the learner’s ability to self-reflect and understand problem-solving processes (Carpenter & MacKay, 2021). By providing a detailed explanation of each step of the framework, the prototype’s usability will improve significantly.
A second key takeaway is that not all educators will understand how to create sufficient assessment criteria, so it may be necessary to provide additional guidance for some individuals. For example, Norum (2021) suggested that the framework address how CT unfolds in groups, stating that educators and students rely on the class as a “whole entity” rather than individual students to drive critical thought. While team dynamics do reportedly drive CT (Liljedahl, 2014), not all learning contexts require group collaboration, which is why the framework’s design accommodates all learning contexts. Norum’s critique does, however, support the need to build more guidance into the framework, such as how to construct a questionnaire that serves a specific function. For instance, a questionnaire that evaluates collaboration skills could inform educators how to formulate teams, groups, or pairings, such as matching innate leaders with inexperienced collaborators. Still, the question remains, how can we provide such detailed instructions without limiting the framework’s adaptability? Perhaps another round of the d.School design thinking process is needed to conceptualize new ways to deliver an informative, yet adaptable framework design.
Implications for Practice
Although I had never followed the d.School design thinking process before taking LRNT 524, I can see how it would benefit my day-to-day professional practice as an educator. First, I plan to document all aspects of my ID process, as it was throughout the design thinking challenge that I realized the power of transcribing interviews and ideation sessions. Michael, my design challenge partner, and I frequently referred to our interview recordings during the later stages of the design process to remind ourselves of our initial design intent: Design for the user and not the designer. Second, I will incorporate the empathy stage when analyzing DFW reports (D, F, withdrawal). Merely asking the question, “why are my students struggling” is not going to produce working solutions; instead, educators and designers must evaluate a flawed course design by assuming the learner’s perspective to reveal causation of performance-related issues (d.School, 2018). Lastly, I will share the d.School design thinking process with my colleagues to provide an efficient way to collaborate. Higher education instructors often spend countless hours searching for teaching and ID solutions (Kebritchi et al., 2017), so the proposition of an efficient ID methodology, such as the d.School design thinking process would be well received.
My Design Principles
The effects of an intensified and digitalized global market have led to unprecedented student diversification in higher education over the past decade (OECD, 2019). It is speculated that global mobility, a key contributor to globalization, will “double” (p.24) by 2030, suggesting an even steeper increase in student diversity than decades past (2019). Higher education is to accommodate this evolving student population by fostering “knowledge, skills, values and attitudes” (p.30) that encourage students to take action for the collective well-being and the global market (2019). Moreover, ID must appropriate multi-cultural interactions in all classroom settings, support cross-cultural differences such as languages and perspectives, and foster real-world skill development for students wishing to enter a globalized workforce (2019).
Use Design Models
Instructional design models provide roadmaps that systematize pedagogical design processes to achieve instructional goals (Dousay, 2017). Although not necessary for effective ID, those who incorporate ID models are likely to participate in more in-depth analysis and evaluation processes to uncover critical insight into learner needs, learning conditions, and revision (2017). Furthermore, ID models are often suited toward specific educational contexts or adapted to suit the user’s needs (Bates, 2015). For example, the CLE and 4C-ID models reportedly support problem-based learning (Goksu et al., 2017), so designers and educators looking to satisfy a particular learning need can look to ID models for technical direction. Furthermore, ID models like ADDIE, a foundational design process historically rooted in industry, can easily be adapted to fit unique educational requirements. One such adaptation, entitled “PADDIE,” takes ADDIE’s acronym “analysis, design, develop, implement, evaluate,” and adds “Planning or Preparation” at the beginning of the process (2015). The flexibility of specific ID models, such as ADDIE, allows designers to appropriate their design thinking efforts toward specific learning outcomes to produce a more usable and practical educational design.
Make it Learner-Centered
A critical aspect of modern ID is designing for the learner instead of the teacher or technology. While historically, many academic organizations created teacher-centered instructional content, an approach where the teacher presents the information and students passively absorb the content being taught (Lathan, n.d), organizations now embrace the learner-centered approach to ID. A learner-centered approach is one that is interactive and individualized (Zawacki-Richter and Anderson, 2014) and views the learner as “active and instrumental, making choices about when, how, and with whom to seek support” (p. 307). Moreover, the learner-centered design is a highly effective design principle that enhances student learning and performance outcomes (Dano-Hinosolango, 2014). Implementing learner-centered ID can be accomplished by embracing technology-based customizations and active learning activities; furthermore, it also helps to inform pedagogical adjustments by reviewing surveys and formative assessment data throughout a course.
Use Formative Assessment
One of the most powerful and often overlooked aspects of design is the role of formative assessment. Formative assessments are instruments used as part of formative evaluation, which “involves gathering information on adequacy and using this information as a basis for further development” (Seels and Richey, 1994). Although not typically a significant influencer of final grades, formative assessments are highly valuable as they enable both instructors and students to understand each student’s level of knowledge or skill at any given point in a course (Carnegie Melon University, n.d.). Accordingly, adjustments to the course content and teaching approach can be made, in real-time, following the learning outcomes. From a design lens, the implications are to allow time and resources to complete formative activities and allow sufficient flexibility for instructors to make real-time adjustments to focus on student needs of support before completing their summative assessments.
Implement Active Learning
Over the past decade, active learning has become an essential aspect of ID. Active learning encourages knowledge acquisition through the use of practical activities, and as a result, promotes the use of higher-level thinking skills such as analysis and synthesis (Queens, n.d.). Thought of as a critical principle of adult learning (Collins, 2004), active learning enhances student engagement as students actively engage with the course material (Chi and Wylie, 2014). The implications for designers are to not only design courses and programs that support active learning activities, such as demonstrations and hands-on applications (2004), but also to align the activities with the desired level of engagement with the course learning outcomes: Passive, active, constructive, or interactive (2014). Speculation suggests active learning to be a fixture of tomorrow’s ID, so understanding its principles and methodologies will likely serve considerable value in any ID toolbox.
Consider Backward Design
Critical to many ID endeavors is the backward design model (BDM), a design logic that focuses on the course learning objectives and the journey learners take to achieve them (McTighe & Wiggins, 2012). Crocker (2020) explains how the BDM can act as a useful link between content and essential learning questions or challenges. Furthermore, the backward design approach is an effective means to develop a curriculum that targets a specific function (Wiggins and McTighe, 1998). The implications from an instructional design perspective include having clear expectations of the learning requirements, articulating evidence of what learners are to demonstrate as they work towards the learning outcomes, and structuring the learning experience to enable students to achieve results (Georgetown University, n.d.). For many designers and educators, establishing the course learning outcomes is an appropriate first step in the ID process, so it is only logical to gain a solid understanding of the BDM to produce effective ID.
Carnegie Mellon University (no date). What is the difference between formative and summative assessment? Retrieved from: https://www.cmu.edu/teaching/assessment/basics/formative-summative.html
Carpenter, J., & MacKay, M. (2021). Empathetic informed design. Design Thinking Challenge [Unpublished assignment]. Royal Roads University. https://malat-webspace.royalroads.ca/rru0163/solution-summary-empathetic-informed-design/
Chi, M.T.H., and Wylie, R. (2014). The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Retrieved from: https://files.eric.ed.gov/fulltext/EJ1044018.pdf
Collins, J. (2004). Education techniques for lifelong learning: principles of adult learning. Radiographics, 24(5), 1483-9.
Crocker, W. (2020). Backward course design. Centre for Teaching and Learning. https://teaching.uwo.ca/curriculum/coursedesign/backward-design.html
Dano-Hinosolango, M.A., and Vedua-Dinagsao, A. (December 2014). The Impact of Learner-Centered Teaching on Students’ Learning Skills and Strategies. Retrieved from: https://infonomics-society.org/wp-content/uploads/ijcdse/published-papers/volume-5-2014/The-Impact-of-Learner-Centered-Teaching-on-Students-Learning-Skills-and-Strategies.pdf
Dougan, S. (2021, January 9). Activity 3 – Peer Feedback on Design Challenge – Team 4 (Michael and Jonathan) [Discussion post]. Royal Roads University Moodle Discussion Board. https://moodle.royalroads.ca/moodle/mod/forum/discuss.php?d=721506
Dousay. T. A. (2017). Chapter 22. Instructional Design Models. In R. West (Ed.), Foundations of Learning and Instructional Design Technology (1st ed.). Available at https://edtechbooks.org/lidtfoundations.
d.School. (2018). Design Thinking Bootleg. Standford d.School.
Georgetown University. (n.d.). Designing Backward. Retrieved from: https://commons.georgetown.edu/teaching/design/designing_backward/
Göksu, I., Özcan, K. V., Çakir, R., & Göktas, Y. (2017). Content Analysis of Research Trends in Instructional Design Models: 1999-2014. Journal of Learning Design, 10(2), 85-109.
Goldman, S. et al. (2012). Assessing d.learning: Capturing the journey of becoming a design thinker. In H. Plattner, C. Meinel & L. Leifer (eds). Design thinking research: Understanding innovation. (pp. 13-33). Berlin: Springer.
Kebritchi, M., Lipschuetz, A., & Santiague, L. (2017). Issues and challenges for teaching successful online courses in higher education: a literature review. Journal of Educational Technology Systems, 46(1), 4–29.
Koval, D. (2021, January 10). Activity 3 – Peer Feedback on Design Challenge – Team 4 (Michael and Jonathan) [Discussion post]. Royal Roads University Moodle Discussion Board. https://moodle.royalroads.ca/moodle/mod/forum/discuss.php?d=721506
Lathan, J. (n.d.) Complete Guide to Teacher-Centered vs. Student-Centered Learning. Retrieved from: https://onlinedegrees.sandiego.edu/teacher-centered-vs-student-centered-learning/#:~:text=In%20teacher%2Dcentered%20learning%20%E2%80%94%20the,receive%20the%20knowledge%20being%20presented.
Liljedahl P. (2014). The Affordances of Using Visibly Random Groups in a Mathematic. Retrieved from https://www.researchgate.net/publication/275953522_The_Affordances_of_Using_Visibly_Random_Groups_in_a_Mathematics_Classroom
Mattelmäki, T., Vaajakallio, K., & Koskinen, I. (2014). What happened to empathic design?. Design Issues, 30(1), 67-77.
McTighe, J. & Wiggins, G. (2012). Understanding by Design Framework. Retrieved from http://www.ascd.org/ASCD/pdf/siteASCD/publications/UbD_WhitePaper0312.pdf.
Norum, S. (2021, January 5). Activity 3 – Peer Feedback on Design Challenge – Team 4 (Michael and Jonathan) [Discussion post]. Royal Roads University Moodle Discussion Board. https://moodle.royalroads.ca/moodle/mod/forum/discuss.php?d=721506
Organisation for Economic Co-operation and Development. (2019), Trends Shaping Education 2019, OECD Publishing, Paris, https://doi.org/10.1787/trends_edu-2019-en
Piechnik, D. (2021, January 5). Activity 3 – Peer Feedback on Design Challenge – Team 4 (Michael and Jonathan) [Discussion post]. Royal Roads University Moodle Discussion Board. https://moodle.royalroads.ca/moodle/mod/forum/discuss.php?d=721506
Queens University. (n.d.). What is Active Learning? Retrieved from https://www.queensu.ca/teachingandlearning/modules/active/04_what_is_active_learning.html
Seels, B. B., and R. C. Richey. 1994. Instructional Technology: The Definition and Domains of the Field. Bloomington, IN: Association for Educational Communication and Technology
Schell, R., & Kaufman, D. (2009). Critical thinking in a collaborative online PBL tutorial. Journal of Educational Computing Research, 41(2), 155–170. https://doi.org/10.2190/EC.41.2.b
Schwartz, D.M., Lin, X., Brophy, S., & Bransford, J.D. (2009). Chapter 9: Toward the development of flexibly adaptive instructional designs. In Reigeluth & Carr-Chellman (Eds.), Instructional-design theories and models: volume III. Building a common knowledge base. New York: Routledge. Retrieved from https://books.google.ca/books?hl=en&lr=&id=OUq55prZIMEC&oi=fnd&pg=PT196&dq=adaptive+instructional+design+&ots=1MuxRDZujt&sig=UQKR0R5UPmS8B1YcZfuEIleO5U8#v=onepage&q=adaptive%20instructional%20design&f=false
Wiggins, G. & McTighe, J. (1998). What is Backward Design? Understanding by Design. https://valenciacollege.edu/faculty/development/courses-resources/documents/WhatisBackwardDesignWigginsMctighe.pdf
Zawacki-Richter, O. and Anderson, T. (2014) Online Distance Education: Towards a Research Agenda. AU Press. Canada