In response to Michelle’s question on how do we evaluate the success of a change project in education:
I would agree that evaluating the success of a change project can be difficult.
Al-Haddad & Kotnour (2015) say that a change project “is deemed successful if it is completed within the predetermined objectives (i.e. completed within budget, within schedule, conforming to the customer requirements and satisfies the main stakeholders)” (p.251).
In this situation for education, I would say the stakeholders include:
– students
– other institutions (transferring into/out of)
– coop and external project contributors
– employers
– other instructors within the program (pre/post requisites)
– social events, conferences, competitions
– ACM Learning Outcomes (Association of Computing Machinery)
Some of the things to consider are:
# of Coop positions filled
– types of positions (jobs related to field)
Potential Downsides:
– varies by job market (this might always be reliable if there is a downturn in the market)
Surveys of Students (we survey our graduates about the program)
– what works
– what doesn’t
Potential Downsides:
– only vocal students answer surveys
– many surveys responses are negative
– students sometimes ask for things that aren’t ultimately what they need
– complaints – attempt to minimize, but you’ll never completely eliminate
Students getting jobs when the leave the program
– what kind of jobs
– was it the job they wanted to get?
– what kinds of employers
Potential Downside:
– again a downturn in the job market can give misleading information here
What Employers are saying about our graduates?
– what skills are the good at
– what skills are they lacking?
– How are students doing on interviews?
Grades
– are students doing better this year vs last?
Potential Downsides:
– good students vs bad students (not all years have the same quality of students)
– varies by instructor, slight variability in content delivered, textbooks, experience, how new/busy instructors were
How easily students can do follow on tasks
– how many questions you get (lots vs little)
– what kinds of tasks students are able to achieve as they progress through the course and other courses
Potential Downside:
– It’s not always possible to watch every student to see where all their struggles are.
Real-Life Projects
– Do students excel or struggle
– what kinds of questions / challenges do they ask or struggle with
– Do they produce high quality projects
Are students able to compete and communicate professionally
– are they successful at completing external hack-a-thons
– how are they able to communicate at conferences to other employers and businesses
Transferability of students into or out of our programs
– follow up from students asking them their competency in other programs
– comparing student experiences as they come in from other programs
Do students meet the ACM (Association for Computing Machinery) Learning Outcomes
– For the established standards for what is expected from computing students,
are our students able to achieve those goals?
References
Al-Haddad, S., & Kotnour, T. (2015). Integrating the organizational change literature: A model for successful change. Journal of organizational change management, 28(2), 234-262. https://doi.org/10.1108/JOCM-11-2013-0215
Featured Image: “Success” by Adrien Coquet, FR from the Noun Project
Recent Comments