I think I know what the problem at my work is. The calendar says it is 2019 but according to the Weller piece we have not moved past the year 2000. For this post we are supposed to pick one lesson and even Weller states a lesson should resonate with most practitioners. How about all eighteen of them. No let me correct that because there is some technology, we have not even discovered yet. So, they cannot be even be a problem yet-can it?
My section is about to embark into the world of Learning Analytics. Weller describes the notion that the “edtech field needs to avoid the mistakes of data capitalism: it should embed learner agency and ethics in the use of data, and it should deploy data sparingly”. The ethics of monitoring keystrokes of students to see what resource is being visited can quickly lead to a slippery slope of privacy invasion. Specific to my situation is that my colleagues and I are proposing measuring student performance in a simulated environment. Digitizing the data (yes -unbelievably we are still using paper and clipboard) will allow analytics to detect trends in a single student, allows trends across cohorts in the same year and allow cyclic trends over a number of years. We also are contemplating utilizing tombstone data (age, gender, education, language) in conjunction with performance to detect what recruiting should seek in the next person signing up. This is definitely “reduc[ing] students to data and that ownership over the data becomes a commodity in itself” (Weller, 2018, p44). However, executed with the right intent and it might lead to the detection of discreet signals where changes in the academic stages preceding the simulation phase may lead to greater success rates. It might also allow predictability as to how long a phase of training should be and reduce the cost of overall training or get them to their next assignment quicker.
All to say it is an exciting time. I remember opening up the Globe and Mail newspaper in 1983 and the sports section had a new category* for the NHL player. Along with listing goals and assists it had what was called a player’s plus-minus (+/-). It revolutionized hockey data and now every second of every movement is captured. I remember thinking that even back then with something that simple we worried about how data was being captured and what allowances were being made for the power-play or the shorthanded situation or the fact that ice time varied between forward and defenceman.
*it has actually been recorded since 1968 but no one was rewarded or punished for it and it was not published
Weller, M. (2018). Twenty years of EdTech. EDUCAUSE Review, 53(4).
September 15, 2019 at 9:36 am
Great post! I can understand the hesitation from Weller about learner analytics and potentially dealing with issues with data ownership. News flash from 2019, that’s something the government has been actively debating about how user data is utilized by social media giants (i.e. Facebook and Google) and potentially getting involved in regulating that! There’s a lot of value in looking at data analytics for the purpose of improving learner experience and to support their transfer of learning. As someone who’s developing more content for online use, I often rely on data and feedback from users to continuously improve their experience.
September 15, 2019 at 9:55 am
Thank you for this thoughtful post, Arv. It sounds like the Weller reading comes at an opportune time. I’m really curious though, given the hockey analogy you describe (and, I should note, as an aside, that hockey has grown on Martin over the last couple of years) are you at all concerned about every moment of a learner’s interaction with the simulations being captured? What I am having a bit of trouble with is the notion of the “right intent.” What does that look like? And if we’re talking about huge amounts of data (e.g., tracking all actions within the simulation), how do you see building in the right intent into the algorithms that will be used to work with that data?
September 15, 2019 at 9:49 pm
George,
The length of the post did not do justice to describing the situation. We have a Marine Simulator that we use for development and assessment of Junior Naval Officers. The background of these Officers is that most of then have completed an undergraduate degree from a university if Canada in a broad range of disciplines. The enter into our school system for approximately on year where they undertake the study of Marine Sciences. The studies are classroom based, simulator based and in situ on training platforms at sea. In one cohort there are usually 20 students and during the simulator phase each student has a one hour sim run in one of our eight simulators. So in one day you each student has one data point and the school has 20 data points. Very quickly in one week you have 100 data points and in one month you have upwards of 400 data points. The school has been in operation since 1998. Big math big data. The problem right now the data is captured on assessment sheets by instructors who use a rubric to score the student. What is measured? Some data is binary -did the student do this y-n? Some data is scalar ie on a scale of of 1 to 4 –again each stage with a rigorous description. I have in the midst of seeking a digital solution so as to crack the analytics. What I meant with by ‘right intent’ was asking myself why are we doing this? Leigh has commented that Admin is always looking to save money and she is right in our case we would be able to graduate students faster if and only if we could ask the data at what stage were we not seeing any improvement in ability. Should the current three week-phase in the simulator be reduced to one week or 10 days? Could the data provide us insight where the training in the classroom could be improved? Could the data determine that English as a native tongue and a male between the age of 21-24 is the ideal candidate. So now armed with this do you recruit only that demographic? I know I need to flush out the analytics..again like with some other technology some people just want it because everyone else has it. We need to be more reflective on how best to gather, analyze and use the data.
September 15, 2019 at 6:51 pm
Interesting post, Arv. As a K-12 educator and facilitator of a digital learning platform (with a direct link for teacher access to Learning Analytics – data collected from specific student activities) — I find this all very useful and exciting!
In your very different educational setting, you shared how learning analytics in a simulated environment, “might also allow predictability as to how long a phase of training should be and reduce the cost of overall training or get them to their next assignment quicker.” This use of analytics serves student and educator needs, as well as admin’s (re: cost). With the exception of cost, learning analytics do the same thing in my work with elementary students. As you have identified, it is the slippery slope of privacy invasion that remains a challenging area where we still struggle for balance, and understanding followed by the need to then educate concerned parents about benefits and risks.
Are your post-secondary students overly concerned about the data being collected on them, albeit for Learning Analytics?
September 15, 2019 at 9:24 pm
Leigh,
The concern over the collection of data is complex. Firstly, during the development phase of the training I don’t think that there would be a concern. The concern will probably be during the assessment phase where a pass/fail is determined. As we develop an ask for industry to help in the analytics we are not sure where the data will be analyzed and stored. The COTS solution will undoubtedly want to house the data in a cloud or on a server on their footprint. This then gets complex from my school house POV (read military) to address. We go through many ‘what if’ scenarios of data breach etc. The next thought was to have the analytics and storage done on our footprint. The problem there is that the simulator is in contract with the military and there would be many ‘hoops’ to jump through to integrate the simulator to conduct the analytics. An interesting road ahead…