Schools rethink how to pilot edtech programs

How do schools decide which educational technology products to use in the classroom?

Administrators and school CTOs are often left scratching their heads at the abundance of trendy new gadgets and applications on the market, but one nonprofit is trying to help research and share schools’ best pilot-to-purchase practices.

Employees from Digital Promise, a bipartisan education advocacy group in Washington, D.C., are in the process of visiting and observing six different school districts around the country to study their tech pilot programs – looking at what’s working and what can be done better from an academic and industry perspective.

“How did schools structure the pilot? Why did they do it that way? What did they learn, and how does that information drive their procurement decision-making?” Phil Martin, who oversees education marketplace initiatives for the nonprofit, said about what they’re trying to learn. “We aim to share with school districts best practices and protocols that they could use to do pilots better.”

The result of Digital Promise’s months of research and fieldwork will be unveiled in October, Martin said.

The project was prompted by a study conducted by Digital Promise and Johns Hopkins University that showed a significant gap in how CTOs and vendors view the purchasing process – about half of CTOs and school administrators said they were satisfied with the edtech procurement process, while just 6 percent of providers said they were satisfied.

But technology directors also question the credibility of the companies, many of which are simply pushing their products into schools without thinking about standards or metrics by which to gauge student success.

“If you’re going to do a pilot with rigor, then you’ve got to be thoughtful about making sure teachers are trained and infrastructure is in place so teachers and students have access to the program,” Martin said. “Hopefully this leads to better decision-making, and hopefully we see companies responding to the way [schools] do business that’s better for everyone.”

West Ada School District in Meridian, Idaho – one of the districts Digital Promise partnered with for the project – is currently trying out several digital programs, said Eian Harm, research and special projects coordinator for the district.

Harm, who calls technology the “frontier” for modern-day learning, is tasked with thinking creatively when it comes to innovative programs and tools with real-world applications that the 37,000 kids in the district can use.

“I’m looking at how students need to build skills to make it in the real world,” Harm said in an interview with StateScoop. “Part of my incentive is, are we using the tools as we would use them in a company or a business?”

For the last three years, Harm and other colleagues have been looking at how different online programs affect student outcomes. Various schools in the district are currently testing math programs like ALEKS, Sumdog, Think Through Math and DreamBox Learning.

One class, which serves as a sort of control group, does not use any of these programs, so Harm can test whether students using the technology perform better on exams.

He said he is now able to see that the time a student spends on the products isn’t a good indicator of whether the child understands the material – it’s the algorithms, which are constantly collecting student data, that determine whether the kid has mastered a particular subject that counts.

“The program has to have some sort of metric to say, that kid got this, let’s move on,” Harm said. “It’s really important that everything is in alignment and that kids that master five points on the program grow five points on the test.”

Now, he added, a fundamental shift is starting to happen.

“We are starting to send a company a template of how we measure mastery, which is Common Core-based,” he said, referring to new standards adopted by several states. “It’s a different approach. If these companies want to have access to our funding and students, we need them to figure out how to measure mastery.”

Though technology purchasing opens doors for new ways of learning and teaching – Harm said the school has moved away from a “one size fits all” approach to students – he added a caveat. Teachers still make all the difference.

“I am searching for the silver bullet technology-wise,” he said. “But really, so far, what I’m seeing is that good teachers – with or without technology – get really good outcomes, and we want to empower those teachers.”