How to Run an Edtech Pilot That Gives You a Real Answer
Most edtech pilots end in 'teachers liked it.' How schools and tutoring centers scope a 30-day pilot, set success criteria up front, vet student data...
GrowWise · Team · Last updated
Short answer: Pick one narrow scope (one subject, one grade, one or two teachers), write down what success means before starting, vet the vendor's student-data practices before any data flows, and measure an outcome, not usage. At the end, decide yes, no, or extend with a named reason. A pilot without success criteria almost always ends in "the teachers seemed to like it."
Why do educators insist on pilots?
Because a pilot is the most trusted evidence educators have. In LXD Research's 2025 survey of 200+ educators across 37 states:
| Finding | Share of educators |
|---|---|
| Consider pilots critical to the decision | Over 80% |
| Say free trials are the most significant validation factor | 40% |
| Say research evidence is crucial for tool selection | 84% |
| Trust tools more when scientifically validated | 75% |
The same survey found the top barriers to adopting AI tools were ethical concerns (54%), data privacy risks (53%), and cost (35%). A good pilot addresses all three before a purchase decision.
Why do so many edtech tools get dropped after purchase?
Because they were bought on usage, not outcomes, and never fit the classroom. SchoolAI cites a Gallup/NewSchools survey finding that 65% of district administrators have discontinued a tool they previously adopted. Common reasons include added teacher workload, unclear instructional purpose, and one-time training that doesn't stick. A well-designed pilot surfaces these problems before the contract, not after.
Step 1: How narrow should the pilot scope be?
Narrow enough to measure, real enough to matter: one subject, one grade level, one or two teachers, one cohort. A narrow pilot lets you compare against a baseline and keeps teacher workload manageable. Expand only after the narrow version works.
Step 2: What should the success criteria be?
Write them down before day one, and make at least one of them an outcome:
| Type | Example criterion |
|---|---|
| Outcome | Share of diagnosed skill gaps closed within two weeks |
| Teacher time | Minutes per week spent on tracking before vs. during the pilot |
| Adoption | Teachers used it weekly without reminders |
| Parent or family | Parents could describe what their child is working on |
Usage alone ("logins per week") isn't a success criterion. Lightspeed Systems' vendor checklist makes the same point: ask for evidence the tool improves outcomes, not just usage statistics.
Step 3: What should you ask about student data before the pilot starts?
Ask before any student data flows. These questions come from EdWeek's reporting on how districts evaluate AI tools and Lightspeed's vendor checklist:
- Does the AI interact with students directly?
- Is student data used to train AI models, yours or a third party's? Can families opt out?
- Who controls the data, and who at the vendor can access it?
- Will the vendor sign your data privacy agreement? (In California, many schools use the California Student Data Privacy Agreement.)
- Can you export and delete all data if you don't continue?
- If the vendor shut down, what happens to your data?
In California, SOPIPA prohibits operators serving K-12 students from selling student data or using it for targeted advertising, and requires deletion on the school's request. Confirm how any vendor meets it. This is general information, not legal advice.
Step 4: What does a good pilot timeline look like?
A 30-day pilot is usually enough for one subject:
| Week | Focus |
|---|---|
| Week 0 | Data agreement signed; baseline recorded; teacher onboarding |
| Weeks 1-2 | Normal teaching with the tool in the loop; weekly 15-minute check-in |
| Weeks 3-4 | Measure against the success criteria; gather teacher and parent feedback |
| End | Decision meeting: yes, no, or extend, with a named reason |
Step 5: How do you make the final decision?
Use the criteria you wrote in Step 2, and require a named reason for any outcome. If the answer is "extend," say what the extension must show. If it's "no," record why; that's useful for the next evaluation.
Pilot scorecard template
| Criterion | Baseline | Result | Met? |
|---|---|---|---|
| Outcome: | |||
| Teacher time: | |||
| Adoption: | |||
| Family visibility: | |||
| Data agreement signed | - | ||
| Decision: Yes / No / Extend - Reason: |
Where Omnix360 fits - and where it doesn't
Where it fits: Omnix360 offers a scoped pacing pilot on one subject and one teacher team. It's teacher-facing only; the AI never interacts with students. It uses its built-in LMS today. LTI 1.3 and OneRoster connections are on the roadmap; see integrations. Student data: Teacher-facing only; review training, retention, export/deletion, and DPA/CSDPA in your pilot agreement; see AI student data privacy requirements and schools privacy notice.
Where it doesn't: Omnix360 is early. It's proven live at one school (GrowWise School, Dublin, CA: 50 students, 4 teachers), and independent outcome studies don't exist yet. A pilot is how you'd find out whether it works for you.
About Omnix360: Omnix360 is teacher-facing instructional infrastructure built by GrowWise. It reads students' wrong answers, names the misconception behind them, gives the teacher a reteach plan to approve, and shows whether the reteach worked. The AI never interacts with students. It runs on its own built-in LMS today. LTI 1.3 tool launches for connecting beside an existing LMS are on the roadmap; see integrations.
Sources
- LXD Research, educator survey on AI tool adoption (Oct 2025): https://lxdresearch.com/essa-evidence-educator-survey-2025
- EdWeek, "Schools Spend Billions on AI, But Struggle to Figure Out What's Worth Buying" (Aug 2026): https://www.edweek.org/technology/schools-spend-billions-on-ai-but-struggle-to-figure-out-whats-worth-buying/2026/08
- Lightspeed Systems, "10 Questions to Ask Your Edtech Vendor": https://www.lightspeedsystems.com/blog/10-questions-to-ask-your-edtech-vendor/
- SchoolAI, "Why Edtech Adoption Fails in Schools" (cites Gallup/NewSchools): https://schoolai.com/blog/why-edtech-adoption-fails-in-schools
- Edlink, "What Is SOPIPA?": https://ed.link/community/what-is-sopipa/
Related: How to run a tutoring technology pilot with success metrics · Student Progress Tracking Software: Percentage vs. Misconception · Is Omnix360 Right for Your School?
</user_query>
Frequently asked questions
- How long should an edtech pilot be?
- About 30 days for one subject and one grade is usually enough to measure an outcome, provided success criteria are set before the pilot starts.
- What should success criteria for an edtech pilot include?
- At least one learning outcome, plus teacher time, adoption, and family visibility. Usage statistics alone don't show impact.
- What data privacy questions should schools ask edtech vendors?
- Whether AI interacts with students, whether student data trains AI models, who can access data, whether they'll sign your privacy agreement, and how data is exported and deleted.
- Why do schools abandon edtech tools?
- Added teacher workload, unclear instructional purpose, weak training, and buying on usage rather than outcomes. One cited survey found 65% of district administrators had discontinued a tool.
Ready to prove pacing alignment in one program?
Start with the pilot playbook, then scope one program with our team, pricing comes after the scorecard.