Evidence brief · for committees

ProfilEd: why it works, and how to test it.

A short, printable brief. Every research claim is numbered and sourced at the end. Product claims describe what is live today, unless marked as being built.

Save as PDF / printRead the full version online

1. The problem

Generative AI can produce a polished essay, case analysis or code exercise in seconds, so written work no longer shows that a person understands anything. Completion badges show attendance, and multiple-choice quizzes show recognition. Neither shows reasoning.

2. The approach

ProfilEd is a spoken AI tutor. Learners explain, defend and reflect out loud, in short daily check-ins, study sessions, structured interviews and branching scenarios. It builds on what made one-to-one tutoring work1, and is designed against the failure mode of answer-giving AI3.

3. Six principles, the evidence, and the mechanism

PrincipleEvidenceIn ProfilEd
I. Tutor, don’t tellOne-to-one tutoring far outperforms classroom teaching1,2; answer-giving AI raised practice scores 48% but cut exam scores 17%3; a research-designed AI tutor more than doubled gains4.Never hands over the answer. Every mode makes the learner produce the thinking.
II. Explain it to know itPeople overrate their understanding until they try to explain5; self-explanation and teaching deepen learning6,7; effortful recall beats re-reading8,9.Spoken check-ins; Feynman teach-back; concepts named but not explained are recorded as gaps.
III. A worthy opponentGroups and AI both drift to agreement nobody believes10,11; debate between AI agents improves reasoning12; lectures carry 1.5× the failure rate of active learning13.Separate roles: Professor AI questions, Devil’s Advocate argues the other side, ALICE hosts.
IV. Fix the rootMastery learning: average effect size about 0.5214; dependency-network planning (PERT)15.Curriculum Graph of dependencies; a seven-step mastery chain that watching or reading can never skip. Automatic prerequisite tracing is being built.
V. Change the beliefDouble-loop learning16,17; 15 minutes of daily reflection: +22.8% performance in ten days18.Daily spoken reflection; beliefs, confidence and avoided patterns tracked across weeks.
VI. Earn the right to speakOver a third of feedback interventions make performance worse19; Grice’s maxims20.Confidence rules on every output (e.g. a pattern needs 3+ sessions); a quality gate that sends nothing rather than something generic.

4. The evidence it produces

  • Spoken and live. Each follow-up is built from the learner’s previous answer, so rehearsed or outsourced answers tend to fall apart within two or three questions.
  • How it is said. In interviews: speaking pace, pauses before answering, filler words and tone. In scenarios: hesitation before each decision21.
  • Built over time. Confidence-gated conclusions across many sessions, not a single sitting.
  • Reviewable. Transcripts; a proof record for every concept proved (unique ID, date, score, hash); CSV export of session data to support direct-assessment evidence for assurance of learning, such as AACSB Standard 522.

5. The pilot

  1. Week 0Success measures agreed with you
  2. Week 4Joint review
  3. Week 8Data, CSV export and written report

Eight weeks, a cohort of at least 15 learners, with a cohort dashboard. Example measures: blind-rated oral explanations before and after, the trend in concepts learners can’t yet explain, depth of reflection over time, completion, and faculty review of transcripts. If the numbers don’t move, you’ll see that too.

6. Honest limits

  • ProfilEd has not yet been tested in its own published randomized trial. The methods it is built on have been.
  • We are just launching, so there are no usage figures or testimonials yet.
  • The AI can be wrong. That is why it is built to stay quiet when it isn’t sure.
  • Being built: automatic tracing to missing prerequisites; tracking whether each teach-back attempt passed; linking reminders to return visits.
  • It is harder to fake than written work, not impossible to cheat.

Talk to usinstitutions@thoughtjumper.com · Book 20 minutes with the founder · profiled.thoughtjumper.com/why-it-works

Sources (22)

Every figure and quote on this page, with a link to the original where one exists.
  1. Bloom, B. S. (1984). The 2 Sigma Problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. Source ↗
  2. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. Source ↗
  3. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS, 122(26). Source ↗
  4. Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, 17458. Source ↗
  5. Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: an illusion of explanatory depth. Cognitive Science, 26(5), 521–562. Source ↗
  6. Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477. Source ↗
  7. Fiorella, L., & Mayer, R. E. (2013). The relative benefits of learning by teaching and teaching expectancy. Contemporary Educational Psychology, 38(4), 281–288. Source ↗
  8. Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. Source ↗
  9. Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about Knowing, 185–205. MIT Press.
  10. Harvey, J. B. (1974). The Abilene Paradox: The management of agreement. Organizational Dynamics, 3(1), 63–80. Source ↗
  11. Sharma, M., et al. (2023). Towards understanding sycophancy in language models. arXiv:2310.13548. Source ↗
  12. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023). Improving factuality and reasoning in language models through multiagent debate. arXiv:2305.14325. Source ↗
  13. Freeman, S., et al. (2014). Active learning increases student performance in science, engineering, and mathematics. PNAS, 111(23), 8410–8415. Source ↗
  14. Kulik, C.-L. C., Kulik, J. A., & Bangert-Drowns, R. L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60(2), 265–299. Source ↗
  15. Malcolm, D. G., Roseboom, J. H., Clark, C. E., & Fazar, W. (1959). Application of a technique for research and development program evaluation. Operations Research, 7(5), 646–669. Source ↗
  16. Argyris, C. (1977). Double loop learning in organizations. Harvard Business Review, September. Source ↗
  17. Argyris, C., & Schön, D. A. (1978). Organizational Learning: A Theory of Action Perspective. Addison-Wesley.
  18. Di Stefano, G., Gino, F., Pisano, G. P., & Staats, B. R. (2016). Making experience count: The role of reflection in individual learning. Harvard Business School Working Paper 14-093. Source ↗
  19. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance. Psychological Bulletin, 119(2), 254–284. Source ↗
  20. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and Semantics, Vol. 3: Speech Acts, 41–58. Academic Press.
  21. Donders, F. C. (1868/1969). On the speed of mental processes (W. G. Koster, Trans.). Acta Psychologica, 30, 412–431. Source ↗
  22. AACSB International (2020). 2020 Guiding Principles and Standards for Business Accreditation, Standard 5: Assurance of Learning. Source ↗

Five minutes. Out loud. See what it finds.

Just your first name. Your first check-in is free, and you’ll leave with something to do tomorrow.

Start a free check-in

No account needed · about five minutes

Do I have to talk out loud?
For the daily check-in, yes: it’s a voice conversation and you don’t type. Interviews can be voice or fast text, and study sessions can be voice or chat.
Can I do it on my phone?
Yes. The check-in runs in the browser on your phone, tablet, or computer.
Do I need an account?
Not for your first check-in. A free account keeps your conversation and picks up where you left off.
Is it private?
The check-in is confidential, and nothing is shared unless you share your footprint link. For exactly what’s stored, read the Privacy Policy.
What does it cost?
The first check-in is free, with no account or card, and new accounts get starter gems to try deeper things. ProfilEd Pro is $19 a month for unlimited use. Full details are on the pricing page.