- A learner answered fast and perfect across 18 unrelated compliance concepts. Our knowledge-tracing model read high mastery. Then a few lines of deterministic code overruled it and certified zero of 18. That override is the whole product. 🧵
- The response-time distribution didn't look like genuine recall. It looked like an AI assistant breezing through the module. The completion record would say "trained." The learning never happened. This is the gap your LMS can't see.
- We built Attest to close it. It reads each learner's raw interaction stream, infers concept-level mastery with a real self-attentive knowledge-tracing transformer (SAKT), then hands the decision to code the model cannot override.
- The competence gate certifies a concept only if mastery is at least 0.747 AND there are at least 3 supporting interactions AND the response pattern is not anomalous. Miss any one and the concept routes to "needs proof," not certified.
- So the fast-perfect learner: apparent mastery high, evidence flagged, 0 of 18 certified, seat-time withheld pending verification. Not "gotcha, you cheated." Just: we certify what we can prove, and this we could not.
- Same gate, the senior-analyst learner, same module: 16 of 18 certified, 80.3% seat time saved (240 min down to 47.3). The gate isn't strict or lenient. It reads the evidence and separates proof from performance. That distinction is the point.
- The rule we kept coming back to: agents advise, code decides. An LLM tags the course into 18 concepts. The transformer infers mastery. But a deterministic sequencer and gate decide what ships. A better base model doesn't get a vote.
- The output isn't a green checkmark. It's a signed Competence Certificate (HMAC-SHA256): per-concept mastery, the evidence trail behind each estimate, model version and benchmark AUC, thresholds applied. What you hand an auditor instead of "Completed."
- For the record: the cohort and the LMS connector are synthetic and stubbed in this demo. The model is real, AUC 0.83 on our seeded synthetic KT benchmark. The durable claim isn't the AUC. It's that governance outside the model refuses to bluff.
- For the ML and L&D folks: when your model infers high mastery but the evidence pattern looks gamed, should the system trust the model or the anomaly check? Where do you draw that line? #KnowledgeTracing #AIGovernance #CorporateTraining #LearningEngineering #AMLcompliance
- We built the full pipeline as a runnable demo: the model, the gate, the signed certificate. If your team is wrestling with proving competence instead of logging completion, we'd like to hear how you're approaching it. https://veriprajna.com/demos/adaptive-learning-ai
Published on X · July 30, 2026
On social media
See this post on its original platform
In our archive