eLibrary
  • Home
  • Sources
  • Search
  • About

Retrieval-Conditioned Counterfactual Consistency for Classroom Discourse Analysis

Emily Garcia, Hazel Xu
Journal of Computing and Electronic Information Management, (2026), Vol.23, No.1, pp.50-57
Published: October 28, 2026
DOI: 10.54097/nf8t5j05
PDF: Download Full Text PDF
Abstract

Classroom discourse analysis depends on detecting pedagogically meaningful reasoning from turns whose relevant context may occur far earlier. This study evaluates a small-footprint retrieval-conditioned classifier augmented with an explicit counterfactual consistency penalty. We manually curated 136 publicly displayed, human-labeled student-response records from the TalkMoves corpus (47 labeled as providing evidence or reasoning). A controlled benchmark inserts each record’s original paired context among 8, 32, or 64 distractor turns sampled from the same public excerpt pool; these reconstructed histories are synthetic, whereas the classification labels are authentic. A character n-gram TF-IDF selector supplies one context to a word n-gram logistic classifier. We adapt the symmetric Kullback–Leibler consistency objective and Counterfactual Flipping Rate audit of the neuro-symbolic pedagogical alignment literature to a binary, CPU-feasible estimator. Across five grouped random splits, the 32-candidate setting attains Macro-F1 0.753 with consistency regularization and 0.753 without it, while mean flipping falls from 10.38% to 3.07%. Retrieval recall at one declines as context grows, and the utterance-only baseline remains competitive. Thus, the supported contribution is improved stability under specified orthographic perturbations, together with a reproducible measurement of retrieval failures—not a claim of superior full-lesson performance or demographic fairness.

Keywords: Classroom discourse, Retrieval-augmented classification, Long-history stress testing, Counterfactual consistency, Educational NLP
APA Citation: Emily Garcia, Hazel Xu (2026). Retrieval-Conditioned Counterfactual Consistency for Classroom Discourse Analysis. Journal of Computing and Electronic Information Management, 23(1), 50-57. https://doi.org/10.54097/nf8t5j05

References

  1. Suresh, A., Jacobs, J., Harty, C., Perkoff, M., Martin, J. H., & Sumner, T. (2022). The TalkMoves dataset: K–12 mathematics lesson transcripts annotated for teacher and student discursive moves. In Proceedings of the 13th Language Resources and Evaluation Conference (LREC) (pp. 4654-4662). European Language Resources Association. https://doi.org/10.63317/2azjpibjyj5e
  2. Teng, D. (2025). TRIAD: Deadline- and reliability-aware scheduling for post-earthquake structural assessment workflows. World Journal of Engineering Research, 3(6), 55-66.
  3. Suresh, A., Sumner, T., Jacobs, J., Foland, B., & Ward, W. (2019). Automating analysis and feedback to improve mathematics teachers' classroom discourse. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1), 9721-9728. https://doi.org/10.1609/aaai.v33i01.33019721
  4. Jacobs, J., Scornavacco, K., Harty, C., Suresh, A., Lai, V., & Sumner, T. (2022). Promoting rich discussions in mathematics classrooms: Using personalized, automated feedback to support reflection and instructional change. Teaching and Teacher Education, 112, Article 103631. https://doi.org/10.1016/j.tate.2022.103631
  5. Demszky, D., Liu, J., Mancenido, Z., Cohen, J., Hill, H., Jurafsky, D., & Hashimoto, T. (2021). Measuring conversational uptake: A case study on student-teacher interactions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP) (pp. 1638-1653). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.acl-long.130
  6. Demszky, D., & Hill, H. (2023). The NCTE transcripts: A dataset of elementary math classroom transcripts. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA) (pp. 528-538). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.bea-1.44
  7. Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26(4), 430-511. https://doi.org/10.1080/07370000802177235
  8. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157-173. https://doi.org/10.1162/tacl_a_00638
  9. Beltagy, I., Peters, M. E., & Cohan, A. (2020). Longformer: The long-document transformer. arXiv, arXiv:2004.05150. https://doi.org/10.48550/arXiv.2004.05150
  10. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459-9474. https://doi.org/10.48550/arXiv.2005.11401
  11. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 6769-6781). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.550
  12. Zhou, F. (2026). Moment-constrained side-force allocation for wind-assisted merchant ships: Hydrodynamic performance of retractable anti-leeway fins and flap rudders. Innovation and Technology Studies, 3(1), 27-41.
  13. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 3982-3992). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1410
  14. Wang, J., Ge, Y., Liu, J., & Jiang, B. (2024). Early warning of liquidity stress in U.S. banks with seasonally aligned frailty hazards and delayed-feedback conformal alarms. Journal of Trends in Finance and Economics, 1(1), 85-100.
  15. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 4171-4186). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
  16. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations (ICLR). https://doi.org/10.48550/arXiv.2106.09685
  17. Fang, Q., & Liu, W. (2026). Neuro-symbolic pedagogical alignment (NSPA) for long-horizon classroom discourse analysis: Mitigating dialect bias via counterfactual preference optimization. Computers and Education: Artificial Intelligence, 11, Article 100664. https://doi.org/10.1016/j.caeai.2026.100664
  18. Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36. https://doi.org/10.52202/075280-2338
  19. Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences of the United States of America, 117(14), 7684-7689. https://doi.org/10.1073/pnas.1915768117
  20. Yue, X., & Zhu, R. (2024). An explainable hybrid machine learning framework for cash-flow-conditioned multi-horizon liquidity risk early warning: An SME-oriented benchmark study. World Journal of Management Science, 2(1), 63-72.
  21. Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32, 1052-1092. https://doi.org/10.1007/s40593-021-00285-9
  22. Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024). Bias and fairness in large language models: A survey. Computational Linguistics, 50(3), 1097-1179. https://doi.org/10.1162/coli_a_00524
  23. Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2024). Can automated feedback improve teachers' uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course. Educational Evaluation and Policy Analysis, 46(3), 483-505. https://doi.org/10.3102/01623737231169270
  24. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825-2830.
  25. Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895-1923. https://doi.org/10.1162/089976698300017197
  26. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
  27. McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157. https://doi.org/10.1007/BF02295996
  28. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*) (pp. 220-229). ACM. https://doi.org/10.1145/3287560.3287596
  29. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86-92. https://doi.org/10.1145/3458723
  30. Zhao, W., Chen, T., Wang, B., & Shen, Z. (2026). PIPE-RAG: An execution-guided retrieval and reuse framework for reliable machine learning pipeline code generation. WEP Transactions on Computer Science and Intelligent Systems Research, 13, 286-297.
  31. Chen, J., Han, X., Zhang, X., & Yang, Y. (2023). Exact concept-level explanations for early detection of financial misstatements. Eurasia Journal of Science and Technology, 5(2), 34-48.
© 2026 eLibrary · Stand on the shoulders of giants