Multimodal Learning Analytics

Embodied cognitive science conceptualizes bodily activity as part of cognitive dynamics. I work with multimodal data (including movement and eye tracking, speech, video, electrodermal activity, sensor and touchscreen data) to empirically test this view. I draw upon nonlinear methods like Recurrence Quantification Analysis, new to education research, that can capture the complex dynamics of learning data. I apply these methods to study changing coordinations of bodily and ecological resources as learning, such as the emergence of new ways of looking and acting preceding conceptual breakthroughs in problem-solving.

A series of data visualizations showing children's rocking movements as they collaborated on an activity. In a middle row, we see a set of graphs that becomes more regular for one child, and somewhat more regular for the other. Above and below this row, we see recurrence plots for each child's movements, visualizing patterns of repetition in their actions. For the child whose rocking becomes periodic, the recurrence plots go from spotty patterns towards consistent diagonal stripes. For the child whose rocking remains somewhat irregular, we see a great variety of forms on her recurrence plots including one with a large square and many inconsistent dot-like patterns and shorter lines.

Relevant Publications

Abrahamson, D., Tancredi S., Xiao, Z., Weiss, M., Potęga vel Żabik, K., & Dimmel, J. (under review). It’s our move: Mathematical perception emerges for coordinating joint action.

Zhang, F. & Tancredi, S. (2025). Predicting electrodermal activity from conceptual and physical activity in an embodied learning environment. Proceedings of the Cognitive Science Society 2025 (Cogsci 2025) (Vol. “Posters”). San Francisco, CA.

Zhang & Tancredi 2025 Download

Sar-Israel, M., Zhang, F. E., Liu, Y., & Tancredi, S. (2024). Tracking sensory regulation during embodied learning with electrodermal activity. Proceedings of the 18th International Conference of the Learning Sciences – ICLS 2024 (Vol. “Short papers”). International Society for the Learning Sciences (ISLS), Buffalo, NY. https://doi.org/10.22318/icls2024.908961

Sar-Israel, Zhang, Liu, & Tancredi 2024 ISLS Download

Abdu, R., Tancredi, S., Abrahamson, D., & Balasubramaniam, R. (2023). A complex systems outlook on hand-eye coordination in mathematical learning. In M. Schindler, A. Shvarts, & A. Lilienthal. (Eds.), Eye-tracking research in mathematics education [Special issue]. Educational Studies in Mathematics.

abdu-et-al.-2023 Download

Tancredi, S., Abdu, R., Balasubramaniam, R., & Abrahamson, D. (2022). Intermodality in multimodal learning analytics for cognitive theory development: A case from embodied design for mathematics learning. In M. Giannakos, D. Spikol, D. Di Mitri, K. Sharma, X. Ochoa, & R. Hammad (Eds.), Multimodal learning analytics. Springer.

tancredi_abdu_balasubramaniam_abrahamson-2022-intermodality-in-multimodal-learning-analytics-for-cognitive-theory-development-a-case-from-embodied-design-for-mathematics-learning_mmla_ha Download

Tancredi, S., Abdu, R., Abrahamson, D., & Balasubramaniam, R. (2021, 2021/04/03/). Modeling nonlinear dynamics of fluency development in an embodied-design mathematics learning environment with Recurrence Quantification Analysis. International Journal of Child-Computer Interaction, 100297. https://doi.org/10.1016/j.ijcci.2021.100297

tancredi-abdu-abrahamson-balasubramaniam-ijcci-2021 Download