Visual Design and Engineering Laboratory

Carnegie Mellon University

The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems


[Paper PDF]    

[Publisher Version]


The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems Engineering education includes significant attention to problem solving skills, with students gradually confronting problems of increasing complexity. Even within any single fundamental engineering science course, which addresses a limited set of concepts,students must learn to solve problems that require coordinating and organizing multiple parts. To solve, the student needs to decompose a problem into inter-related sub-problems, define variables of different types, carry out analyses of sub-problems, and finally combine and interpret the results. Problems typically have multiple pathways to the correct answers; students should be granted considerable latitude in constructing apathway.In general, formative assessment, that is feedback to students enabling them to revise and refine their understanding and actions, can significantly promote learning. Formative assessment can be provided through in-class activities, and also through computer-based instruction outside of the classroom. This paper addresses the issue of providing formative feedback for students confronting complex problems that involve significant latitude in decomposition and construction of solutions. Traditionally, students solve complex problems as part of written homework assignments that are hand graded. In such circumstances, offering effective formative assessment is exceptionally challenging,requiring careful attention to solution details and rapid, rather than weeklong, turnaround.Furthermore, since later work builds upon earlier work, grading of a completed solution often involves judging off-path steps that may be irrelevant to the intended learning ormay build upon prior incorrect work.The research questions this paper seeks to answer are: (1) Under what circumstances isautomated, formative assessment on complex problem solving possible, (2) What metricsallow us to judge whether the feedback indeed promotes learning, and (3) On what basiscan we target ongoing improvements to the formative assessment offered?We address these questions in the context of a test case: a tutor to help students in statics learning to solve truss problems. Trusses exemplify complex problems: students select multiple portions of the truss, draw free body diagrams, write down appropriate equilibrium equations for each diagram, organize the solving of equations, and interpret results physically in terms of the original truss. Mastery requires clarity, systematic organization, as well as conceptual and mathematical competence. Building on a catalog of typical errors, a computer interface was created where correct steps and typical errorsin solving truss problems can be executed with wide latitude to pursue solution paths.The tutor reflects careful trade-offs between granting latitude to the solver and retaining ability to interpret work. The student can solve unimpeded until errors are made that can interfere with future solving steps; feedback is then offered which enables students to correct their errors. Through task analysis, steps hypothesized to involve the same components of knowledge have been grouped, and data is collected on the fly of attempts to apply the different knowledge components. Statistical models are used to determine whether errors in using different knowledge components decrease in frequency with practice. The determined learning rates give insights into whether feedback is effective and can inform future improvements in the tutor.


Paul S. Steif, Luoting Fu, Levent Burak Kara. (2014). The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems. American Society for Engineering Education Annual Conference. June 15-18, 2014. Indianapolis, Indiana.

 author = {Steif, Paul S., Fu, Luoting, and Kara, Levent Burak},
 title = {The Potential for Computer Tutors to Assist Students Learning to Solve Complex Problems},
 booktitle = {American Society for Engineering Education Annual Conference},
 year = {2014},
 location = {Indianapolis, IN, USA},