Visual Design and Engineering Laboratory

Carnegie Mellon University

Recognizing Network-Like Hand-Drawn Sketches – A Convolutional Neural Network Approach


[Paper PDF]    

[Publisher Version]


Hand-drawn sketches are powerful cognitive devices for the efficient exploration, visualization and communication of emerging ideas in engineering design. It is desirable that CAD/CAE tools be able to recognize the back-of-the-envelope sketches and extract the intended engineering models. Yet this is a nontrivial task for freehand sketches. Here we present a novel, neural network-based approach designed for the recognition of network-like sketches. Our approach leverages a trainable, detector/recognizer and an autonomous procedure for the generation of training samples. Prior to deployment, a Convolutional Neural Network is trained on a few labeled prototypical sketches and learns the definitions of the visual objects. When deployed, the trained network scans the input sketch at different resolutions with a fixed-size sliding window, detects instances of defined symbols and outputs an engineering model. We demonstrate the effectiveness of the proposed approach in different engineering domains with different types of sketching inputs.


Luoting Fu, Levent Burak Kara. (2009). Recognizing Network-Like Hand-Drawn Sketches – A Convolutional Neural Network Approach. ASME International Design Engineering Technical Conferences/DAC. San Diego, September 2009.

 title={Recognizing network-like hand-drawn sketches: a convolutional neural network approach},
 author={Fu, Luoting and Kara, Levent Burak},
 booktitle = {ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference},
 pages = {671--681},
 year = {2009},
 publisher={American Society of Mechanical Engineers}