Visual Design and Engineering Laboratory

Carnegie Mellon University

An Image-Based, Trainable Symbol Recognizer for Hand-drawn Sketches


[Paper PDF]    

[Publisher Version]


We describe a trainable, hand-drawn symbol recognizer based on a multi-layer recognition scheme. Symbols are internally represented as binary templates. An ensemble of four different classifiers compares and ranks definition symbols according to their similarity to the unknown symbol. The scores of the individual classifiers are aggregated to produce a combined score for each definition. The definition with the best combined score is assigned to the unknown symbol. All four classifiers use template-matching techniques to compute similarity (and dissimilarity) between symbols. Ordinarily, template-matching is sensitive to rotation, and existing solutions for rotation invariance are too expensive for interactive performance. We have developed a fast technique that uses a polar coordinate representation to achieve rotational invariance. This technique is applied prior to the multi-classifier recognition step to determine the best alignment of the unknown with each definition. One advantage of this technique is that it filters out the bulk of unlikely definitions, thereby reducing the number of definitions the multi-classifier recognition step must consider.


Levent Burak Kara, Thomas F. Stahovich. (2005). An Image-Based, Trainable Symbol Recognizer for Hand-drawn Sketches. Computers & Graphics 29(4): 501-517.

 author = {Kara, Levent Burak and Stahovich, Thomas F.},
  title = {An image-based, trainable symbol recognizer for hand-drawn sketches},
 journal = {Comput. Graph.},
 issue_date = {August, 2005},
 volume = {29},
 number = {4},
 month = aug,
 year = {2005},
 issn = {0097-8493},
 pages = {501--517},
 numpages = {17},
 url = {},
 doi = {10.1016/j.cag.2005.05.004},
 acmid = {1652712},
 publisher = {Pergamon Press, Inc.},
 address = {Elmsford, NY, USA},
 keywords = {Hausdorff distance, Pattern recognition, Pen computing, Polar transform, Sketch understanding, Symbol recognition, Tanimoto coefficient, Yule coefficient},