Computer-Aided Design (2011), doi:10.1016/j.cad.2010.12.011.

Abstract:
We present a computational recognition approach to convert network-like, image-based engineering diagrams into engineering models with which computations of interests, such as CAD modeling, simulation, information retrieval and semantic-aware editing, are enabled. The proposed approach is designed to work on diagrams produced using computer-aided drawing tools or hand sketches, and does not rely on temporal information for recognition. Our approach leverages a Convolutional Neural Network (CNN) as a trainable engineering symbol recognizer. The CNN is capable of learning the visual features of the defined symbol categories from a few user-supplied prototypical diagrams and a set of synthetically generated training samples. When deployed, the trained CNN is applied either to the entire input diagram using a multi-scale sliding window or, where applicable, to each isolated pixel cluster obtained through Connected Component Analysis (CCA). Then the connectivity between the detected symbols are analyzed to obtain an attributed graph representing the engineering model conveyed by the diagram. We evaluate the performance of the approach with benchmark datasets and demonstrate its utility in different application scenarios, including the construction and simulation of control system or mechanical vibratory system models from hand-sketched or camera-captured images, content-based image retrieval for resonant circuits and sematic-aware image editing for floor plans.

BibTeX:
@article{fu_kara:2011:sketch_recog,
title={From engineering diagrams to engineering models: Visual recognition and applications},
author={Luoting Fu and Kara, L.B.},
journal={Computer-Aided Design},
volume={43},
number={3},
pages={278--292},
year={2011},
publisher={Elsevier}
}


PDF: CAD Elsevier

ASME Journal of Computing and Information Science in Engineering, Volume 12, Issue 3.

Abstract: In product design, designers often create a multitude of concept sketches as part of the ideation and exploration process. Transforming such sketches to 3D digital models requires special expertise due to a lack of intuitive Computer Aided Design (CAD) tools suitable for rapid modeling. Recent advances in sketch-based user interfaces and immersive environments have introduced novel curve design methods that facilitate the transformation of such sketches into 3D digital models. However, rapid surfacing of such data remains an open challenge. Based on the observation that a sparse network of curves is reasonably sufficient to convey the intended geometric shape, we propose a new method for creating approximate surfaces on curve clouds. A notable property of our method is that it relieves many topological and geometric restrictions of 3D conventional networks such as the curves do not need to be connected to one another or gently drawn. Our method calculates a 3D guidance vector field in the space that the curve cloud appears. This guidance field helps drive a deformable closed surface onto the curves. During this deformation, surface smoothness is controlled through a set of surface smoothing and subdivision operations. The resulting surface can be further beautified by the user using selective surface modification and fairing operations. We demonstrate the effectiveness of our approach on several case examples. Our studies have shown that the proposed technique can be particularly useful for rapid visualization.

BibTeX:

@article{arisoy2012free,
title = "Free Form Surface Skinning of 3D Curve Clouds for Conceptual Shape Design",
journal = "Journal of Computing and Information Science in Engineering",
volume = "12",
number = "3",
pages = "031005",
year = "2012",
note = "",
issn = "0010-4485",
doi = "http://dx.doi.org/10.1115/1.4007152",
url = "http://asmedl.org/getabs/servlet/GetabsServlet?prog=normal&id=JCISB6000012000003031005000001&idtype=cvips&gifs=Yes&ref=no",
author = "Erhan Batuhan Arisoy, Gunay Orbay and Levent Burak Kara",
}

pdf ASME

ASME Journal of Mechanical Design, 2012.

Abstract: We describe a new technique that works from a set of concept sketches to support the exploration and engineering of products. Our approach allows the capture and reuse of geometric shape information contained in concept sketches, as a means to generate solutions that can concurrently satisfy aesthetic and functional requirements. At the heart of our approach is a graph-based representation of sketches that allows the determination of topological and geometric similarities in the input sketches. This analysis, when combined with a geometric deformation analysis, results in a design space from which new shapes can be synthesized, or a developing design can be optimized to satisfy prescribed objectives. Moreover, it facilitates a sketch-based, interactive editing of existing designs that preserves the shape characteristics captured in the design space. A key advantage of the proposed method is that shape features common to all sketches as well as those unique to each sketch can be separately identified, thus allowing a mixing of different sketches to generate a topologically and geometrically rich set of conceptual alternatives. We demonstrate our technique with 2D and 3D examples.

BibTeX:

@article{orbayASMEJMD2012,
author = {Gunay Orbay and Levent Burak Kara},
title = {Shape Design From Exemplar Sketches Using Graph-Based Sketch Analysis},
journal ={Journal of Mechanical Design},
volume = {134},
number = {11},
year = {2012},
publisher = {ASME},
}


pdf

Abstract: We present a co-abstraction method that takes as input a collection of 3D objects, and produces a mutually consistent and individually identity-preserving abstraction of each object. In general, an abstraction is a simpler version of a shape that preserves its main characteristics. We hypothesize, however, that there is no single abstraction of an object. Instead, there is a variety of possible abstractions, and an admissible one can only be chosen conjointly with other objects’ abstractions. To this end, we introduce a new approach that hierarchically generates a spectrum of abstractions for each model in a shape collection. Given the spectra, we compute the appropriate abstraction level for each model such that shape simplification and inter-set consistency are collectively maximized, while individual shape identities are preserved.

BibTeX:

@article{Coabstraction2012,
title = {Co-Abstraction of Shape Collections},
author = {Mehmet Ersin Yumer and Levent Burak Kara},
journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012)},
volume = {31},
issue = {6},
pages = {158:1-158:11},
year = {2012},
}


Paper and Supplemental Materials
[Paper pdf] [Supplemental pdf]

Abstract: We present a new 3D surface modeling approach that enables curve-based creation and modification of smooth surfaces by sketching. The key feature of the proposed methods is a two-way communication between the user-designed curve networks and the generated surfaces. A user-drawn curve network serves as a control cage, from which a subdivision surface is generated. The subdivision surface is updated to match the curve network while minimizing the curvature variation throughout the surface. Surface fairness is controlled independently to modify the curve network into suitable configurations that guarantee a smooth underlying surface. This approach enables a concurrent modeling of the curve network and the underlying surface, thus eliminating the need for a laborious, iterative adjustment of the curve network for smooth surface creation. We demonstrate our approach with example models, and evaluate it with a user study.

BibTeX:

@article{Orbay2012CAG,
author = {Gunay Orbay and Levent Burak Kara},
title = {Sketch-Based Surface Design Using Malleable Curve Networks},
journal ={Computers /& Graphics}, volume = {36}, number = {8}
year = {2012},
pages = {916-929},
doi = {http://dx.doi.org/10.1016/j.cag.2012.08.008},
publisher = {Elsevier}, }

pdf

Elsevier Computer Aided Design, Volume 44, Issue 7, pp. 644-656.

Abstract: We present a new point set surfacing method based on a data-driven mapping between the parametric and geometric spaces. Our approach takes as input an unstructured and possibly noisy point set representing a two-manifold in R3. To facilitate parameterization, the set is first embedded in R2 using neighborhood-preserving locally linear embedding. A learning algorithm is then trained to learn a mapping between the embedded two-dimensional (2D) coordinates and the corresponding three-dimensional (3D) space coordinates. The trained learner is then used to generate a tessellation spanning the parametric space, thereby producing a surface in the geometric space. This approach enables the surfacing of noisy and non-uniformly distributed point sets. We discuss the advantages of the proposed method in relation to existing methods, and show its utility on a number of test models, as well as its applications to modeling in virtual reality environments.

BibTeX:

@article{YumerKara2012,
title = "Surface creation on unstructured point sets using neural networks",
journal = "Computer-Aided Design",
volume = "44",
number = "7",
pages = "644 - 656",
year = "2012",
note = "",
issn = "0010-4485",
doi = "10.1016/j.cad.2012.03.002",
url = "http://www.sciencedirect.com/science/article/pii/S0010448512000541",
author = "Mehmet Ersin Yumer and Levent Burak Kara",
keywords = "Surface fitting",
keywords = "Point sets",
keywords = "Neural networks",
keywords = "Virtual reality",
keywords = "Design" }

pdf Elsevier

Abstract: We propose a new sketch parsing and beautification method that converts digitally created design sketches into beautified line drawings. Our system uses a trainable, sequential bottom-up and top-down stroke clustering method that learns how to parse input pen strokes into groups of strokes each representing a single curve, followed by point-cloud ordering that facilitates curve fitting and smoothing. This approach enables greater conceptual freedom during visual ideation activities by allowing designers to develop their sketches using multiple, casually drawn strokes without requiring them to indicate the separation between different stroke groups. With the proposed method, raw sketches are seamlessly converted into vectorized geometric models, thus, facilitating downstream assessment and editing activities.

BibTeX:

@article{10.1109/TVCG.2010.105,
author = {Gunay Orbay and Levent Burak Kara},
title = {Beautification of Design Sketches Using Trainable Stroke Clustering and Curve Fitting},
journal ={IEEE Transactions on Visualization and Computer Graphics}, volume = {17},
issn = {1077-2626},
year = {2011},
pages = {694-708},
doi = {http://doi.ieeecomputersociety.org/10.1109/TVCG.2010.105},
publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }

pdf IEEE

AI-EDAM, Special Issue on Sketching and Pen-based Design Interaction, Volume 26, Issue 3, Pages 337-350, 2012.

Abstract: The hierarchical construction of solid models with current CAD systems provide little support in creating and editing free-form surfaces commonly encountered in industrial design. In this work, we propose a new design exploration method that enables sketch-based editing of free-form surface geometries where specific modifications can be applied at different levels of detail. This multi-level detail approach allows the designer to work from existing models and make alterations at coarse and fine representations of the geometry, thereby providing increased conceptual flexibility during modeling. At the heart of our approach lies a multi-scale representation of the geometry obtained through a spectral analysis on the discrete free-form surface. This representation is accompanied by a sketch-based surface editing algorithm that enables edits to be made at different levels. The seamless transfer of modifications across different levels of detail facilitates a fluid exploration of the geometry by eliminating the need for a manual specification of the shape hierarchy. We demonstrate our method with several design examples.

BibTeX:

@article{orbayAIEDAM2012,
author = {Gunay Orbay and Mehmet Ersin Yumer and Levent Burak Kara},
title = {Sketch-based shape exploration using multiscale free-form surface editing},
journal ={Artificial Intelligence for Engineering Design, Analysis and Manufacturing},
volume = {26},
number = {03},
year = {2012},
pages = {337-350},
doi = {http://dx.doi.org/10.1017/S0890060412000182},
publisher = {Cambridge},
}


pdf