Graphic Design Layout


Graphics layout concerns about the arrangement of graphic design elements to form a structure that can achieve some objectives. While the layout may refer to a 2D page, a 2D/3D object, or a 3D scene, the objectives of a layout may include visual aesthetics and functionality. The arrangement of graphic design elements is typically via the adjustment of their properties, such as location, orientation, color, size and shape. In this project, we aim at developing automatic tools for synthesizing novel layouts. In particular, we are interested in learning the design rules through the data-driven approach, using machine learning or deep learning techniques, to learn layout features. Concurrently, we are also investigating how visual saliency may affect graphic design layout. (See also our work on image saliency.)


Modeling Fonts in Context: Font Prediction on Web Design [paper] [suppl]

Nanxuan Zhao, Ying Cao, and Rynson Lau

Computer Graphics Forum (Proc. Pacific Graphics 2018), Oct. 2018

Abstract. Web designers often carefully select fonts to fit the context of a web design to make the design look aesthetically pleasing and effective in communication. However, selecting proper fonts for a web design is a tedious and time-consuming task, as each font has many properties, such as font face, color, and size, resulting in a very large search space. In this paper, we aim to model fonts in context, by studying a novel and challenging problem of predicting fonts that match a given web design. To this end, we propose a novel, multi-task deep neural network to jointly predict font face, color and size for each text element on a web design, by considering multi-scale visual features and semantic tags of the web design. To train our model, we have collected a CTXFont dataset, which consists of 1k professional web designs, with labeled font properties. Experiments show that our model outperforms the baseline methods, achieving promising qualitative and quantitative results on the font selection task. We also demonstrate the usefulness of our method in a font selection task via a user study.

Task-driven Webpage Saliency [paper] [suppl]

Quanlong Zheng, Jianbo Jiao, Ying Cao, and Rynson Lau

Proc. ECCV, Sept. 2018

Fig. 1: Given an input webpage (a), our model can predict a different saliency map under a different task, e.g., information browsing (b), form filling (c) and shopping (d).

Abstract. In this paper, we present an end-to-end learning framework for predicting task-driven visual saliency on webpages. Given a webpage, we propose a convolutional neural network to predict where people look at it under different task conditions. Inspired by the observation that given a specific task, human attention is strongly correlated with certain semantic components on a webpage (e.g., images, buttons and input boxes), our network explicitly disentangles saliency prediction into two independent sub-tasks: task-specific attention shift prediction and task-free saliency prediction. The task-specific branch estimates task-driven attention shift over a webpage from its semantic components, while the task-free branch infers visual saliency induced by visual features of the webpage. The outputs of the two branches are combined to produce the final prediction. Such a task decomposition framework allows us to efficiently learn our model from a small-scale task-driven saliency dataset with sparse labels (captured under a single task condition). Experimental results show that our method outperforms the baselines and prior works, achieving state-of-the-art performance on a newly collected benchmark dataset for task-driven webpage saliency detection.

What Characterizes Personalities of Graphic Designs? [paper] [video] [suppl] [code] [dataset]

Nanxuan Zhao, Ying Cao, and Rynson Lau

ACM Trans. on Graphics (Proc. ACM SIGGRAPH 2018), 37(4), Aug. 2018

Abstract: Graphic designers often manipulate the overall look and feel of their designs to convey certain personalities (e.g., cute, mysterious and romantic) to impress potential audiences and achieve business goals. However, understanding the factors that determine the personality of a design is challenging, as a graphic design is often a result of thousands of decisions on numerous factors, such as font, color, image, and layout. In this paper, we aim to answer the question of what characterizes the personality of a graphic design. To this end, we propose a deep learning framework for exploring the effects of various design factors on the perceived personalities of graphic designs. Our framework learns a convolutional neural network (called personality scoring network) to estimate the personality scores of graphic designs by ranking the crawled web data. Our personality scoring network automatically learns a visual representation that captures the semantics necessary to predict graphic design personality. With our personality scoring network, we systematically and quantitatively investigate how various design factors (e.g., color, font, and layout) affect design personality across different scales (from pixels, regions to elements). We also demonstrate a number of practical application scenarios of our network, including element-level design suggestion and example-based personality transfer.

Directing User Attention via Visual Flow on Web Designs [paper] [video] [suppl] [models] [dataset]

Xufang Pang*, Ying Cao*, Rynson Lau, and Antoni Chan (* joint first authors)

ACM Trans. on Graphics (Proc. ACM SIGGRAPH Asia 2016), 35(6), Article 240, Dec. 2016

Patent pending (US application number: 15/776,568)

Abstract: We present a novel approach that allows web designers to easily direct user attention via visual flow on web designs. By collecting and analyzing users' eye gaze data on real-world webpages under the task-driven condition, we build two user attention models that characterize user attention patterns between a pair of page components. These models enable a novel web design interaction for designers to easily create a visual flow to guide users' eyes (i.e., direct user attention along a given path) through a web design with minimal effort. In particular, given an existing web design as well as a designer-specified path over a subset of page components, our approach automatically optimizes the web design so that the resulting design can direct users' attention to move along the input path. We have tested our approach on various web designs of different categories. Results show that our approach can effectively guide user attention through the web design according to the designer's high-level specification.

Look Over Here: Attention-Directing Composition of Manga Elements [paper] [suppl] [video]

Ying Cao, Rynson Lau, and Antoni Chan

ACM Trans. on Graphics (Proc. ACM SIGGRAPH 2014), 33(4), Article 94, Aug. 2014

Abstract: Picture subjects and text balloons are basic elements in comics, working together to propel the story forward. Japanese comics artists often leverage a carefully designed composition of subjects and balloons (generally referred to as panel elements) to provide a continuous and fluid reading experience. However, such a composition is hard to produce for people without the required experience and knowledge. In this paper, we propose an approach for novices to synthesize a composition of panel elements that can effectively guide the reader's attention to convey the story. Our primary contribution is a probabilistic graphical model that describes the relationships among the artist's guiding path, the panel elements, and the viewer attention, which can be effectively learned from a small set of existing manga pages. We show that the proposed approach can measurably improve the readability, visual appeal, and communication of the story of the resulting pages, as compared to an existing method. We also demonstrate that the proposed approach enables novice users to create higher-quality compositions with less time, compared with commercially available programs.

Structured Mechanical Collage [paper] [video] [more results]

Zhe Huang, Jiang Wang, Hongbo Fu, and Rynson Lau

IEEE Trans. on Visualization and Computer Graphics, 20(7):1076-1082, July 2014

Abstract: We present a method to build 3D structured mechanical collages consisting of numerous elements from the database given artist-designed proxy models. The construction is guided by some graphic design principles, namely unity, variety and contrast. Our results are visually more pleasing than previous works as confirmed by a user study.

Automatic Stylistic Manga Layout [paper] [video] [more results]

Ying Cao, Antoni Chan, and Rynson Lau

ACM Trans. on Graphics (Proc. ACM SIGGRAPH Asia 2012), 31(6), Article 141, Nov. 2012

Abstract: Manga layout is a core component in manga production, characterized by its unique styles. However, stylistic manga layouts are difficult for novices to produce as it requires hands-on experience and domain knowledge. In this paper, we propose an approach to automatically generate a stylistic manga layout from a set of input artworks with user-specified semantics, thus allowing less-experienced users to create high-quality manga layouts with minimal efforts. We first introduce three parametric style models that encode the unique stylistic aspects of manga layouts, including layout structure, panel importance, and panel shape. Next, we propose a two-stage approach to generate a manga layout: 1) an initial layout is created that best fits the input artworks and layout structure model, according to a generative probabilistic framework; 2) the layout and artwork geometries are jointly refined using an efficient optimization procedure, resulting in a professional-looking manga layout. Through a user study, we demonstrate that our approach enables novice users to easily and quickly produce higher-quality layouts that exhibit realistic manga styles, when compared to a commercially-available manual layout tool.

Last updated in August 2018