Abstract: Many people are interested in taking astonishing photos and sharing with others. Emerging high-tech hardware and software facilitate ubiquitousness and functionality of digital photography. Because composition matters in photography, researchers have leveraged some common composition techniques, such as the rule of thirds, the triangle technique, and the perspective-related techniques, to assess the aesthetic quality of photos computationally. However, composition techniques developed by professionals are far more diverse than well-documented techniques can cover. We leverage the vast under-explored innovations in photography for computational composition assistance, and there is a lack of a holistic framework to capture important aspects of a given scene and help individuals by constructive clues to take a better shot in their adventure. We propose a comprehensive framework, named CAPTAIN (Composition Assistance for Photo Taking), containing integrated deep-learned semantic detectors, sub-genre categorization, artistic pose clustering, personalized aesthetics-based image retrieval, and style set matching. The framework is backed by a large dataset crawled from a photo-sharing Website with mostly photography enthusiasts and professionals.
The work proposes a sequence of steps that have not been explored in the past by researchers.
The work addresses personal preferences for composition through presenting a ranked-list of photographs to the user based on user-specified weights in the similarity measure. We believe our design leveraging user-defined preferences. Our framework extracts ingredients of a given snapshot of a scene (i.e. the scene that the user is interested in taking a picture of) as a set of composition-related features ranging from low-level features such as color, pattern, and texture to high-level features such as pose, category, rating, gender, and object. Our composition model, indexed offline, is used to provide visual exemplars as recommendations for the scene, which is a novel model for aesthetics-related image retrieval. We believe our design leveraging user-defined preferences The matching algorithm recognizes the best shot among a sequence of shots with respect to the user’s preferred style set. We have conducted a number of experiments on the newly proposed components and reported findings. A user study demonstrates that the work is useful to those taking photos.
Keywords: Computational Composition, Image Aesthetics, Photography, Deep Learning, Image Retrieval