Improving Image Quality, Content, and Practicality: Knowledge-Oriented Methods for Information Enhancement

Abstract

With the all-around change caused by artificial intelligence (AI) technology, the need for perception in real scenes is increasingly urgent. As an essential medium to perceive the world, the integrity and richness of information often determine the performance of the algorithm. However, in the process of image capture, there are usually a variety of uncontrollable physical factors, resulting in image information degeneration and missing, thus affecting the acquisition and utilization of information in the application process of high-level visual tasks. The concept of information enhancement is becoming increasingly important in today’s world, as we are constantly bombarded with vast amounts of information from multiple sources. It is not enough to have access to information. We need to be able to extract the most relevant and useful information from this abundance of data. This is where information enhancement comes in, as it aims to improve the quality and relevance of information through various techniques. The present thesis examines that the core theme is information enhancement and a knowledge-oriented systematic framework of enhancement for various applications is proposed, which includes single-modal information enhancement (SIE), multi-modal information enhancement (MIE), and task-driven information enhancement (TIE). First, SIE investigates the single modal medical image quality enhancement techniques for improved visual inspection with the downstream application. Second, MIE aims to integrate multiple modal medical images by featurelevel fusion for further understanding and complementing information enhancement. Finally, TIE extends the generality of enhancement perspective that embeds into high-level visual tasks. Single-modal information enhancement (SIE) is focused on enhancing the quality and clarity of information from a single source or modality. The goal of SIE is to improve the accuracy and usefulness of the information within a single modality, which can be especially important in situations where the data quality is low or the signal-to-noise ratio is poor. To achieve the research goal of this thesis, a framework of image information enhancement for two knowledge-oriented methods based on generative adversarial neural networks (GAN) is proposed. Multi-modal information enhancement (MIE) involves integrating and enhancing information from multiple sources or modalities. This can include combining images to improve understanding or merging data from multiple sensors to provide a complete picture. The goal of MIE is to increase the amount and quality of information available by integrating multiple sources, thereby enabling better decision making and understanding. To investigate the feasibility of MIE, one optimization based fusion method and one deep learning GAN-based method are proposed and verified by medical image segmentation task and object detection/tracking. Task-driven information enhancement (TIE) focuses on enhancing information specifically for a particular application. For this topic, two methods are proposed to verify how to embed knowledge-oriented information enhancement in highlevel vision applications, including medical image segmentation and object detection/ tracking. The research conducted as part of the Ph.D. thesis on information enhancement will likely involve developing and evaluating various knowledge-oriented methods for implementing these information enhancement techniques. This may include analyzing image data from various modalities and developing algorithms to improve the quality, content, and relevance practicability of the information contained within these sources. Overall, the Ph.D. thesis on information enhancement is to contribute to our understanding of how to extract the most useful and relevant information from the vast amount of data, to advance our knowledge of critical principles and techniques.