Compound-Gaussian-regularized inverse problems: theory, algorithms, and neural networks
Date
2024
Journal Title
Journal ISSN
Volume Title
Abstract
Linear inverse problems are frequently encountered in a variety of applications including compressive sensing, radar, sonar, medical, and tomographic imaging. Model-based and data-driven methods are two prevalent classes of approaches used to solve linear inverse problems. Model-based methods incorporate certain assumptions, such as the image prior distribution, into an iterative estimation algorithm, often, as an example, solving a regularized least squares problem. Instead, data-driven methods learn the inverse reconstruction mapping directly by training a neural network structure on actual signal and signal measurement pairs. Alternatively, algorithm unrolling, a recent approach to inverse problems, combines model-based and data-driven methods through the implementation of an iterative estimation algorithm as a deep neural network (DNN). This approach offers a vehicle to embed domain-level and algorithmic insights into the design of neural networks such that the network layers are interpretable. The performance, in reconstructed signal quality, of unrolled DNNs often exceeds that of corresponding iterative algorithms and standard DNNs while doing so in a computationally efficient fashion. In this work, we leverage algorithm unrolling to combine a powerful statistical prior, the compound Gaussian (CG) prior, with the powerful representational ability of machine learning and DNN approaches. Specifically, first we construct a novel iterative CG-regularized least squares algorithm for signal reconstruction and provide a computational theory for this algorithm. Second, using algorithm unrolling, the newly developed CG-based least squares iterative algorithm is transformed into an original DNN in a manner to facilitate the learning of the optimization landscape geometry. Third, a generalization on the newly constructed CG regularized least squares iterative algorithm is developed, theoretically analyzed, and unrolled to yield a novel state-of-the-art DNN that provides a partial learning of the prior distribution constrained to the CG class of distributions. Fourth, techniques in statistical learning theory are employed for deriving original generalization error bounds on both unrolled DNNs to substantiate theoretical guarantees of each neural network when estimating signals from linear measurements after training. Finally, ample numerical experimentation is conducted for every new CG-based iterative and DNN approach proposed in this paper. Simulation results show our methods outperform previous state-of-the-art iterative signal estimation algorithms and deep-learning-based methods, especially with limited training datasets.
Description
Rights Access
Subject
inverse problems
machine learning
deep neural networks
nonlinear programming
least squares methods