Repository logo

Browsing by Author "Lyons, Carter, author"

Now showing 1 - 2 of 2
  • Thumbnail Image
    ItemOpen Access
    Compound-Gaussian-regularized inverse problems: theory, algorithms, and neural networks
    (Colorado State University. Libraries, 2024) Lyons, Carter, author; Cheney, Margaret, advisor; Raj, Raghu G., advisor; Azimi, Mahmood, committee member; King, Emily, committee member; Mueller, Jennifer, committee member
    Linear inverse problems are frequently encountered in a variety of applications including compressive sensing, radar, sonar, medical, and tomographic imaging. Model-based and data-driven methods are two prevalent classes of approaches used to solve linear inverse problems. Model-based methods incorporate certain assumptions, such as the image prior distribution, into an iterative estimation algorithm, often, as an example, solving a regularized least squares problem. Instead, data-driven methods learn the inverse reconstruction mapping directly by training a neural network structure on actual signal and signal measurement pairs. Alternatively, algorithm unrolling, a recent approach to inverse problems, combines model-based and data-driven methods through the implementation of an iterative estimation algorithm as a deep neural network (DNN). This approach offers a vehicle to embed domain-level and algorithmic insights into the design of neural networks such that the network layers are interpretable. The performance, in reconstructed signal quality, of unrolled DNNs often exceeds that of corresponding iterative algorithms and standard DNNs while doing so in a computationally efficient fashion. In this work, we leverage algorithm unrolling to combine a powerful statistical prior, the compound Gaussian (CG) prior, with the powerful representational ability of machine learning and DNN approaches. Specifically, first we construct a novel iterative CG-regularized least squares algorithm for signal reconstruction and provide a computational theory for this algorithm. Second, using algorithm unrolling, the newly developed CG-based least squares iterative algorithm is transformed into an original DNN in a manner to facilitate the learning of the optimization landscape geometry. Third, a generalization on the newly constructed CG regularized least squares iterative algorithm is developed, theoretically analyzed, and unrolled to yield a novel state-of-the-art DNN that provides a partial learning of the prior distribution constrained to the CG class of distributions. Fourth, techniques in statistical learning theory are employed for deriving original generalization error bounds on both unrolled DNNs to substantiate theoretical guarantees of each neural network when estimating signals from linear measurements after training. Finally, ample numerical experimentation is conducted for every new CG-based iterative and DNN approach proposed in this paper. Simulation results show our methods outperform previous state-of-the-art iterative signal estimation algorithms and deep-learning-based methods, especially with limited training datasets.
  • Thumbnail Image
    ItemOpen Access
    Stability in the weighted ensemble method
    (Colorado State University. Libraries, 2022) Lyons, Carter, author; Aristoff, David, advisor; Cheney, Margaret, committee member; Krapf, Diego, committee member
    In molecular dynamics, a quantity of interest is the mean first passage time, or average transition time, for a molecule to transition from a region A to a different region B. Often, significant potential barriers exist between A and B making the transition from A to B a rare event, which is an event that is highly improbable to occur. Correspondingly, the mean first passage time for a molecule to transition from A to B will be immense. So, using direct Markov chain Monte Carlo techniques to effectively estimate the mean first passage time is computationally infeasible due to the protracted simulations required. Instead, the Markov chain modeling the underlying molecular dynamics is simulated to steady-state and the steady-state flux from A into B is estimated. Then through the Hill relation, the mean first passage time is obtained as the reciprocal of the estimated steady-state flux. Estimating the steady-state flux into B is still a rare event but the difficulty has shifted from lengthy simulation times to a substantial variance on the desired estimate. Therefore, an importance sampling or importance splitting technique that emphasizes reaching B and reduces estimator variance must be used. Weighted ensemble is one importance sampling Markov chain Monte Carlo method often used to estimate mean first passage times in molecular dynamics. Broadly, weighted ensemble simulates a collection of Markov chain trajectories that are assigned a weight. Periodically, certain trajectories are copied while others are removed, to encourage a transition from A to B, and the trajectory weights are adjusted accordingly. By time-averaging the weighted average of these Markov chain trajectories, weighted ensemble estimates averages with respect to the Markov chain steady-state distribution. We focus on the use of weighted ensemble for estimating the mean first passage time from A to B, through estimating the steady-state flux from A into B, of a Markov chain where upon reaching B is restarted in A according to an initial, or recycle, distribution. First, we give a mathematical detailing of the weighted ensemble algorithm and provide an unbiased property, ergodic property, and variance formula. The unbiased property gives that the weighted ensemble average of many Markov chain trajectories produces an unbiased estimate for the underlying Markov chain law. Next, the ergodic property states that the weighted ensemble estimator converges almost surely to the desired steady-state average. Lastly, the variance formula provides exact variance of the weighted ensemble estimator. Next, we analyze the impact of the initial or recycle distribution, in A, on bias and variance of the weighted ensemble estimate and compare against adaptive multilevel splitting. Adaptive multilevel splitting is an importance splitting Markov chain Monte Carlo method also used in molecular dynamics for estimating mean first passage times. It has been studied that adaptive multilevel splitting requires a precise importance sampling of the initial, or recycle, distribution to maintain reasonable variance bounds on the adaptive multilevel splitting estimator. We show that the weighted ensemble estimator is less sensitive to the initial distribution since importance sampling the initial distribution frequently does not reduce the variance of the weighted ensemble estimator significantly. For a generic three state Markov chain and one dimensional overdamped Langevin dynamics, we develop specific conditions which must be satisfied for initial distribution importance sampling to provide a significant variance reduction on the weighted ensemble estimator. Finally, for bias, we develop conditions on A, such that the mean first passage time from A to B is stable with respect to changes in the initial distribution. That is, under a perturbation of the initial distribution the resulting change in the mean first passage time is insignificant. The conditions on A are verified with one dimensional overdamped Langevin dynamics and an example is provided. Furthermore, when the mean first passage time is unstable, we develop bounds, for one dimensional overdamped Langevin dynamics, on the change in the mean first passage time and show the tightness of the bounds with numerical examples.