Browsing by Author "Kirby, Michael, committee member"

Now showing 1 - 20 of 38

Open Access
Anchor centric virtual coordinate systems in wireless sensor networks: from self-organization to network awareness
(Colorado State University. Libraries, 2012) Dhanapala, Dulanjalie C., author; Jayasumana, Anura P., advisor; Kirby, Michael, committee member; Pezeshki, Ali, committee member; Ray, Indrakshi, committee member
Future Wireless Sensor Networks (WSNs) will be collections of thousands to millions of sensor nodes, automated to self-organize, adapt, and collaborate to facilitate distributed monitoring and actuation. They may even be deployed over harsh geographical terrains and 3D structures. Low-cost sensor nodes that facilitate such massive scale networks have stringent resource constraints (e.g., in memory and energy) and limited capabilities (e.g., in communication range and computational power). Economic constraints exclude the use of expensive hardware such as Global Positioning Systems (GPSs) for network organization and structuring in many WSN applications. Alternatives that depend on signal strength measurements are highly sensitive to noise and fading, and thus often are not pragmatic for network organization. Robust, scalable, and efficient algorithms for network organization and reliable information exchange that overcome the above limitations without degrading the network's lifespan are vital for facilitating future large-scale WSN networks. This research develops fundamental algorithms and techniques targeting self-organization, data dissemination, and discovery of physical properties such as boundaries of large-scale WSNs without the need for costly physical position information. Our approach is based on Anchor Centric Virtual Coordinate Systems, commonly called Virtual Coordinate Systems (VCSs), in which each node is characterized by a coordinate vector of shortest path hop distances to a set of anchor nodes. We develop and evaluate algorithms and techniques for the following tasks associated with use of VCSs in WSNs: (a) novelty analysis of each anchor coordinate and compressed representation of VCSs; (b) regaining lost directionality and identifying a 'good' set of anchors; (c) generating topology preserving maps (TPMs); (d) efficient and reliable data dissemination, and boundary identification without physical information; and (f) achieving network awareness at individual nodes. After investigating properties and issues related to VCS, a Directional VCS (DVCS) is proposed based on a novel transformation that restores the lost directionality information in VCS. Extreme Node Search (ENS), a novel and efficient anchor placement scheme, starts with two randomly placed anchors and then uses this directional transformation to identify the number and placement of anchors in a completely distributed manner. Furthermore, a novelty-filtering-based approach for identifying a set of 'good' anchors that reduces the overhead and power consumption in routing is discussed. Physical layout information such as physical voids and even relative physical positions of sensor nodes with respect to X-Y directions are absent in a VCS description. Obtaining such information independent of physical information or signal strength measurements has not been possible until now. Two novel techniques to extract Topology Preserving Maps (TPMs) from VCS, based on Singular Value Decomposition (SVD) and DVCS are presented. A TPM is a distorted version of the layout of the network, but one that preserves the neighborhood information of the network. The generalized SVD-based TPM scheme for 3D networks provides TPMs even in situations where obtaining accurate physical information is not possible. The ability to restore directionality and topology-based Cartesian coordinates makes VCS competitive and, in many cases, a better alternative to geographic coordinates. This is demonstrated using two novel routing schemes in VC domain that outperform the well-known physical information-based routing schemes. The first scheme, DVC Routing (DVCR) uses the directionality recovered by DVCS. Geo-Logical Routing (GLR) is a technique that combines the advantages of geographic and logical routing to achieve higher routability at a lower cost by alternating between topology and virtual coordinate spaces to overcome local minima in the two domains. GLR uses topology domain coordinates derived solely from VCS as a better alternative for physical location information. A boundary detection scheme that is capable of identifying physical boundaries even for 3D surfaces is also proposed. "Network awareness" is a node's cognition of its neighborhood, its position in the network, and the network-wide status of the sensed phenomena. A novel technique is presented whereby a node achieves network awareness by passive listening to routine messages associated with applications in large-scale WSNs. With the knowledge of the network topology and phenomena distribution, every node is capable of making solo decisions that are more sensible and intelligent, thereby improving overall network performance, efficiency, and lifespan. In essence, this research has laid a firm foundation for use of Anchor Centric Virtual Coordinate Systems in WSN applications, without the need for physical coordinates. Topology coordinates, derived from virtual coordinates, provide a novel, economical, and in many cases, a better alternative to physical coordinates. A novel concept of network awareness at nodes is demonstrated.
Open Access
Automated deep learning architecture design using differentiable architecture search (DARTS)
(Colorado State University. Libraries, 2019) Sharma, Kartikay, author; Anderson, Chuck, advisor; Beveridge, Ross, committee member; Kirby, Michael, committee member
Creating neural networks by hand is a slow trial-and-error based process. Designing new architectures similar to GoogleNet or FractalNets, which use repeated tree-based structures, is highly likely to be inefficient and sub-optimal because of the large number of possibilities for composing such structures. Recently, neural architecture search algorithms have been able to automate the process of architecture design and have often attained state-of-the-art performances on CIFAR-10, ImageNet and Penn Tree Bank datasets. Even though the search time has been reduced to tens of GPU hours from tens of thousands of GPU hours, most search algorithms rely on additional controllers and hypernetworks to generate architecture encoding or predict weights for sampled architectures. These controllers and hypernetworks might require optimal structure when deployed on a new task on a new dataset. And since this is done by hand, the problem of architecture search is not really solved. Differentiable Architecture Search (DARTS) avoids this problem by using gradient descent methods. In this work, the DARTS algorithm is studied under various conditions and search hyperparameters. DARTS is applied to CIFAR-10 to check reproducibility of the original results. It is also tested in a new setting — on the CheXpert dataset — to discover new architectures and is compared to a baseline DenseNet121 model. The architectures searched using DARTS achieve better performance on the validation set than the baseline model.
Open Access
Automated identification of objects in aerial imagery using a CNN: oil/gas sites to Martian volcanoes
(Colorado State University. Libraries, 2021) Dileep, Sonu, author; Beveridge, Ross, advisor; Azimi-Sadjadi, Mahmood R., committee member; Kirby, Michael, committee member
The recent advancements in Deep Learning techniques have revolutionized the field of computer vision. Deep Learning has received massive attention in remote sensing. The availability of open-source satellite imagery has opened up lots of remote sensing applications, from object detection to disaster assessment. In this work, I explore the application of deep learning for automated identification of oil/gas sites in DJ Basin, Colorado and detection of volcanoes on Mars. Oil and gas production sites are one of the significant sources of methane emissions all over the world. Methane emission studies from oil/gas sites require a count of major equipment in a site. However, these counts are not properly documented, and manual annotation of each piece of equipment in a site takes a lot of time and effort. To solve this challenge, an end-to-end deep learning model is developed that finds the well sites from satellite imagery and returns a count of major equipment at each site. Second, an end-to-end deep learning approach is used to detect volcanoes on Mars. Volcanic constructs are fundamental in studying the potential for past and future habitable environment on Mars. Even though large volcanic constructs are well documented, there is no proper documentation for smaller volcanoes. Manually finding all the smaller volcanoes will be a tedious task. In the second part of my work, I explore the potential of deep learning approaches for Martian volcano detection.
Open Access
Automatic question detection from prosodic speech analysis
(Colorado State University. Libraries, 2019) Hirsch, Rachel, author; Draper, Bruce, advisor; Whitley, Darrell, advisor; Kirby, Michael, committee member
Human-agent spoken communication has become ubiquitous over the last decade, with assistants such as Siri and Alexa being used more every day. An AI agent needs to understand exactly what the user says to it and respond accurately. To correctly respond, the agent has to know whether it is being given a command or asked a question. In Standard American English (SAE), both word choice and intonation of the speaker are necessary to discern the true sentiment of an utterance. Much Natural Language Processing (NLP) research has been done into automatically determining these sentence types using word choice alone. However, intonation is ultimately the key to understanding the sentiment of a spoken sentence. This thesis uses a series of attributes to characterize vocal prosody of utterances to train classifiers to detect questions. The dataset used to train these classifiers is a series of hearings by the Supreme Court of the United States (SCOTUS). Prosody-trained classifier results are compared against a text-based classifier, using Google Speech-to-Text transcriptions of the same dataset.
Open Access
Causal inference using observational data - case studies in climate science
(Colorado State University. Libraries, 2020) Samarasinghe, Savini M., author; Ebert-Uphoff, Imme, advisor; Anderson, Chuck, committee member; Chong, Edwin, committee member; Kirby, Michael, committee member
We are in an era where atmospheric science is data-rich in both observations (e.g., satellite/ sensor data) and model output. Our goal with causal discovery is to apply suitable data science approaches to climate data to make inferences about the cause-effect relationships between climate variables. In this research, we focus on using observational studies, an approach that does not rely on controlled experiments, to infer cause-effect. Due to reasons such as latent variables, these observational studies do not allow us to prove causal relationships. Nevertheless, they provide data-driven hypotheses of the interactions, which can enable us to get insights into the salient interactions as well as the timescales at which they occur. Even though there are many different causal inference frameworks and methods that rely on observational studies, these approaches have not found widespread use within the climate or Earth science communities. To date, the most commonly used observational approaches include lagged correlation/regression analysis, as well as the bivariate Granger causality approach. We can attribute this lack of popularity to two main reasons. First is the inherent difficulty of inferring cause-effect in climate. Complex processes in the climate interact with each other at varying time spans. These interactions can be nonlinear, the distributions of relevant climate variables can be non-Gaussian, and the processes can be chaotic. A researcher interested in these causal inference problems has to face many challenges varying from identifying suitable variables, data, preprocessing and inference methods, as well as setting up the inference problem in a physically meaningful way. Also, the limited exposure and accessibility to modern causal inference approaches is another reason for their limited use within the climate science community. In this dissertation, we present three case studies related to causal inference in climate science, namely, (1) causal relationships between the Arctic temperature and mid-latitude circulations, (2) relationships between the Madden Julian Oscillation (MJO) and the North Atlantic Oscillation (NAO) and (3) the causal relationships between atmospheric disturbances of different spatial scales (e.g., Planetary vs. Synoptic). We use methods based on probabilistic graphical models to infer cause-effect, specifically constraint-based structure learning methods, and graphical Granger methods. For each case study, we analyze and document the scientific thought process of setting up the problem, the challenges faced, and how we have dealt with the challenges. The challenges discussed include, but not limited to, method selection, variable representation, and data preparation. We also present a successful high-dimensional study of causal discovery in spectral space. The main objectives of this research are to make causal inference methods more accessible to a researcher/climate scientist who is at entry-level to spatiotemporal causality and to promote more modern causal inference methods to the climate science community. The case studies, covering a wide range of questions and challenges, are meant to act as a resourceful starting point to a researcher interested in tackling more general causal inference problems in climate.
Open Access
Classification of P300 from non-invasive EEG signal using convolutional neural network
(Colorado State University. Libraries, 2022) Farhat, Nazia, author; Anderson, Charles W., advisor; Kirby, Michael, committee member; Blanchard, Nathaniel, committee member
Brain-Computer Interface system is a communication tool for the patients of neuromuscular diseases. The efficiency of such a system largely depends on the accurate and reliable detection of the brain signal employed in its operation. P300 Speller, a well-known BCI system, which helps the user select the desired alphabet in the communication process uses an Electroencephalography signal called P300 brain wave. The spatiotemporal nature and the low Signal-to-noise ratio along with the high dimensionality of P300 signal imposes difficulties in its accurate recognition. Moreover, its inter- and intra-subject variability necessitates case-specific experimental setup requiring considerable amount of time and resources before the system's deployment for use. In this thesis Convolutional Neural Network is applied to detect the P300 signal and observe the distinguishing features of P300 and non-P300 signals extracted by the neural network. Three different shapes of the filters, namely 1-D CNN, 2-D CNN, and 3-D CNN are examined separately to evaluate their detection ability of the target signals. Virtual channels created with three different weighting techniques are explored in 3-D CNN analysis. Both within-subject and cross-subject examinations are performed. Single trial accuracy with CNN implementation. Higher single trial accuracy is observed for all the subjects with CNN implementation compared to that achieved with Stepwise Linear Discriminant Analysis. Up to approximately 80% within-subject accuracy and 64% cross- subject accuracy are recorded in this research. 1-D CNN outperforms all the other models in terms of classification accuracy.
Open Access
Convolutional neural networks for EEG signal classification in asynchronous brain-computer interfaces
(Colorado State University. Libraries, 2019) Forney, Elliott M., author; Anderson, Charles, advisor; Ben-Hur, Asa, committee member; Kirby, Michael, committee member; Rojas, Donald, committee member
Brain-Computer Interfaces (BCIs) are emerging technologies that enable users to interact with computerized devices using only voluntary changes in their mental state. BCIs have a number of important applications, especially in the development of assistive technologies for people with motor impairments. Asynchronous BCIs are systems that aim to establish smooth, continuous control of devices like mouse cursors, electric wheelchairs and robotic prostheses without requiring the user to interact with time-locked external stimuli. Scalp-recorded Electroencephalography (EEG) is a noninvasive approach for measuring brain activity that shows considerable potential for use in BCIs. Inferring a user's intent from spontaneously produced EEG signals remains a challenging problem, however, and generally requires specialized machine learning and signal processing methods. Current approaches typically involve guided preprocessing and feature generation procedures used in combination with with carefully regularized, often linear, classification algorithms. The current trend in machine learning, however, is to move away from approaches that rely on feature engineering in favor of multilayer (deep) artificial neural networks that rely on few prior assumptions and are capable of automatically learning hierarchical, multiscale representations. Along these lines, we propose several variants of the Convolutional Neural Network (CNN) architecture that are specifically designed for classifying EEG signals in asynchronous BCIs. These networks perform convolutions across time with dense connectivity across channels, which allows them to capture spatiotemporal patterns while achieving time invariance. Class labels are assigned using linear readout layers with label aggregation in order to reduce susceptibility to overfitting and to allow for continuous control. We also utilize transfer learning in order to reduce overfitting and leverage patterns that are common across individuals. We show that these networks are multilayer generalizations of Time-Delay Neural Networks (TDNNs) and that the convolutional units in these networks can be interpreted as learned, multivariate, nonlinear, finite impulse-response filters. We perform a series of offline experiments using EEG data recorded during four imagined mental tasks: silently count backward from 100 by 3's, imagine making a left-handed fist, visualize a rotating cube and silently sing a favorite song. Data were collected using a portable, eight-channel EEG system from 10 participants with no impairments in a laboratory setting and four participants with motor impairments in their home environments. Experimental results demonstrate that our proposed CNNs consistently outperform baseline classifiers that utilize power-spectral densities. Transfer learning yields an additional performance improvement, but only when used in combination with multilayer networks. Our final test results achieve a mean classification accuracy of 57.86%, which is 8.57% higher than the 49.29% achieved by our baseline classifiers. In terms of information transfer rates, our proposed methods achieve a mean of 15.82 bits-per-minute while our baseline methods achieve 9.35 bits-per-minute. For two individuals, our CNNs achieve a classification accuracy of 90.00%, which is 10-20% higher than our baseline methods. A comparison with external studies suggests that these results are on par with the state-of-the-art, despite our relatively rigorous experimental design. We also perform a number of experiments that analyze the types of patterns our classifiers learn to utilize. This includes a detailed analysis of aggregate power-spectral densities, examining the layer-wise activations produced by our CNNs, extracting the frequency responses of convolutional layers using Fourier analysis and finding optimized input sequences for trained networks. These analyses highlight several ways that the patterns our methods learn to utilize are related to known patterns that occur in EEG signals while also creating new questions about some types of patterns, including high-frequency information. Examining the behavior of our CNNs also provides insights into the inner workings of these networks and demonstrates that they are, in fact, learning to form hierarchical, multiscale representations of EEG signals.
Open Access
Efficient representation, measurement, and recovery of spatial and social networks
(Colorado State University. Libraries, 2021) Mahindre, Gunjan S., author; Jayasumana, Anura, advisor; Paffenroth, Randy, committee member; Maciejewski, Anthony, committee member; Kirby, Michael, committee member
To view the abstract, please see the full text of the document.
Open Access
General model-based decomposition framework for polarimetric SAR images
(Colorado State University. Libraries, 2017) Dauphin, Stephen, author; Cheney, Margaret, advisor; Kirby, Michael, committee member; Pinaud, Olivier, committee member; Morton, Jade, committee member
Polarimetric synthetic aperture radars emit a signal and measure the magnitude, phase, and polarization of the return. Polarimetric decompositions are used to extract physically meaningful attributes of the scatterers. Of these, model-based decompositions intend to model the measured data with canonical scatter-types. Many advances have been made to this field of model-based decomposition and this work is surveyed by the first portion of this dissertation. A general model-based decomposition framework (GMBDF) is established that can decompose polarimetric data with different scatter-types and evaluate how well those scatter-types model the data by comparing a residual term. The GMBDF solves for all the scatter-type parameters simultaneously that are within a given decomposition by minimizing the residual term. A decomposition with a lower residual term contains better scatter-type models for the given data. An example is worked through that compares two decompositions with different surface scatter-type models. As an application of the polarimetric decomposition analysis, a novel terrain classification algorithm of polSAR images is proposed. In the algorithm, the results of state-of-the-art polarimetric decompositions are processed for an image. Pixels are then selected to represent different terrain classes. Distributions of the parameters of these selected pixels are determined for each class. Each pixel in the image is given a score according to how well its parameters fit the parameter distributions of each class. Based on this score, the pixel is either assigned to a predefined terrain class or labeled unclassified.
Open Access
High-dimensional nonlinear data assimilation with non-Gaussian observation errors for the geosciences
(Colorado State University. Libraries, 2023) Hu, Chih-Chi, author; van Leeuwen, Peter Jan, advisor; Kummerow, Christian, committee member; Anderson, Jeffrey, committee member; Bell, Michael, committee member; Kirby, Michael, committee member
Data assimilation (DA) plays an indispensable role in modern weather forecasting. DA aims to provide better initial conditions for the model by combining the model forecast and the observations. However, modern DA methods for weather forecasting rely on linear and Gaussian assumptions to seek efficient solutions. These assumptions can be invalid, e.g., for problems associated with clouds, or for the assimilation of remotely-sensed observations. Some of these observations are either discarded, or not used properly due to these inappropriate assumptions in DA. Therefore, the goal of this dissertation is to seek solutions to tackle the issues arising from the linear and Gaussian assumptions in DA. This dissertation can be divided into two parts. In the first part, we explore the potential of the particle flow filter (PFF) in high dimensional systems. First, we tested the PFF in the 1000- dimensional Lorenz 96 model. The key innovation is we find that using a matrix kernel in the PFF can prevent the collapse of particles along the observed directions, for a sparsely observed and high-dimensional system with only a small number of particles. We also demonstrate that the PFF is able to represent a multi-modal posterior distribution in a high-dimensional space. Next, in order to apply the PFF for the atmospheric problem, we devise a parallel algorithm for PFF in the Data Assimilation Research Testbed (DART), called PFF-DART. A two-step PFF was developed that closely resembles the original PFF algorithm. A year-long cycling data assimilation experiment with a simplified atmospheric general circulation model shows PFF-DART is able to produce stable and comparable results to the Ensemble Adjustment Kalman Filter (EAKF) for linear and Gaussian observations. Moreover, PFF-DART can better assimilate the non-linear observations and reduce the errors of the ensemble, compared to the EAKF. In the second part, we shift our focus to the observation error in data assimilation. Traditionally, observation errors have been assumed to follow a Gaussian distribution mainly for two reasons: it is difficult to estimate observation error statistics beyond its second moment, and most of the DA methods assume a Gaussian observation error by construction. We developed the so-called Deconvolution-based Observation Error Estimation (DOEE), that can estimate the full distribution of the observation error. We apply DOEE to the all-sky microwave radiances and show that they indeed have non-Gaussian observation errors, especially in a cloudy and humid environment. Next, in order to incorporate the non-Gaussian observation errors into variational methods, we explore an evolving-Gaussian approach, that essentially uses a state dependent Gaussian observation error in each outer loop of the minimization. We demonstrate the merits of this method in an idealized experiment, and implemented it in the Integrated Forecasting System of the European Centre for Medium-Range Weather Forecasts. Preliminary results show improvement for the short-term forecast of lower-tropospheric humidity, cloud, and precipitation when the observation error models of a small set of microwave channels are replaced by the non-Gaussian error models. In all, this dissertation provides possible solutions for outstanding non-linear and non-Gaussian data assimilation problems in high-dimension systems. While there are still important remaining issues, we hope this dissertation lays a foundation for the future non-linear and non-Gaussian data assimilation research and practice.
Open Access
Improvements to the tracking process
(Colorado State University. Libraries, 2024) Lewis, Codie T., author; Cheney, Margaret, advisor; Chandrasekaran, Venkatacha, advisor; Crouse, David, committee member; Kirby, Michael, committee member
Accurate target tracking is a fundamental requirement of modern automated systems. An accurate tracker must correctly associate new observations to existing tracks and update those tracks to reflect the new information. An accurate tracker is one which predicts assignments and measurement distributions closely matching the ground truth. This work will show that aspects of the GNP algorithm and IMM filter require amendments and renewed investigation. To aid the framing of the solutions in the context of tracking, some general background will be presented first. More specific background will be given prior to the corresponding contributions. Modern sensor networks require the alignment of track pictures from multiple sensors (sometimes called sensor registration). This issue was described in the 1990s and termed the global nearest pattern problem in the early 2000s. The following work presents a correction and extension of the solution to the global nearest pattern problem with a heuristic error estimation algorithm. Its use for sensor calibration is demonstrated. Once measurements have been associated to tracks, there still remain several choices that define the tracking algorithm, one being the filtering algorithm which updates the track state. One common solution for filtering is the interacting multiple model filter which was originally developed in the 1980s. This is essentially a bank of Kalman filters which are weighted and mixed based on a predefined Markov chain. The validity of the assumptions on that Markov chain will be discussed and recommendation for replacing those assumptions with neural networks will be proposed and assessed. Finally, following association of two tracks for a single target, it is necessary to combine their information while respecting the lack of knowledge about correlations between the tracks. Covariance intersection was developed in the 1990s and 2000s for track-to-track fusion when tracks are assumed Gaussian. A generalization of covariance intersection, Chernoff fusion, was developed in the 2000s for handling general track states. A connection made in the literature which allows for direct analysis of the error of Chernoff fusion is used to evaluate the effectiveness of Fibonacci lattices for quasi-Monte Carlo integration solutions required by Chernoff fusion.
Open Access
Iterative matrix completion and topic modeling using matrix and tensor factorizations
(Colorado State University. Libraries, 2021) Kassab, Lara, author; Adams, Henry, advisor; Fosdick, Bailey, committee member; Kirby, Michael, committee member; Peterson, Chris, committee member
With the ever-increasing access to data, one of the greatest challenges that remains is how to make sense out of this abundance of information. In this dissertation, we propose three techniques that take into account underlying structure in large-scale data to produce better or more interpretable results for machine learning tasks. One of the challenges that arise when it comes to analyzing large-scale datasets is missing values in data, which could be challenging to handle without efficient methods. We propose adjusting an iteratively reweighted least squares algorithm for low-rank matrix completion to take into account sparsity-based structure in the missing entries. We also propose an iterative gradient-projection-based implementation of the algorithm, and present numerical experiments showcasing the performance of the algorithm compared to standard algorithms. Another challenge arises while performing a (semi-)supervised learning task on high-dimensional data. We propose variants of semi-supervised nonnegative matrix factorization models and provide motivation for these models as maximum likelihood estimators. The proposed models simultaneously provide a topic model and a model for classification. We derive training methods using multiplicative updates for each new model, and demonstrate the application of these models to document classification (e.g., 20 Newsgroups dataset). Lastly, although many datasets can be represented as matrices, datasets also often arise as high-dimensional arrays, known as higher-order tensors. We show that nonnegative CANDECOMP/PARAFAC tensor decomposition successfully detects short-lasting topics in temporal text datasets, including news headlines and COVID-19 related tweets, that other popular methods such as Latent Dirichlet Allocation and Nonnegative Matrix Factorization fail to fully detect.
Open Access
Kinematic structures, diabatic profiles, and precipitation systems in West Africa during summer 2006
(Colorado State University. Libraries, 2013) Davis, Adam James, author; Johnson, Richard, advisor; Maloney, Eric, committee member; Kirby, Michael, committee member
West Africa is a region characterized by great spatial contrasts in temperature, precipitation, and topography, which combine to create many complex and interesting weather phenomena. In particular, the area is home to a seasonal monsoon, propagating easterly waves, and some of the most intense thunderstorm systems on Earth. These types of events have both local and global effects - precipitation variability has a major bearing on regional water resource issues, while West Africa is also the source of many of the disturbances that develop into tropical cyclones in the North Atlantic Ocean. Unfortunately, atmospheric data has historically been very sparse in West Africa, leading to an incomplete understanding of many of these meteorological features and a corresponding difficulty in modeling them accurately. An exceptional opportunity for improvement on these fronts exists thanks to the African Monsoon Multidisciplinary Analysis (AMMA) field campaign, which collected an unprecedented quantity of observations throughout the region, with the most concentrated effort during the summer of 2006. This work uses a gridded analysis of radiosonde measurements obtained during AMMA and places those observations in the context of AMMA radar data and satellite rainfall estimates to examine the patterns of kinematic and diabatic quantities in West Africa relative to the summer monsoon phase, easterly wave disturbances, precipitation systems, and the diurnal cycle. Many unique aspects of West African weather compared to conditions elsewhere in the tropics are revealed by this study. The meridional transitions related to the West African monsoon comprise the predominant control on the location and intensity of precipitation at seasonal time scales, with variations in convective activity related to the Madden-Julian Oscillation contributing at 25 to 60 day periods. On shorter time scales of two to six days, easterly wave disturbances look to be the principal factor governing the timing of rainfall events, though especially persistent cold pools and residual cloudiness generated by thunderstorm systems also act as constraints on convective evolution on the days following a precipitation episode. One of the most distinctive traits of the study region in West Africa compared to other tropical areas is the particular prevalence of convective downdrafts, chiefly those associated with mesoscale zones of stratiform precipitation in thunderstorm complexes. These features, along with the gravity waves forced by their characteristic heating pattern, have an especially large influence on the time-mean atmospheric structure relative to the majority of the tropics. A comparison of the diabatic profiles from the AMMA dataset with those from other field projects indicates that the signals of both convective downdrafts and diurnal variations of the planetary boundary layer are much stronger in West Africa than in the previously studied regions. Beyond the mentioned differences, though, the AMMA profiles show resemblance to those from both western Pacific and eastern Atlantic field campaigns. The vertical patterns of atmospheric variables tend to be complex and multi-layered in West Africa, suggesting that the area is home to an especially diverse cloud population, with contributions from numerous height regimes prominent enough to influence the mean state. Meridional differences within the domain of the analysis are evident, including indications of more intense convective updrafts toward the north, stronger effects of boundary layer mixing in the north, and a greater net influence of mesoscale convective system downdrafts toward the south. The diurnal cycle of precipitation appears most prominently shaped by convective initiation near areas of high topography and the subsequent development and long-distance propagation of extensive, well-organized thunderstorm systems, though there seem to be effects related to diurnal flow patterns near the Gulf of Guinea coast too. Inland, moisture transport achieved by the nocturnal low-level jet is a key influence on rainfall, with mixing by the daytime boundary layer playing an important function as well. Changes in the relative contribution and intensity of deep convective and stratiform heating and moistening patterns arise among different times of day and night, as the leading precipitation regime transitions from developing deep convection at midday to organizing thunderstorm systems by evening and propagating thunderstorm complexes with extensive stratiform rainfall overnight. The analyses in the present work demonstrate a few different issues and caveats that need to be considered when utilizing observational or remote sensing datasets. Namely, the timing of radiosonde launches and the spacing of the sounding site array combined to create a delay between when convective systems passed the Niamey, Niger measurement site and when their effects were detected in the gridded AMMA sounding data. Similarly, infrared satellite rainfall estimates from the Tropical Rainfall Measuring Mission (TRMM) are shown to have a time lag of about three hours between when precipitation actually occurs and when it appears in the estimate product, complicating the intended use of the data in evaluating the diurnal cycle of rainfall.
Open Access
Long-term learning for adaptive underwater UXO classification
(Colorado State University. Libraries, 2022) Hall, John Joseph, author; Azimi-Sadjadi, Mahmood R., advisor; Pezeshki, Ali, committee member; Luo, J. Rockey, committee member; Kirby, Michael, committee member
Classification of underwater objects such as unexploded ordnances (UXO) and mines from sonar datasets poses a difficult problem. Among factors that complicate classification of these objects are: variations in the operating and environmental conditions, presence of spatially varying clutter, variations in target shape, composition, orientation and burial conditions. Furthermore, collection of large quantities of real and representative data for training and testing in various background conditions is very difficult and impractical in many cases. In this dissertation, we build on our previous work in [1] where sparse-reconstruction based classification models were trained on synthetically generated sonar datasets to perform classification on real datasets. While this earlier work helped address issues of data poverty that are intrinsic to the underwater mine-hunting problem, in this work we change course to focus on the adaptation of such models. Particularly, we investigate approaches to adapting linear and kernelized forms of sparse reconstruction based classifiers (SRCs) to function in a lifelong learning setting in order to perform classification as environmental parameters are constantly evolving, without sacrificing performance on previously encountered environments. In this dissertation, we try to address several key questions for designing robust classifiers for UXO and munitions classification from low frequency sonar in a Lifelong learning setting. These include: (1) What are the most viable mechanisms to allow an unmanned underwater vehicle to accumulate and incorporate novel labeled or un-labeled data into its target identification system without sacrificing performance in old environments? (2) What are the most viable mechanisms for allowing an underwater ATR system to extract class labels despite varying environmental conditions? (3) What are the advantages, shortcomings, and major differences, of compressed-sensing based approaches to target identification, such as the modified MSC with incremental dictionaries, versus popular alternatives such as multi-task learning approaches? (4) How can the modified MSC framework from [1, 2] be extended to allow for kernelized solutions in an efficient manner? In this work, we propose several novel algorithms in order to address the problems of kernelizing compressed-sensing systems and transitioning these systems to an efficient incremental learning that does not depend on the full kernel matrix of all training samples. By kernelizing the sparse reconstruction classifier, the benefits of: sparse representations and non-linear embedding of samples can be coupled. Among the novel algorithms presented in this dissertation include: an incremental linearized kernel embedding (LKE) that leverages Nystrom approximation [3–5] for useful geometric interpretation in the embedded space; A novel algorithm for updating the eigen-decomposition of a growing kernel matrix which leverages fast arrowhead matrix eigendecompositions; and a method for optimizing a custom kernel function for M-ary discrimination tasks. A major technical question that is addressed in this work pertains to whether or not the Matched Subspace Classifier (MSC) [2, 6] can successfully be kernelized and converted into an adaptive form for use in a lifelong learning setting. The comprehensive testing of the incremental kernelized MSC and its application to the classification of munitions using low frequency sonar is another primary objective of this work. To this end, we test the hypothesis that the non-linearly mapped spectral features captured in the acoustic color (AC) data [2,7,8], extracted from the sonar back-scattered from various objects, display unique features providing superior discrimination between different classes of detected objects to the standard features. In this dissertation we present new classification results using three variants of a kernelized MSC, including an incremental linearized kernel embedding (LKE) MSC with uniform and ridge-leverage score (RLS) sampling, along with an incremental version of the linear version of the modified MSC from [2]. These classifier systems are applied to real sonar datasets, namely the TREX13 and PondEX09-10, to test the generalization ability of classifiers whose baseline training is performed on synthetic (i.e model generated) sonar datasets generated by a fast ray model (FRM), also known as the Target in environment response (TIER) model [8, 9]. In the incremental cases, a very limited number of labeled samples are utilized to augment the signal models when moving into a new operating environment. The methods presented here have provided extremely promising results so far, with the incremental LKE based MSC system providing PCC = 94.6%, PFA = 5.4%, and PCC = 99.3%, PFA = 0.7%, when using seven aspects (AC features) per decision, for the TREX13 and PondEX09-10 respectively.
Embargo
Machine learning and deep learning applications in neuroimaging for brain age prediction
(Colorado State University. Libraries, 2023) Vafaei, Fereydoon, author; Anderson, Charles, advisor; Kirby, Michael, committee member; Blanchard, Nathaniel, committee member; Burzynska, Agnieszka, committee member
Machine Learning (ML) and Deep Learning (DL) are now considered as state-of-the-art assistive AI technologies that help neuroscientists, neurologists and medical professionals with early diagnosis of neurodegenerative diseases and cognitive decline as a consequence of unhealthy brain aging. Brain Age Prediction (BAP) is the process of estimating a person's biological age using Neuroimaging data, and the difference between the predicted age and the subject's chronological age, known as Delta, is regarded as a biomarker for healthy versus unhealthy brain aging. Accurate and efficient BAP is an important research topic, and hence ML/DL methods have been developed for this task. There are different modalities of Neuroimaging such as Magnetic Resonance Imaging (MRI) that have been used for BAP in the past. Diffusion Tensor Imaging (DTI) is an advanced quantitative Neuroimaging technology that gives insight into microstructure of White Matter tracts that connect different parts of the brain to function properly. DTI data is high-dimensional, and age-related microstructural changes in White Matter include non-linear patterns. In this study, we perform a series of analytical experiments using ML and DL methods to investigate the applicability of DTI data for BAP. We also investigate which Diffusivity Parameters, which are DTI metrics that reflect direction and magnitude of diffusion of water molecules in the brain, are relevant for BAP as a Supervised Learning task. Moreover, we propose, implement, and analyze a novel methodology that can detect age-related anomalies (high Deltas), and can overcome some of the major and fundamental limitations of the current supervised approach for BAP, such as "Chronological Age Label Inconsistency". Our proposed methodology, which combines Unsupervised Anomaly Detection (UAD) and supervised BAP, focuses on addressing a fundamental challenge in BAP which is how to interpret a model's error. Should a researcher interpret a model's error as an indication of unhealthy brain aging or the model's poor performance that should be eliminated? We argue that the underlying cause of this problem is the inconsistency of chronological age labels as the ground truth of the Supervised Learning task, which is the common basis of training ML/DL models. Our Unsupervised Learning methods and findings open a new possibility to detect irregularities and abnormalities in the aging brain using DTI scans, independent of inconsistent chronological age labels. The results of our proposed methodology show that combining label-independent UAD and supervised BAP provides a more reliable and methodical way for error analysis than the current supervised BAP approach when it is used in isolation. We also provide visualization and explanations on how our ML/DL methods make their decisions for BAP. Explainability and generalization of our ML/DL models are two important aspects of our study.
Open Access
Mixture of factor models for joint dimensionality reduction and classification
(Colorado State University. Libraries, 2016) Tabaghi, Puoya, author; Azimi-Sadjadi, Mahmood R., advisor; Scharf, Louis L., advisor; Pezeshki, Ali, committee member; Kirby, Michael, committee member
In many areas such as machine learning, pattern recognition, information retrieval, and data mining one is interested in extracting a low-dimensional data that is truly representative of the properties of the original high dimensional data. For example, one application could be extracting representative low-dimensional features of underwater objects from sonar imagery suitable for detection and classiﬁcation. This is a diﬃcult problem due to various factors such as variations in the operating and environmental conditions, presence of spatially varying clutter, and variations in object shapes, compositions, and orientation. The goal of this work is to develop a novel probabilistic method using a mixture of factor models for simultaneous nonlinear dimensionality reduction and classiﬁcation. The framework used here is inspired by the work in [1] which uses a mixture of local PCA projections leading to an unsupervised nonlinear dimensionality reduction algorithm. In contrast, the proposed method provides a supervised probabilistic approach suitable for analyzing labeled high-dimensional data with complex structures by exploiting a set of low-dimensional latent variables which are both discriminative and generative. With the aid of these low-dimensional latent variables, a mixture of linear models is introduced to represent the high-dimensional data. An optimum linear classiﬁer is then built in the latent variable-domain to separate the support of the latent variable associated with each class. Introducing these hidden variables allow us to derive the joint probability density function of the data and class label, reduce data dimension and perform clustering, classiﬁcation and parameter estimation. This probabilistic approach provides a mechanism to traverse between the input space and latent (feature) space and vice versa as well as cluster and classify data. A supervised training based on the Expectation-Maximization (EM) and steepest descent algorithms is then introduced to derive the ML estimates of the unknown parameters. It is shown that parameters associated with dimensionality reduction can be estimated using the EM algorithm whereas those of the classiﬁer are estimated using the steepest descent method. The introduction of latent variables not only helps to represent the pdf of data and reduce the dimension of them but also in parameter estimation using EM algorithm which is used to ﬁnd ML estimates of the parameters when the available data is incomplete. A comprehensive study is carried out to assess the performance of the proposed method using two diﬀerent data sets. The ﬁrst data set consists of Synthetic Aperture Sonar (SAS) images of model-generated underwater objects superimposed on background clutter. These images correspond to two diﬀerent object types namely Cylinder (mine-like) and Block (non-mine-like). The signatures of each object are synthetically generated and are placed at various aspect angles from 1 to 180 degrees for each object type. The goal of our classiﬁer is to assign non-target versus target labels to these image snippets. The other data set consists of two sets of facial images of diﬀerent individuals. Each image set contains 2 series of 93 images of the same person at diﬀerent poses. The goal of the classiﬁer for this case is to identify each indi vidual correctly. The dimensionality reduction performance of the proposed method is compared to two relevant dimensionality reduction methods, namely Probabilistic PCA [2] and Mixture of Probabilistic PCA (MPPCA) [1] while its classiﬁcation performance is benchmarked against a Support Vector Machine (SVM). The results on both data sets indicate promising dimensionality reduction and reconstruction capabilities compared to PPCA/MPPCA methods. On the other hand, classiﬁcation performance is competitive with SVM when the data is linearly separable.
Open Access
Multidimensional scaling: infinite metric measure spaces
(Colorado State University. Libraries, 2019) Kassab, Lara, author; Adams, Henry, advisor; Kirby, Michael, committee member; Fosdick, Bailey, committee member
Multidimensional scaling (MDS) is a popular technique for mapping a finite metric space into a low-dimensional Euclidean space in a way that best preserves pairwise distances. We study a notion of MDS on infinite metric measure spaces, along with its optimality properties and goodness of fit. This allows us to study the MDS embeddings of the geodesic circle S1 into Rm for all m, and to ask questions about the MDS embeddings of the geodesic n-spheres Sn into Rm. Furthermore, we address questions on convergence of MDS. For instance, if a sequence of metric measure spaces converges to a fixed metric measure space X, then in what sense do the MDS embeddings of these spaces converge to the MDS embedding of X? Convergence is understood when each metric space in the sequence has the same finite number of points, or when each metric space has a finite number of points tending to infinity. We are also interested in notions of convergence when each metric space in the sequence has an arbitrary (possibly infinite) number of points.
Open Access
Policy optimization for industrial benchmark using deep reinforcement learning
(Colorado State University. Libraries, 2020) Kumar, Anurag, author; Anderson, Charles, advisor; Chitsaz, Hamid, committee member; Kirby, Michael, committee member
Significant advancements have been made in the field of Reinforcement Learning (RL) in recent decades. Numerous novel RL environments and algorithms are mastering these problems that have been studied, evaluated, and published. The most popular RL benchmark environments produced by OpenAI Gym and DeepMind Labs are modeled after single/multi-player board, video games, or single-purpose robots and the RL algorithms modeling optimal policies for playing those games have even outperformed humans in almost all of them. However, the real-world applications using RL is very limited, as the academic community has limited access to real industrial data and applications. Industrial Benchmark (IB) is a novel RL benchmark motivated by Industrial Control problems with properties such as continuous state and action spaces, high dimensionality, partially observable state space, delayed effects combined with complex heteroscedastic stochastic behavior. We have used Deep Reinforcement Learning (DRL) algorithms like Deep Q-Networks (DQN) and Double-DQN (DDQN) to study and model optimal policies on IB. Our empirical results show various DRL models outperforming previously published models on the same IB.
Open Access
Resource allocation for wildland fire suppression planning using a stochastic program
(Colorado State University. Libraries, 2011) Masarie, Alex Taylor, author; Rideout, Douglas, advisor; Bevers, Michael, committee member; Kirby, Michael, committee member
Resource allocation for wildland fire suppression problems, referred to here as Fire-S problems, have been studied for over a century. Not only have the many variants of the base Fire-S problem made it such a durable one to study, but advances in suppression technology and our ever-expanding knowledge of and experience with wildland fire behavior have required almost constant reformulations that introduce new techniques. Lately, there has been a strong push towards randomized or stochastic treatments because of their appeal to fire managers as planning tools. A multistage stochastic program with variable recourse is proposed and explored in this paper as an answer to a single-fire planning version of the Fire-S problem. The Fire-S stochastic program is discretized for implementation according to scenario trees, which this paper supports as a highly useful tool in the stochastic context. Our Fire-S model has a high level of complexity and is parameterized with a complicated hierarchical cluster analysis of historical weather data. The cluster analysis has some incredibly interesting features and stands alone as an interesting technique apart from its application as a parameterization tool in this paper. We critique the planning model in terms of its complexity and options for an operational version are discussed. Although we assume no interaction between fire spread and suppression resources, the possibility of incorporating such an interaction to move towards an operational, stochastic model is outlined. A suppression budget analysis is performed and the familiar "production function" fire suppression curve is created, which strongly indicates the Fire-S model performs in accordance with fire economic theory as well as its deterministic counterparts. Overall, this exploratory study demonstrates a promising future for the existence of tractable stochastic solutions to all variants of Fire-S problems.
Open Access
Revealing and analyzing the shared structure of deep face embeddings
(Colorado State University. Libraries, 2022) McNeely-White, David G., author; Beveridge, J. Ross, advisor; Blanchard, Nathaniel, committee member; Kirby, Michael, committee member; Peterson, Chris, committee member
Deep convolutional neural networks trained for face recognition are found to output face embeddings which share a fundamental structure. More specifically, one face verification model's embeddings (i.e. last--layer activations) can be compared directly to another model's embeddings after only a rotation or linear transformation, with little performance penalty. If only rotation is required to convert the bulk of embeddings between models, there is a strong sense in which those models are learning the same thing. In the most recent experiments, the structural similarity (and dissimilarity) of face embeddings is analyzed as a means of understanding face recognition bias. Bias has been identified in many face recognition models, often analyzed using distance measures between pairs of faces. By representing groups of faces as groups, and comparing them as groups, this shared embedding structure can be further understood. Specifically, demographic-specific subspaces are represented as points on a Grassmann manifold. Across 10 models, the geodesic distances between those points are expressive of demographic differences. By comparing how different groups of people are represented in the structure of embedding space, and how those structures vary with model designs, a new perspective on both representational similarity and face recognition bias is offered.