Publications

Permanent URI for this collection

https://hdl.handle.net/10217/239510

Browse

Now showing 1 - 20 of 23

Open Access
Trust-aware planning: modeling trust evolution in iterated human-robot interaction
(Colorado State University. Libraries, 2023-03-13) Zahedi, Zahra, author; Verma, Mudit, author; Sreedharan, Sarath, author; Kambhampati, Subbarao, author; ACM, publisher
Trust between team members is an essential requirement for any successful cooperation. Thus, engendering and maintaining the fellow team members' trust becomes a central responsibility for any member trying to not only successfully participate in the task but to ensure the team achieves its goals. The problem of trust management is particularly challenging in mixed human-robot teams where the human and the robot may have diferent models about the task at hand and thus may have diferent expectations regarding the current course of action, thereby forcing the robot to focus on the costly explicable behavior.We propose a computational model for capturing and modulating trust in such iterated human-robot interaction settings, where the human adopts a supervisory role. In our model, the robot integrates human's trust and their expectations about the robot into its planning process to build and maintain trust over the interaction horizon. By establishing the required level of trust, the robot can focus on maximizing the team goal by eschewing explicit explanatory or explicable behavior without worrying about the human supervisor monitoring and intervening to stop behaviors they may not necessarily understand. We model this reasoning about trust levels as a meta reasoning process over individual planning tasks. We additionally validate our model through a human subject experiment.
Open Access
Claim extraction and dynamic stance detection in COVID-19 tweets
(Colorado State University. Libraries, 2023-04-30) Faramarzi, Noushin Salek, author; Chaleshtori, Fateme Hashemi, author; Shirazi, Hossein, author; Ray, Indrakshi, author; Banerjee, Ritwik, author; ACM, publisher
The information ecosystem today is noisy, and rife with messages that contain a mix of objective claims and subjective remarks or reactions. Any automated system that intends to capture the social, cultural, or political zeitgeist, must be able to analyze the claims as well as the remarks. Due to the deluge of such messages on social media, and their tremendous power to shape our perceptions, there has never been a greater need to automate these analyses, which play a pivotal role in fact-checking, opinion mining, understanding opinion trends, and other such downstream tasks of social consequence. In this noisy ecosystem, not all claims are worth checking for veracity. Such a check-worthy claim, moreover, must be accurately distilled from subjective remarks surrounding it. Finally, and especially for understanding opinion trends, it is important to understand the stance of the remarks or reactions towards that specific claim. To this end, we introduce a COVID-19 Twitter dataset, and present a three-stage process to (i) determine whether a given Tweet is indeed check-worthy, and if so, (ii) which portion of the Tweet ought to be checked for veracity, and finally, (iii) determine the author's stance towards the claim in that Tweet, thus introducing the novel task of topic-agnostic stance detection.
Open Access
SAFE-PASS: stewardship, advocacy, fairness and empowerment in privacy, accountability, security, and safety for vulnerable groups
(Colorado State University. Libraries, 2023-05-24) Ray, Indrajit, author; Thuraisingham, Bhavani, author; Vaidya, Jaideep, author; Mehrotra, Sharad, author; Atluri, Vijayalakshmi, author; Ray, Indrakshi, author; Kantarcioglu, Murat, author; Raskar, Ramesh, author; Salimi, Babak, author; Simske, Steve, author; Venkatasubramanian, Nalini, author; Singh, Vivek, author; ACM, publisher
Our vision is to achieve societally responsible secure and trustworthy cyberspace that puts algorithmic and technological checks and balances on the indiscriminate sharing and analysis of data. We achieve this vision in a holistic manner by framing research directions with four major considerations: (i) Expanding knowledge and understanding of security and privacy perceptions and expectations in vulnerable groups, which significantly contribute to their unwillingness to share data, and use that knowledge to drive research in (a) mitigating missing/imbalanced data problems, (b) understanding and modeling security and privacy risks of data sharing, and (c) modeling utility of data sharing. (ii) Developing a risk-adaptive, policy model capable of capturing and articulating security and privacy expectations of users that are relevant in a particular context and develops associated technology to ensure provenance and accountability. (iii) Developing robust AI/ML algorithms that are transparent and explainable with respect to fairness and bias to reduce/eliminate discrimination, misuse, privacy violations, or other cyber-crimes. (iv) Developing models and techniques for a nuanced, contextually adaptive, and graded privacy paradigm that allows trade-offs between privacy and utility. Towards this, in this paper we present the SAFE-PASS framework to provide Stewardship, Advocacy, Fairness and Empowerment in Privacy, Accountability, Security, and Safety for Vulnerable Groups.
Open Access
Synthesizing and analyzing attribute-based access control model generated from natural language policy statements
(Colorado State University. Libraries, 2023-05-24) Abdelgawad, Mahmoud, author; Ray, Indrakshi, author; Alqurashi, Saja, author; Venkatesha, Videep, author; Shirazi, Hosein, author; ACM, publisher
Access control policies (ACPs) are natural language statements that describe criteria under which users can access resources. We focus on constructing NIST Next Generation Access Control (NGAC) ABAC model from ACP statements. NGAC is more complex than RBAC or XACML ABAC as it supports dynamic, event-based policies, as well as prohibitions. We provide algorithms that use spaCy, a NLP library, to extract entities and relations from ACP sentences and convert them into the NGAC model. We then convert this NGAC model into Neo4j representation for the purpose of analysis. We apply the approach to various real-world ACP datasets to demonstrate the feasibility and assess scalability. We demonstrate that the approach is scalable and effectively extracts the NGAC ABAC model from large ACP datasets. We also show that redundancies and inconsistencies of ACP sentences are often found in unclean datasets.
Open Access
Sparse binary transformers for multivariate time series modeling
(Colorado State University. Libraries, 2023-08-04) Gorbett, Matt, author; Shirazi, Hossein, author; Ray, Indrakshi, author; ACM, publisher
Compressed Neural Networks have the potential to enable deep learning across new applications and smaller computational environments. However, understanding the range of learning tasks in which such models can succeed is not well studied. In this work, we apply sparse and binary-weighted Transformers to multivariate time series problems, showing that the lightweight models achieve accuracy comparable to that of dense floating-point Transformers of the same structure. Our model achieves favorable results across three time series learning tasks: classification, anomaly detection, and single-step forecasting. Additionally, to reduce the computational complexity of the attention mechanism, we apply two modifications, which show little to no decline in model performance: 1) in the classification task, we apply a fixed mask to the query, key, and value activations, and 2) for forecasting and anomaly detection, which rely on predicting outputs at a single point in time, we propose an attention mask to allow computation only at the current time step. Together, each compression technique and attention modification substantially reduces the number of non-zero operations necessary in the Transformer. We measure the computational savings of our approach over a range of metrics including parameter count, bit size, and floating point operation (FLOPs) count, showing up to a 53x reduction in storage size and up to 10.5x reduction in FLOPs.
Open Access
Exploring unimodal notification interaction and display methods in augmented reality
(Colorado State University. Libraries, 2023-10-09) Plabst, Lucas, author; Raikwar, Aditya, author; Oberdörfer, Sebastian, author; Ortega, Francisco, author; Niebling, Florian, author; ACM, publisher
As we develop computing platforms for augmented reality (AR) head-mounted display (HMDs) technologies for social or workplace environments, understanding how users interact with notifications in immersive environments has become crucial. We researched effectiveness and user preferences of different interaction modalities for notifications, along with two types of notification display methods. In our study, participants were immersed in a simulated cooking environment using an AR-HMD, where they had to fulfill customer orders. During the cooking process, participants received notifications related to customer orders and ingredient updates. They were given three interaction modes for those notifications: voice commands, eye gaze and dwell, and hand gestures. To manage multiple notifications at once, we also researched two different notification list displays, one attached to the user's hand and one in the world. Results indicate that participants preferred using their hands to interact with notifications and having the list of notifications attached to their hands. Voice and gaze interaction was perceived as having lower usability than touch.
Open Access
Paying attention to wildfire: using U-Net with attention blocks on multimodal data for next day prediction
(Colorado State University. Libraries, 2023-10-09) Fitzgerald, Jack, author; Seefried, Ethan, author; Yost, James, author; Pallickara, Sangmi, author; Blanchard, Nathaniel, author; ACM, publisher
Predicting where wildfires will spread provides invaluable information to firefighters and scientists, which can save lives and homes. However, doing so requires a large amount of multimodal data e.g., accurate weather predictions, real-time satellite data, and environmental descriptors. In this work, we utilize 12 distinct features from multiple modalities in order to predict where wildfires will spread over the next 24 hours. We created a custom U-Net architecture designed to train as efficiently as possible, while still maximizing accuracy, to facilitate quickly deploying the model when a wildfire is detected. Our custom architecture demonstrates state-of-the-art performance and trains an order of magnitude more quickly than prior work, while using fewer computational resources. We further evaluated our architecture with an ablation study to identify which features were key for prediction and which provided negligible impact on performance.
Open Access
A methodology for evaluating multimodal referring expression generation for embodied virtual agents
(Colorado State University. Libraries, 2023-10-09) Alalyani, Nada, author; Krishnaswamy, Nikhil, author; ACM, publisher
Robust use of definite descriptions in a situated space often involves recourse to both verbal and non-verbal modalities. For IVAs, virtual agents designed to interact with humans, the ability to both recognize and generate non-verbal and verbal behavior is a critical capability. To assess how well an IVA is able to deploy multimodal behaviors, including language, gesture, and facial expressions, we propose a methodology to evaluate the agent's capacity to generate object references in a situational context, using the domain of multimodal referring expressions as a use case. Our contributions include: 1) developing an embodied platform to collect human referring expressions while communicating with the IVA. 2) comparing human and machine-generated references in terms of evaluable properties using subjective and objective metrics. 3) reporting preliminary results from trials that aimed to check whether the agent can retrieve and disambiguate the object the human referred to, if the human has the ability to correct misunderstanding using language, deictic gesture, or both; and human ease of use while interacting with the agent.
Open Access
Teach students to study using quizzes, study behavior visualization, and reflection: a case study in an introduction to programming course
(Colorado State University. Libraries, 2024-01-15) Moraes, Marcia C., author; Lionelle, Albert, author; Ghosh, Sudipto, author; Folkestad, James E., author; ACM, publisher
Due to a long history of using rote memorization and rereading as the primary means to study, students are coming to the University with misconceptions about study strategies that are beneficial for their performance and long-term learning. Techniques such as spaced retrieval practice, interleaving, and metacognition are proven by cognitive and educational researchers as strategies that greatly improve learning. They focus on helping students to own responsibility for their learning and retention of information. Considering their benefits, quizzes were re-branded to be formative low-stakes retrieval practice activities (RPAs) in an Introduction to Programming Course (CS1), meaning that students would use the quizzes as learning tools, testing themselves in a spaced and interleaved manner as many times as they want during the semester. Additionally, the U-Behavior learning and teaching method was used. This method applies visualizations of student's study habits and self-reflections to help students to be aware of their study practices, reflect on them, and change their study routine to improve performance and long-term learning. Study behaviors were analyzed and the final Canvas exam, final coding exam, and final course grades were compared for students who spaced and interleaved their practice with students who did not. Results showed a statistically significant increase in all grades evaluated for students who practiced using this novel combination of spacing and interleaving integrated with U-Behavior visualizations and RPA reflection activities for learning.
Open Access
Formal verification of source-to-source transformations for HLS
(Colorado State University. Libraries, 2024-04-02) Pouchet, Louis-Noël, author; Tucker, Emily, author; Zhang, Niansong, author; Chen, Hongzheng, author; Pal, Debjit, author; Rodríguez, Gabriel, author; Zhang, Zhiru, author; ACM, publisher
High-level synthesis (HLS) can greatly facilitate the description of complex hardware implementations, by raising the level of abstraction up to a classical imperative language such as C/C++, usually augmented with vendor-specific pragmas and APIs. Despite productivity improvements, attaining high performance for the final design remains a challenge, and higher-level tools like source-to-source compilers have been developed to generate programs targeting HLS toolchains. These tools may generate highly complex HLS-ready C/C++ code, reducing the programming effort and enabling critical optimizations. However, whether these HLS-friendly programs are produced by a human or a tool, validating their correctness or exposing bugs otherwise remains a fundamental challenge. In this work we target the problem of efficiently checking the semantics equivalence between two programs written in C/C++ as a means to ensuring the correctness of the description provided to the HLS toolchain, by proving an optimized code version fully preserves the semantics of the unoptimized one. We introduce a novel formal verification approach that combines concrete and abstract interpretation with a hybrid symbolic analysis. Notably, our approach is mostly agnostic to how control-flow, data storage, and dataflow are implemented in the two programs. It can prove equivalence under complex bufferization and loop/syntax transformations, for a rich class of programs with statically interpretable control-flow. We present our techniques and their complete end-to-end implementation, demonstrating how our system can verify the correctness of highly complex programs generated by source-to-source compilers for HLS, and detect bugs that may elude co-simulation.
Open Access
A framework for profiling spatial variability in the performance of classification models
(Colorado State University. Libraries, 2024-04-03) Warushavithana, Menuka, author; Barram, Kassidy, author; Carlson, Caleb, author; Mitra, Saptashwa, author; Ghosh, Sudipto, author; Breidt, Jay, author; Pallickara, Sangmi Lee, author; Pallickara, Shrideep, author; ACM, publisher
Scientists use models to further their understanding of phenomena and inform decision-making. A confluence of factors has contributed to an exponential increase in spatial data volumes. In this study, we describe our methodology to identify spatial variation in the performance of classification models. Our methodology allows tracking a host of performance measures across different thresholds for the larger, encapsulating spatial area under consideration. Our methodology ensures frugal utilization of resources via a novel validation budgeting scheme that preferentially allocates observations for validations. We complement these efforts with a browser-based, GPU-accelerated visualization scheme that also incorporates support for streaming to assimilate validation results as they become available.
Open Access
RUBIKS: rapid explorations and summarization over high dimensional spatiotemporal datasets
(Colorado State University. Libraries, 2024-04-03) Mitra, Saptashwa, author; Young, Matt, author; Breidt, Jay, author; Pallickara, Sangmi, author; Pallickara, Shrideep, author; ACM, publisher
Exponential growth in spatial data volumes have occurred alongside increases in the dimensionality of datasets and the rates at which observations are generated. Rapid summarization and explorations of such datasets are a precursor to several downstream operations including data wrangling, preprocessing, hypothesis formulation, and model construction among others. However, researchers are stymied both by the dimensionality and data volumes that often entail extensive data movements, computation overheads, and I/O. Here, we describe our methodology to support effective summarizations and explorations at scale over arbitrary spatiotemporal scopes, which encapsulate the spatial extents, temporal bounds, or combinations thereof over the data space of interest. Summarizations can be performed over all variables representing the dataspace or subsets specified by the user. We extend the concept of data cubes to encompass spatiotemporal datasets with high-dimensionality and where there might be significant gaps in the data because measurements (or observations) of diverse variables are not synchronized and may occur at diverse rates. We couple our data summarization features with a rapid Choropleth visualizer that allows users to explore spatial variations of diverse measures of interest. We validate these concepts in the context of an Environmental Protection Agency dataset which tracks over 4000 chemical pollutants, presenting in natural water sources across the United States from 1970 onwards.
Open Access
An artists' perspectives on natural interactions for virtual reality 3D sketching
(Colorado State University. Libraries, 2024-05-11) Rodriguez, Richard, author; Sullivan, Brian T., author; Machuca, Mayra Donaji Barrera, author; Batmaz, Anil Ufuk, author; Tornatzky, Cyane, author; Ortega, Francisco R., author; ACM, publisher
Virtual Reality (VR) applications like OpenBrush offer artists access to 3D sketching tools within the digital 3D virtual space. These 3D sketching tools allow users to "paint" using virtual digital strokes that emulate real-world mark-making. Yet, users paint these strokes through (unimodal) VR controllers. Given that sketching in VR is a relatively nascent field, this paper investigates ways to expand our understanding of sketching in virtual space, taking full advantage of what an immersive digital canvas offers. Through a study conducted with the participation of artists, we identify potential methods for natural multimodal and unimodal interaction techniques in 3D sketching. These methods demonstrate ways to incrementally improve existing interaction techniques and incorporate artistic feedback into the design.
Open Access
Fast and scalable monitoring for value-freeze operator augmented signal temporal logic
(Colorado State University. Libraries, 2024-05-14) Ghorbel, Bassem, author; Prabhu, Vinayak S., author; ACM, publisher
Signal Temporal Logic (STL) is a timed temporal logic formalism that has found widespread adoption for rigorous specification of properties in Cyber-Physical Systems. However, STL is unable to specify oscillatory properties commonly required in engineering design. This limitation can be overcome by the addition of additional operators, for example, signal-value freeze operators, or with first order quantification. Previous work on augmenting STL with such operators has resulted in intractable monitoring algorithms. We present the first efficient and scalable offline monitoring algorithms for STL augmented with independent freeze quantifiers. Our final optimized algorithm has a |ρ|log(|ρ|) dependence on the trace length |ρ| for most traces ρ arising in practice, and a |ρ|2 dependence in the worst case. We also provide experimental validation of our algorithms – we show the algorithms scale to traces having 100k time samples.
Open Access
Methodology for resiliency analysis of mission-critical systems
(Colorado State University. Libraries, 2024-05-21) Abdelgawad, Mahmoud, author; Ray, Indrakshi, author; ACM, publisher
Mission-critical systems ensure the safety and security of any nation. Attacks on mission-critical systems can have devastating consequences. We need to design missions that can prevent, detect, survive, recover, and respond to faults and cyber attacks. In other words, we must design missions that are cyber-resilient. System engineering techniques must be used to specify, analyze, and understand where adverse events are possible and how to mitigate them while a mission-critical system is deployed. This work introduces an end-to-end methodology for designing cyber-resilient mission-critical systems. The methodology first specifies a mission in the form of a workflow. It then converts the mission workflow into formal representation using Coloured Petri Nets (CPN). The methodology also derives threat models from the mission specification. The threat models are used to form a formal specification of attacks that can be represented in CPN. These CPN attacks are plugged into potential places in the CPN mission to design various attack scenarios. The methodology finally verifies the state transitions of the CPN mission attached to attacks to analyze the resiliency of the mission. It identifies in which state transition the mission succeeds, fails, and is incomplete. The methodology is applied to a drone surveillance system as a motivating example. The result shows that the methodology is practical for resiliency analysis of mission-critical systems. The methodology demonstrates how to restrict a mission to improve the resiliency of mission-critical systems. The methodology provides crucial insights in the early stages of mission specification to achieve cyber resiliency.
Open Access
The restorative influence of virtual reality environment design
(Colorado State University. Libraries, 2024-08-30) Nicoly, Jalynn Blu, author; Masters, Rachel, author; Gaddy, Vidya, author; Interrante, Victoria, author; Ortega, Francisco, author; ACM, publisher
Virtual reality (VR) could support the need for easily accessible therapeutic techniques, such as viewing art and immersing oneself in nature. Our study searches for the optimal virtual environment (VE) by exploring whether beauty in moving and still VEs contributes to stress reduction and perceived restorativeness. We hypothesized that the moving forest environment would result in the most stress reduction, while the abstract art would result in the least, with additional comparisons to a still forest environment and a control condition. The control condition took place outside the virtual headset to simulate what stress reduction would look like without a nature intervention. After working with 78 participants, we found an increase in statistical significance for stress reduction and perceived restorativeness in the moving forest condition compared to the control, as measured by the Zuckerman Inventory of Personal Reactions (ZIPERS) positive affect and the Perceived Restorativeness Scale (PRS). Additionally, the PRS and heart rate measures showed greater restorativeness in the moving forest condition than in the abstract art condition. Heart rate measures also showed statistical significance between the forest image condition and the control and moving forest conditions.
Open Access
Tiled bit networks: sub-bit neural network compression through reuse of learnable binary vectors
(Colorado State University. Libraries, 2024-10-21) Gorbett, Matt, author; Shirazi, Hossein, author; Ray, Indrakshi, author; ACM, publisher
Binary Neural Networks (BNNs) enable efficient deep learning by saving on storage and computational costs. However, as the size of neural networks continues to grow, meeting computational requirements remains a challenge. In this work, we propose a new form of quantization to tile neural network layers with sequences of bits to achieve sub-bit compression of binary-weighted neural networks. The method learns binary vectors (i.e. tiles) to populate each layer of a model via aggregation and reshaping operations. During inference, the method reuses a single tile per layer to represent the full tensor. We employ the approach to both fully-connected and convolutional layers, which make up the breadth of space in most neural architectures. Empirically, the approach achieves near full-precision performance on a diverse range of architectures (CNNs, Transformers, MLPs) and tasks (classification, segmentation, and time series forecasting) with up to an 8x reduction in size compared to binary-weighted models. We provide two implementations for Tiled Bit Networks: 1) we deploy the model to a microcontroller to assess its feasibility in resource-constrained environments, and 2) a GPU-compatible inference kernel to facilitate the reuse of a single tile per layer in memory.
Open Access
Attacks and defenses for large language models on coding tasks
(Colorado State University. Libraries, 2024-10-27) Zhang, Chi, author; Wang, Zifan, author; Zhao, Ruoshi, author; Mangal, Ravi, author; Fredrikson, Matt, author; Jia, Limin, author; Pasareanu, Corina, author; ACM, publisher
Modern large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities for coding tasks, including writing and reasoning about code. They improve upon previous neural network models of code, such as code2seq or seq2seq, that already demonstrated competitive results when performing tasks such as code summarization and identifying code vulnerabilities. However, these previous code models were shown vulnerable to adversarial examples, i.e., small syntactic perturbations designed to "fool" the models. In this paper, we first aim to study the transferability of adversarial examples, generated through white-box attacks on smaller code models, to LLMs. We also propose a new attack using an LLM to generate the perturbations. Further, we propose novel cost-effective techniques to defend LLMs against such adversaries via prompting, without incurring the cost of retraining. These prompt-based defenses involve modifying the prompt to include additional information, such as examples of adversarially perturbed code and explicit instructions for reversing adversarial perturbations. Our preliminary experiments show the effectiveness of the attacks and the proposed defenses on popular LLMs such as GPT-3.5 and GPT-4.
Open Access
DeepSoil: a science-guided framework for generating high precision soil moisture maps by reconciling measurement profiles across in-situ and remote sensing
(Colorado State University. Libraries, 2024-10-29) Khandelwal, Paahuni, author; Pallickara, Sangmi Lee, author; Pallickara, Shrideep, author; ACM, publisher
Soil moisture plays a critical role in several domains and can be used to inform decision-making in agricultural settings, drought forecasting, forest fire predictions, and water conservation. Soil moisture is measured using in-situ and remote-sensing equipment. Depending on the type of equipment that is used, some challenges must be reconciled, including the density of observations, the measurement precision, and the resolutions at which these measurements are available. In particular, in-situ measurements are high-precision but sparse, while remote sensing measurements benefit from spatial coverage, albeit at lower precision and coarser resolutions. The crux of this study is to produce higher-precision soil moisture estimates at high resolutions (30m). Our methodology combines scientific models, deep networks, topographical characteristics, and information about ambient conditions alongside both in-situ and remote sensing data to accomplish this. Domain science infuses several aspects of our methodology. Our empirical benchmarks profile several aspects and demonstrate that our methodology accounts for spatial variability while accounting for both static (soil properties and elevation) and dynamically varying phenomena to generate accurate, high-precision 30m resolution soil moisture content maps.
Open Access
The impact of nature realism on the restorative quality of virtual reality forest bathing
(Colorado State University. Libraries, 2024-11) Masters, Rachel, author; Nicoly, Jalynn, author; Gaddy, Vidya, author; Interrante, Victoria, author; Ortega, Francisco, author; ACM, publisher
Virtual reality (VR) forest bathing for stress relief and mental health has recently become a popular research topic. As people spend more of their lives indoors and have less access to the restorative benefit of nature, having a VR nature supplement has the potential to improve quality of life. However, the optimal design of VR nature environments is an active area of investigation with many research questions to be explored. One major issue with VR is the difficulty of rendering high-fidelity assets in real time without causing cybersickness, or VR motion sickness, within the headset. Due to this limitation, we investigate if the realism of VR nature is critical for the restorative effects by comparing a low-realism nature environment to a high-realism nature environment. We only found a significant difference in the perceived restorativeness of the two environments, but after observing trends in our data toward the stress reduction potential of the high-realism environment, we suggest exploring more varieties of high and low-realism environments in future work to investigate the full potential of VR and how people respond.