Repository logo
 

Silicon photonic hardware accelerators for transformers and graph neural networks

Simple item page
dc.contributor.authorAfifi, Salma, author
dc.contributor.authorPasricha, Sudeep, advisor
dc.contributor.authorNikdast, Mahdi, committee member
dc.contributor.authorMalaiya, Yashwant, committee member
dc.date.accessioned2023-08-28T10:28:03Z
dc.date.available2023-08-28T10:28:03Z
dc.date.issued2023
dc.description.abstractThe rapid growth of artificial intelligence (AI) applications has revolutionized the way we process data, make decisions, and interact with machines. Specifically, artificial neural networks (ANNs) have significantly evolved and now encompass various advanced neural networks such as transformers and graph neural networks (GNNs). This has enabled the development of innovative AI applications that can transform several industries, including healthcare, recommendation systems, and robotics. Transformer and transformer-based neural networks have outperformed multiple ANNs, such as convolution neural networks (CNNs) and recurrent neural networks (RNNs), across many natural language processing (NLP) tasks. Moreover, transformers are currently being integrated into vision tasks through using the vision transformer model (ViT). Similarly, GNNs have witnessed a surge of advancements over the past few years and have established their proficiency in dealing with graph-structured data. Nevertheless, each of these neural networks imposes unique challenges, hindering their inference and usage in resource-constrained systems. For instance, the transformer model's size, number of parameters, and complexity of operations lead to long inference times, large memory footprint, and low computation-to-memory ratio. On the other hand, GNNs inference challenges are due to their dense and very sparse computations. Additionally, the wide variety of possible input graphs structure and algorithms dictate the need for a system capable of efficiently adapting their execution and operations to the specific graph structure and effectively scaling to extremely large graphs. Accordingly, conventional computing processors and ANN accelerators are not tailored to cater for such challenges, and using them to accelerate transformers and GNN execution can be highly inefficient. ii Furthermore, the utilization of traditional electronic accelerators entails a number of limitations, including escalating fabrication costs due to low yields and diminishing performance improvements, associated with semiconductor-technology scaling. This has led researchers to start investigating other technologies for ANN acceleration such as silicon photonics which enables performing complex operations in the optical domain with low energy consumption and at very high throughput. While several hardware accelerators leveraging silicon photonics have been presented for networks such as CNNs, none have been customized for emerging complex neural networks such as transformers and GNNs. Due to the various challenges associated with each of these networks, designing reliable and efficient inference hardware accelerators for transformers and GNNs is a non-trivial problem. This thesis introduces two novel silicon-photonic-based hardware architectures for energy efficient and high throughput inference acceleration. As our first contribution, we propose a non-coherent silicon photonic hardware accelerator for transformer neural networks, called TRON. We demonstrate how TRON is able to accommodate a wide range of transformer and transformer-based neural networks while surpassing GPU, CPU, TPU, and several state-of-the-art transformer hardware accelerators. For GNN inference acceleration, we propose GHOST, a hardware accelerator that integrates various device-, circuit- and architecture-level optimizations which enable it to efficiently process a broad family of GNNs and real-world graph structures and sizes. When compared to multiple state-of-the-art GNN hardware accelerators, GPUs, CPUs, and TPUs, our experiments showcase how GHOST exhibits significantly better performance and energy efficiency.
dc.format.mediumborn digital
dc.format.mediummasters theses
dc.identifierAfifi_colostate_0053N_17782.pdf
dc.identifier.urihttps://hdl.handle.net/10217/236888
dc.languageEnglish
dc.language.isoeng
dc.publisherColorado State University. Libraries
dc.relation.ispartof2020-
dc.rightsCopyright and other restrictions may apply. User is responsible for compliance with all applicable laws. For information about copyright law, please see https://libguides.colostate.edu/copyright.
dc.subjectgraph neural networks
dc.subjectsilicon photonics
dc.subjectartificial intelligence
dc.subjecttransformer neural networks
dc.subjecthardware accelerators
dc.titleSilicon photonic hardware accelerators for transformers and graph neural networks
dc.typeText
dcterms.rights.dplaThis Item is protected by copyright and/or related rights (https://rightsstatements.org/vocab/InC/1.0/). You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).
thesis.degree.disciplineElectrical and Computer Engineering
thesis.degree.grantorColorado State University
thesis.degree.levelMasters
thesis.degree.nameMaster of Science (M.S.)

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Afifi_colostate_0053N_17782.pdf
Size:
2.26 MB
Format:
Adobe Portable Document Format
Download

Collections

2020-
Theses and Dissertations