Preservation of low latency service request processing in dockerized microservice architectures

Abstract

Organizations are increasingly transitioning from monolithic architectures to microservices based architectures. Software built as microservices can be broken into multiple components that are easily deployable and scalable while providing good utilization of resources. A popular approach to building microservices is through containers. Docker is an open source technology for building, deploying, and executing distributed applications within containers that are referred to as pods in Docker orchestrator terminology. The objective of this thesis is the dynamic and targeted scaling of the pods comprising an application to ensure low latency servicing of requests. Our methodology targets the identication of impending latency constraint violations and performs targeted scaling maneuvers to alleviate load at a particular pod. Empirical benchmarks demonstrate the suitability of our approach.