Canonical coordinates and the geometry of inference, rate, and capacity

Abstract

Canonical correlations measure cosines of principal angles between random vectors. These cosines multiplicatively decompose concentration ellipses for second-order filtering and additively decompose information rate for the Gaussian channel. Moreover, they establish a geometrical connection between error covariance, error rate, information rate, and principal angles. There is a limit to how small these angles can be made, and this limit determines channel capacity.