The Frobenius endomorphism of an abelian variety over a finite field Fq of dimension g can be considered as an element of the finite matrix group GSp2g(Z/lr). The characteristic polynomial of such a matrix defines a union of conjugacy classes in the group, as well as a totally imaginary number field K of degree 2g over Q. Suppose g = 1 or 2. We compute the proportion of matrices with a fixed characteristic polynomial by first computing the sizes of conjugacy classes in GL2(Z/lr) and GSp4(Z/lr. Then we use an equidistribution assumption to show that this proportion is related to the number of abelian varieties over a finite field with complex multiplication by the maximal order of K via a theorem of Everett Howe.