The symplectic reduction of pure spherically symmetric (Schwarzschild) classical gravity in D space-time dimensions yields a 2-dimensional phase space of observables consisting of the Mass M (>0) and a canonically conjugate (Killing) time variable T. Imposing (mass-dependent) periodic boundary conditions in time on the associated quantum mechanical plane waves which represent the Schwarzschild system in the period just before or during the formation of a black hole, yields an energy spectrum of the hole which realizes the old Bekenstein postulate that the quanta of the horizon A_{D-2} are multiples of a basic area quantum. In the present paper it is shown that the phase space of such a Schwarzschild black hole in D space-time dimensions is symplectomorphic to a symplectic manifold S={(phi in R mod 2 pi, p = A_{D-2} >0)} with the symplectic form d phi wedge d p. As the action of the group SO_+(1,2) on that manifold is transitive, effective and Hamiltonian, it can be used for a group theoretical quantization of the system. The area operator p for the horizon corresponds to the generator of the compact subgroup SO(2) and becomes quantized accordingly: The positive discrete series of the irreducible unitary representations of SO_+(1,2) yields an (horizon) area spectrum proportional k+n, where k =1,2,... characterizes the representation and n = 0,1,2,... the number of area quanta. If one employs the unitary representations of the universal covering group of SO_+(1,2) the number k can take any fixed positive real value (theta-parameter). The unitary representations of the positive discrete series provide concrete Hilbert spaces for quantum Schwarzschild black holes. 45 pages, Latex; version accepted for publication in Phys. Rev. D (Refs. added, small changes in the introduction, no changes of results)