Collective quantum behavior of atomic ensembles in high-finesse optical cavities