Collective dynamics of high brightness electron beam compression and manipulation