Control, filtering, learning, and multi-robot algorithms for large graph-based Markov decision processes