Batteries in Series vs Parallel

What does is mean to connect multiple batteries in parallel or in series? It's a simple distinction but it's very, very important to get it right if you've going to connect multiple batteries.