Turns out when you get a collision pair YOU SHOULD HANDLE CONTACT RESPONSE FOR BOTH BODIES, NOT JUST ONE OF THEM
Otherwise they move and their next collision (if a->b, b->a are found and resolved separately) they might penetrate badly or worse and everything blows up
This is probably not as important in sequential impulse type solvers because the actual position update is deferred, but in a position based one it's pretty important as it turns out.
It's also better for performance because I don't have to run the collision query for the same two bodies two times
