Christer Ericson's 'Real Time Collision Detection' is a killer reference on the topic, and he used to run a super interesting blog [1] on various mathy things related to game development.

[1] http://realtimecollisiondetection.net/blog/

Quite old now, but to me the go-to reference is still Brian Mirtich's Ph.D. thesis, "Impulse-based Dynamic Simulation of Rigid Body Systems": https://xkanim.googlecode.com/svn/trunk/mirtichThesis.pdf

I referenced this book up front in part 1 of the series:

http://0fps.net/2015/01/07/collision-detection-part-1/