Book/paper for philosophy of choosing depth, width, stride or pool for CNN?

Is there any book with thoughts and experiments around how to chose number of layers and other parameters for a CNN?

My current approach is trying to shrink number of parameters and remove layers until the accuracy decreases.