And so I find machine learning easier to think about when you have one set of tools for optimizing the cost function J, and when you're focusing on authorizing the cost function J. All you care about is finding w and b, so that J(w,b) is as small as possible. You just don't think about anything else other than reducing this.
https://www.coursera.org/lecture/deep-neural-network/other-regularization-methods-Pa53F
