Cost Function: Does it matter what order the y_predicted and y_actual are in? I'm learning neural networks, specifically backpropagation and am reviewing the cost function. When looking at different educational sources for backprop, I'm seeing the cost function written in different ways. Sometimes it's written as (y_predicted - y_actual)^2 such as in Andrew Ng's notes and in other sources such as Welch Labs youtube NN videos its written as (y_actual - y_predicted)^2 (source: < at 0.55sec). I've run through some test figures and because it's squared it comes the same but the inconsistency has thrown me. Does the order matter or is it just dependent on how the author wants to write it at the time? Cost function: y_predicted - y_actual

You have

$$(y_p-y_a)^2 = (-(y_a-y_p))^2 = ((-1)\cdot (y_a-y_p))^2=(-1)^2(y_a-y_p)^2=(y_a-y_p)^2$$

so no, the order doesn't matter.

* * *

You could also just as easily write $|y_p-y_a|^2$ if that makes it any clearer.