With derivatives, why doesn't '$h$' in the denominator not invalidate the whole function as diving by zero? The '$h$' in the $\frac{df}{dx}$ formula, i.e., $$f'(x) = \lim_{h\to0}\frac{f(x + h) - f(x)}{h}$$ stands for an infinitesimal change, correct? Why isn't this '$h$' equal $0$? If $h\neq 1$ either, why can it be overlooked as inconsequential in the denominator? I'm self-learning, stop me if I'm mistaken or if the question has already been asked.

You’re correct that we can’t simply divide by $h$ if $h$ is zero. But recall that $\frac 0 0$ is an indeterminate form; it does not necessarily diverge to infinity, and in fact when a limit takes on this form it can take on any real value. This is the form of the definition of the derivative: $\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}$ becomes $\frac 0 0$ upon plugging in $0$ for $h$, so we cannot simply plug in and evaluate the limit like this. But as we approach $h=0$ via a limit, both the numerator and denominator will shrink towards zero, while the ratio as a whole will approach some other value. This is how we “get around” the issue of the indeterminate form and division by zero.