Here is a diagram of the situation: !enter image description here
The limit as $x\to0$ exists, because the left-handed and the right-handed limit approach the same value.
According to the diagram, it seems that the tangent line is vertical. When the tangent line is vertical the slope is infinite.
Let's see if the slope is indeed infinite:
$$ f'(x)= \left\\{ \begin{array}{cc} \frac{1}{2\sqrt{x}} & x \ge 0 \\\ \frac{1}{2\sqrt{-x}} & x < 0 \\\ \end{array} \right. $$
For $x=0$ the slope approaches infinity, so the tangent line is vertical.