Yes. The Kolmogorov definition explicitly assumes $\mathbb{P}[B]>0$.
See Wiki link to definition
**EDIT** Typically, one uses $$ (1) \qquad \mathbb{P}[A \cap B] = \mathbb{P}[A|B] \cdot \mathbb{P}[B] $$ as a _definition_ of $\mathbb{P}[A|B]$, meaning that we define $$ (2) \qquad \mathbb{P}[A|B] = \frac{\mathbb{P}[A \cap B]}{\mathbb{P}[B]}. $$ There is obviously no problem with (1) as is, except that $\mathbb{P}[A|B]$ must then be defined in some alternate way, although I am not sure what would that be. Alternatively, if we choose to go the standard way and define $\mathbb{P}[A|B]$ via (2), then the very definition requires $\mathbb{P}[B]>0$.