Uniqueness of the inverse is a consequence of associativity. Suppose $*$ is an associative operation on the set $S$ and that $e$ is the neutral element for $*$, that is $$ e*x=x=x*e $$ for every $x\in S$. Define an inverse of $x\in S$ as an element $x'\in S$ such that $$ x*x'=e=x'*x $$ Then we can prove that if $x'$ and $x''$ are inverse element of $x$, then $x'=x''$.
Indeed $$ x'=x'*e=x'*(x*x'')=(x'*x)*x''=e*x''=x'' $$
With non associative operations, uniqueness of the inverse is not granted, it may or may not hold.