No, in fact the logic works because of some properties:
every map $f:S^n \to S^n$ induced on homology $f_*:H_n(S_n) \to H_n(S_n)$, but since these are just the group $\mathbb Z$, all homomorphisms look like $a \mapsto ka$. We can take $k$ to be the definition of degree. From this, some things follow:
1. $deg(r)$ for any reflection about an axis has degree $-1$
2. $deg(fg)=deg(f)deg(g)$.
Putting these together, and viewing the antipodal map as $n+1$ reflections, we see that $deg(A)=(-1)^{n+1}$.
This is exactly the map the way you define it, and you can see that the degree is $1$ when $n$ is odd, which is the key ingredient in other answers.
* * *
By the way, it would have worked to note that the map is $-I$, where $I$ is the identity matrix. This map is in $SO(n)$, the symmetry group of the sphere when $n$ is odd, and since it is path connected, you can always find a homotopy.