It means $A$-linear endomorphisms. If $M = k^n,$ then $M \otimes_k A = A^n$, and then $End_A(M\otimes_k A)^{\times} = Aut_A(A^n) = GL_n(A).$
* * *
Note that taking $End_k(M\otimes_k A)^{\times}$ gives the wrong thing, since if $A$ has dimension $d$ over $k$, then $M\otimes_K A$ has dimension $nd$, which isn't independent of $A$ (and is not even finite if $A$ is not a finite-dimensional $k$-algebra!). (Not to mention that this expression is not functorial in $A$.)