The probability is (I think clearly) $\frac{1}{n}$. For the number $k$ is equally likely to end up in any one of the positions.
One can also conclude this in a more complicated way. There are $n!$ equally likely permutations of our $n$ numbers. By your reasoning, there are $(n-1)!$ permutations that are "favourable," meaning that the number $k$ ends up in the $i$-th position. So the required probability is $\frac{(n-1)!}{n!}$, which simplifies to $\frac{1}{n}$.