You are trying to eliminate fractions. Obviously $\lambda=8\mu+k$ since the fractions have a least common multiple of 8. Next, you want to figure out a possible value for $k$. So, multiplying out, the scalar part becomes:
$$\begin{bmatrix}\tfrac 7 4 \\\ \tfrac 5 8 \\\ 0\end{bmatrix} + k\begin{bmatrix}\tfrac 3 4 \\\ \tfrac 1 8 \\\ 1\end{bmatrix} = \begin{bmatrix}\tfrac{7+3k}{4} \\\ \tfrac{5+k}{8} \\\ k\end{bmatrix}$$
So we are looking for $7+3k\equiv 0 \pmod 4$ and $5+k\equiv 0 \pmod 8$. This gives $k=3$ as a possible value that will eliminate fractions, which is what was used.