There are two definitions :
* For an integer $n$ let $|n|_p = p^{-k}$ if $n \equiv 0 \bmod p^k, n \
ot \equiv 0 \bmod p^{k+1}$. Then $|.|_p$ is a metric, a norm and an absolute value on $\Bbb{Z}$, so you can take its completion (limits of Cauchy sequences) just as you did when completing $\Bbb{Q}$ to obtain $\Bbb{R}$.
* $\Bbb{Z}_p$ is the set of sequences $(a_k)_{k\ge 1}$ such that $a_k \in \Bbb{Z/p^k Z}$ and $a_{k+1} \equiv a_k \bmod p^k$ which becomes a ring with the pointwise addition and multiplication modulo each $p^k$. Then $\Bbb{Z}$ embeds in $\Bbb{Z}_p$ by sending $n$ to $(n \bmod p, n \bmod p^2,n \bmod p^3,\ldots)$.