You are not done with the linearization, you have something quadratic. Besides, you miss some constraints on $z$ which would tell you it is equal to $1$ iff $x-a_i$ is positive. But there is a simpler way to go here...
You can solve the equivalent problem
$\min\limits_{x,y,t} t $ where the variable $t \in \mathbb{R}$
with constraints
$\forall i \in \\{1,..,m\\}, x-a_i+b_i \leq t$
$\forall i \in \\{1,..,m\\}, a_i-x+b_i \leq t$
And the additional constraints you had before.
This is linear, and $t$ will be equal to $\max\limits_i (|x-a_i|+b_i)$
Rq : it is always possible to do something like this when you have a $\max$ or an absolute value (which is a kind of maximum) in the cost function. It is harder when the $\max$ is in the constraints though.