No, we can't necessarily do that. Take, for instance, the Weierstrass function, whose graph is a fractal, going up and down infinitely many times on any interval.
Note that if your function is indeed a bumb, meaning that it is first increasing then decreasing, then it is necessarily differentiable almost everywhere.