Thanks to copper.hat's remark it finally made -clonk- and here it is: $$ \DeclareMathOperator{abs}{abs} f_+(x) = \max(f(x),0) = \frac{\abs(f(x))+f(x)}{2} \\\ f_-(x) = -\min(f(x), 0) = \frac{\abs(f(x))-f(x)}{2} $$ We need the theorem that if $f$ is Riemann integrable on $[a,b]$, then so is $\abs(f)$ (Link). Together with linearity, this gives integrability of $f_+$ and $f_-$ on $[a,b]$.
Subtraction of the two equations above gives $f_+(x) - f_-(x) = f(x)$, together with the integrability and linearity it should yield the decomposition $$ \int\limits_a^b f(x) \, dx = \int\limits_a^b f_+(x) \, dx - \int\limits_a^b f_-(x) \, dx $$