sit idly by and

sit idly by and

sit (idly) by (and (do something))

to take no action.

I felt we simply couldn't sit by and see the peace process threatened.

We can't afford to sit idly by while dangerous people threaten our way of life.

Cambridge English Idioms Dictionary