At the initial phases of the healing the broken ends are mobile (and this is time when it can be set and put into a cast).
At later stages soft callus than hard callus forms and now any movement will pull on the broken ends and stimulate the nerve endings: it hurts.
This also has the effect that the nearby muscles will spasm in an attempt to immobilize the bone. Another source of the pain can be that these muscles become tired and start to hurt as well.
The pain will continue until the two broken ends are completely welded together so there is no longer any movement that would stimulate those nerves.