Many road calipers don't have you use a ferrule at the brake end. That's the intent here. If you want to anyway or if you must due to using a braided/compressionless brake housing, the solution is to use stepped ferrules such as Jagwire POP ferrules, which are long and have an opening for 5mm housing on one end and reduce back down to 5mm OD on the other end.