Typically you can set the BIOS to handle USB support, which will work with whatever boot-loader from there. At that point you can have a separate image available to load with USB support for those times that everything goes to pot. This may not work as some USB devices aren't supported by some boot-loaders.
Two notes:
1) I haven't tried this, though the theory should be sound (the crowd can shoot me down if I'm not.
2) I'm also not sure that the USB module is that heavy. You could JUST leave in the specific model you have to make it lighter and still have a keyboard available.