From checking `/boot/config-3.11.0-13-generic` (yours might be different), I would guess that it's built into the kernel, thus you can't unload/reload it.
$ grep -i BCMA /boot/config-3.11.0-13-generic
[...]
CONFIG_BCMA_HOST_PCI_POSSIBLE=y
CONFIG_BCMA_HOST_PCI=y
[...]