Git-annex stores its files based on their content hash. So if your two files `fileA` and `fileB` are identical, then they should both be recognised by git-annex as the same file. If they are not being recognised as the same file, then it suggests they might be slightly different. You could try comparing their hashes.
The git-annex docs have a tip page that explains how to combine multiple existing directories of content into a single repository, and it addresses exactly this topic.
<