With the GNU implementation of `grep` (the one that also introduced `-o`) or compatible, you can use the `-h` option.
>
> -h, --no-filename
> Suppress the prefixing of file names on output. This is the
> default when there is only one file (or only standard input) to
> search.
>
With other implementations, you can always concatenate the files with `cat` and `grep` that output:
cat ./*.txt | grep regexp
Or use `sed` or `awk` instead of `grep`:
awk '/regexp/' ./*.txt
(extended regexps like with `grep -E`).
sed '/regexp/!d/' ./*.txt
(basic regexps like with `grep` without `-E`. Many `sed` implementations now also support a `-E` option for extended regexps).