After reviewing the song, I've completely flipped on my answer. The whole song is definitely from his perspective, and this part is definitely him saying he might hurt her feelings.
Original Answer:
I definitely feel it's the 'each other' one. It's just saying that there are days they will hurt each other's feelings, and days they won't think the same way. I expect the rest of the song talks about how good they usually are together, or how much they love each other and it's okay, etc.