Yes, close- / reopen votes do expire after some time. -- To cite the help center:
> Close votes age away harmlessly after 4 days if the threshold is not reached. Each new close vote resets the timer, and close votes only start expiring if the question has had 100 views.