When you rename a file, you don't change the file, you change its parent directory. A file name is an entry in a directory. Think of phone directories, to change the name associated with a phone number in a directory, you need to modify the directory, not the phone line. The name is associated with the phone line only in that directory. That phone number may be in another directory under a different name (hard links).
There's a caveat though for renaming directories as directories contain a reference to their parent (their `..` entry). To be able to move a directory, it's not enough to have write permission to the old parent (to remove the entry) and the new parent (to add a new entry), you also need to have write permission to the directory itself to update the `..` entry (if the old and new parent are different).