Yes setfacl should do it. Try the below, does it work ?
setfacl -m u:user:--- file
Where:
* `-m` is to modify the file/directory ACL
* user is the username for which you want to change permission
* `---` will be the no permissions, replacing r,w,x
* file is the name of the file for which you want to change permissions