`SOCKS5` is a protocol (i.e. in the application layer of OSI), so plain network-routing (e.g. via `iptables`) alone _won't do_. (It's probably necessary, but not sufficient.)
What you need is a _proxifier_. Without having tried it, `tun2socks`, allowing you to _"socksify TCP at the network layer"_ , looks promising (as does `proxychains`, without `iptables` but prefixing programs explicitly).
(Could `socat` be used here? This mentions `socat` for socksifying, but I'm unsure at the moment.)