How 震え出してしまう should be translated? I mean translates as "to **begin** to tremble" and meaning of auxilary verb is "to finish ..., to do ... completely". So, it doesn't make sense for me to complete something you begin to do...

has quite a few meanings, e.g. finish doing something, do something completely, thoroughly, accidentally, regrettably, unexpectedly, or involuntarily, etc...

can be "start shivering" \+ "do...involuntarily/unexpectedly". Does it make sense in your context?