Yes, I believe you're correct. A priori, the guideline curve may not be planar, but by shifting continuously along the generatrices, you may assume the guideline curve lies in a plane orthogonal to the generatrices. Then reflection across that plane or any plane parallel to it will preserve the cylinder.
Algebraically, parametrize the surface by $x(s,t) = \alpha(s) + tv$, where $v$ is a fixed unit vector. Using this formulation, you can replace $\alpha$ by $\tilde\alpha$ with $\tilde\alpha\cdot v = 0$, as I remarked earlier. Now check that sending $t$ to $c-t$ for any $c\in\mathbb R$ gives the reflection across the plane $v\cdot x = c/2$.