What formula will tell if three vertices in 3d space are ordered clockwise or counter-clockwise from the point of view of a camera? Assuming 3 ordered vertices in 3d space and a camera looking toward those points. What formula will tell me if they are seen clockwise or counter-clockwise in relation to their order?

Assuming the camera is at the origin, consider the $3\times 3$ matrix with column vectors equal to the three point vectors. The sign of the determinant of this matrix tells you the orientation.