Here's a first draft of a data filter:
After you have collected the first $N$ data points, call them $x_1, x_2, ..., x_N$, you calculate a mean, call it $m_N$ and some standard deviation $s_N$. Then look back at the the data points and if any of them don't satisfy a test of not being noisy, $|x_j - m_N| < Cs_N$, for a constant $C > 0$, throw them out of the data set. Now recalculate the mean.
You could tune the algorithm to find the best values of $N$ and $C$ for your application either manually or with machine learning.
There are also more sophisticated approaches. See, e.g., <