Multi Valued Model Suppose we have a set of observations $\\{(x_1,y_1),...,(x_n,y_n)\\}$ and a function $f$ that provides an optimal (in some sense) prediction of an output given an input. Also suppose that for some $i$ and $j$, $x_i=x_j$ but $y_i\ne y_j$. That is, there are multiple values for the same output. We would like $f$ to be a function (single valued). What are some techniques to rectify this? For example, perhaps $f$ can give a prediction of a mean given an input and then we can estimate variance (assuming normality). Is there something that can be done to condense the multi valued inputs into a single observation?

Read about Least Square Fitting. I think that this is what you need.

To sum it up briefly, I'd say the you should consider which type of function you expect the approximation would be (linear, polinomial, etc.) and then, using LSF, you should approximate this function with the observations you have.

Hope you'll find it helpful.