If anyone's interested, after some thought, it looks like Brownian motion fits that bill.
If $w_i$ is a set of random numbers like in a random walk, e.g. 1,1,-1,1,-1,-1, etc. a random walk sequence can be constructed like so:
$x_i = \sum_{j=0}^i w_j$
Brownian motion just puts an exponential damping on the sum so that terms back beyond some specified time are not important:
$x_i = \sum_{j=0}^i \exp((j-i)/N)w_j$
Less physically, but maybe more effectively, you can just truncate the sum:
$x_i = \sum_{j=i-i_0}^iw_j$
These are two ways of killing the importance of data in the "distant past" and they both kill the low frequency parts of the random walk that cause unphysically (in my case) large perturbations.