Enforce a minimum number of observations within a metric window

The metric window is currently only time-based. We should make sure we see a minimum number of observations to ensure we don't decide based on too few observations.