David Owen’s blog

In the last post, we simulated some Poisson data and then verified it by looking at its histogram and some descriptive statistics. We also built a basic sliding-window implementation and graphed its output.

To continue on, we’ll need to build a more realistic implementation, along with a method to feed it the simulated events. With that in hand, we’ll build an EMA function specialized for Poisson events.

Continue reading "Exponential Moving Average (EMA) Rates, part 2"

I had been thinking about determining the average rate of occurrences over time of some observation. For example, you might like to measure how much traffic flows through a street throughout the day. Reporting the time that every single car goes by is very accurate, but not very useful. You might bin traffic into hours starting on every hour, but if there is a spike or sudden increase in the middle of an hour you might miss its significance. So, you'd like to see a graph that's smooth like an average but with more detail in time.

One approach is similar to the binning approach, but slide the hour-long window across the data by minutes. Doing this requires keeping the data around, and using each data point repeatedly. If you have a surge of one million cars in a few minutes, you need to use those million points in your calculations 60 times.

This behavior is similar to the Simple Moving Average (SMA). A SMA can easily be transformed into an Exponential Moving Average, which requires only the previous EMA and the new data point to calculate the new EMA. So, I decided to create an Exponential Moving Average Rate (EMAR).

Continue reading "Exponential Moving Average (EMA) Rates, part 1"

The EMA is a very handy tool. It lets us calculate an average over recent data. But, unlike a Simple Moving Average, we don't have to keep a window of samples around—we can update an EMA "online," one sample at a time.

But the perennial question is: how do you start an EMA?

First, here are a couple of wrong ways.

Continue reading "The correct way to start an Exponential Moving Average (EMA)"

In Fixing dispatch, we refactored some code that dispatched things from a switch-statement or cascading if-statements to dispatching by polymorphism.

This time, we'll refactor a different piece of dispatch in a completely different way, and cover another design principle.

Continue reading "Deciding once"

A fun thing about refactoring code is that after one refactor is finished, the next candidate is easier to see.

In An abstraction gone wrong, we refactored the state variable of a simple tokenizer from an integer into an object. Now that that's done, another problem is staring at us in the face. It's in this code:

if (state == INITIAL) {
    // ...
} else if (state == IN_NUMBER) {
    // ...
} else if (state == IN_STRING) {
    // ...
} else if (state == AFTER_STRING) {
    // ...
} else if (state == ESCAPING) {
    // ...
}

Continue reading "Fixing dispatch"