Five Data Preparation Mistakes (and How to Avoid Them!)

After building many predictive models in the Rapid Insight office and helping our customer build many more models outside of the office, we have a list of data preparation mistakes that could fill a room. Here are some of the most common ones we've seen:

1. Including ID Fields as Predictors

Because most IDs look like continuous integers (and older IDs are typically smaller), it is possible that they may make their way into the model as a predictive variables. Be sure to exclude them as early on in the process as possible to avoid any confusion while building your model.

2. Using Anachronistic Variables

Make sure that no predictor variables contain information about the outcome. Because models are built using historical data, it is possible that some of the variables you have accessible when building your model were not available at the time the model is built to reflect. No predictor variables should be proxies for your dependent variable (ie: “made a gift” = donor, “deposited” = enrolled).

3. Allowing Duplicate Records

Don’t include duplicates in a model file. Including just two records per person gives that person twice as much predictive power. To make sure that each person’s influence counts equally, only one record per person or action being modeled should be included. It never hurts to dedupe your model file before you start building a predictive model. 

4. Modeling on Too Small of a Population

Double-check your population size. A good goal to shoot for in a modeling dataset is at least 1,000 records spanning three years. Including at least three years helps to account for any year-to-year fluctuations in your dataset. The larger your population size is, the most robust your model will be. 

5. Not Accounting for Outliers and/or Missing Values

Be sure to account for any outliers and/or missing values. Large rifts in individual variables can add up when you’re combining those variables to build a predictive model. Checking the minimum and maximum values for each variable can be a quick way to spot any records that are out of the usual realm. 

[photo credit]

-Caitlin Garrett, Statistical Analyst at Rapid Insight

Predictive Modeling Mantras

Whether you're new to predictive modeling, or you dream in decile analyses, here are some things to keep in mind as you're embarking on your next modeling project: 

Data preparation makes ALL the difference.
Simply put, if you use junk data to create a model, chances are that your model’s output will be junk too. Thus, it’s very important to clean up your data before building your predictive model. During the data clean-up process, you’ll want to think about things like how to handle any missing values, possibilities for new variables to be added to your dataset, how to handle outliers, and make sure that your data is as error-free as possible.

A complex model isn’t the same as a good model.
More often than not, the best model is a simple one. Although you can almost always find a new variable to add, or new way to slice your data, you want to avoid the trap of overfitting. You want your model to be specific, but not so specific that you sacrifice reliability when scoring a new dataset.

A good model validates what you know while revealing what you don’t .

Don’t be surprised if some of your “common sense” variables outperform the more exotic ones. Although it’s always nice to pick up on some new variables and insights, building a predictive model can also boost your confidence in the rest of your data.  

If a model looks perfect, it’s lying.

As exciting as getting a great model fit statistic can be, there is the possibility of too good to be true when it comes to model building. If you build a particularly great model, you’ll want to double and triple check each of the variables in the model to be sure they make sense. One of the most common reasons for a great model is an anachronistic variable – a variable you would have available only after your y-outcome was decided.

Persistence is a virtue (because building models is an iterative process).

After you’ve taken a first pass at a model, maybe you’ll think of a related variable that would be predictive. Maybe you take a second look at some of the relationships between variables and decide to bin or re-map some of your continuous or categorical variables. Maybe the outputted variables are the opposite of what you expected, so you decide to tweak the way your dataset is set up. The point here is that your first model will likely not be your final model. Be ready.

Trust and verify.
The modeling process doesn’t end after you finish building your model. After implementing your predictive model, you want to be sure that it’s correctly predicting your y-variable over time. To do this, you’ll need to compare your model scores with actual results once they are available. If your model is correctly predicting the desired outcome, you can continue to use it (but still must validate as time goes by); otherwise, you’ll need to take a few steps back to see where you can make improvements.

-Caitlin Garrett, Statistical Analyst at Rapid Insight
[photo credit]

Fundraising: The Science

Now that we’ve discussed the art of fundraising, I think it’s only right that we focus a little bit on the science. After all, knowing which prospects are statistically most likely to give makes a gift officer’s contribution to the art of fundraising that much more successful. As I’ve mentioned before, my function in the fundraising spectrum is as an analyst, helping customers build models identifying which prospects are most likely to donate.

One of the most important things we do during the predictive modeling process is data preparation, which often means creating new variables from the data our customers have on-hand. I’d like to discuss some of these variables, as well as how and why to include them in a fundraising or advancement model. For the purposes of this blog entry, I’ll use a higher education example. Typically a higher education institution might have some extra variables, but these can be tailored to fit other institutions or excluded when not relevant.

Demographic Information          

It’s always smart to have an idea of what each donor looks like at a demographic level. Variables to include here are things like age, gender, marital status, and any occupational data you might have available to you. In a higher education context, this would also include things like the constituent’s class year, major, whether or not their spouse is an alumni, and whether they are a legacy alumni (meaning a parent or grandparent also attended the institution). Additionally, we often create a “reunion year flag” indicating if the analysis year is a reunion year for that person, as donors are often more likely to give (and give larger gifts) during a reunion year.

Location Information

General information about each donor’s location like ZIP code, city, and county can be useful as categorical variables (treating people that live in each one as a group). Once we have a ZIP code, we always calculate a “distance from institution” variable using one of Veera’s pre-programmed functions. This new variable, which is measured in miles, gives you a solid idea of the relationship between location and giving. If you have access to census data, we recommend appending variables relating to neighborhood or housing type. Creating flag variables for wealthy neighborhood ZIP codes can also be useful; constituents coming from these areas may be more likely to give. Although this can be created at a more local level, we often start with Forbes’ list of the top 500 wealthiest ZIP codes in the US, which is available online at http://www.forbes.com/lists/2011/7/zip-codes-11_rank.html.

Contact History

The ways in which a donor engages with you can tell you a lot about their likelihood of giving. For starters, include variables pertaining to their event history. How many events have they attended? Which types of events are they attending? How many days since their last event? Answers to questions like these can sometimes turn out to be predictive of giving. This is also where your social media variables come into play; create flags for whether a constituent is following you on LinkedIn, Facebook, Twitter, Pinterest, etc. A donor following you on one or more of these sites is an indication that they want to be connected, and therefore they may be more likely to give. Conversely, if a constituent has indicated that they do not want to be contacted, you’ll want to include this information as well, as it can be very predictive.

Gift History

This brings us to our last and most predictive set of variables: giving history. These variables should answer all kinds of questions about what a giver looks like historically, like:

• How many gifts have they given in their lifetime?
• What was their last gift?
• How many days since their first gift? How many days since their last gift?
• Have they given in the past 12 months? If so, how much?
• What is the velocity of the gifts - are the increasing, decreasing, or staying the same?

One thing to note here is that gift dates themselves aren’t useful in a predictive model, but their translations – like the number of days since an event – allow us to use the insight they provide.

In building a predictive model, some of these variables may be predictive, while others might turn out not to be. It’s a good idea to include some combination of these variables, plus anything you have on-hand that you think could possibly be predictive.

-Caitlin Garrett, Statistical Analyst at Rapid Insight

Fundraising: The Art

As much as my analytic brain would love to be able to classify the world into black and white binaries, sometimes this division is just not possible. Such is the case with fundraising. As important as prospect and donor research are, just knowing which prospects are statistically most likely to give to your institution does not mean that they will automatically give the amount you predict, when you predict it. Other factors, such as the relationship between the donor and institution and the way in which appeals and touches are made, have a heavy impact on how much and how often a donor chooses to give. My personal role in the fundraising spectrum has been on the analytics side, helping customers build models identifying which prospects are most likely to donate. This August, I was given an opportunity to learn about the other half of the process during APRA’s Data Analytics Symposium.

The thing that resonated most with me was Penelope Burk’s tenets of donor-centered fundraising. She says that donors want:

I. Prompt and meaningful acknowledgement for their gift(s);

II. To know specifically where the money will go;

III. To be updated on the progress of projects they donate to.

Let’s look at these in a little more detail, shall we?

Donors want to be thanked soon after their gift is received.  This thank you should be personalized and delivered in a way that’s meaningful to the contributor.

Donors want to know which fund, building, scholarship, or project their money will be used for. Allocating donor dollars to a specific project is helpful both for the donor and the institution they are donating to; for example, when a gift officer is touching base with the donor, they can focus on project-specific updates, rather than simply speaking to the value of the institution overall.

Donors like being updated on the projects they’ve donated to. They want to know how far along a project is, an expected completion date, and when the project hits major milestones. The key here is communication. The more you communicate with a donor, the more involved and appreciated they feel, and the more likely they are to give again. Most of all, donors want to be sure that their money is making a difference.

One of the reasons that donor-centered fundraising has become so important is that the climate of fundraising is changing. Rather than giving smaller dollar amounts to a wide variety of institutions, donors are trending towards whittling down the number of different institutions, but increasing the dollar amounts given to each. For this reason, it is important to acknowledge and update each individual donor to cultivate gifts.

I think one key take-away message of donor-centered fundraising is exactly that: it’s donor-centered. The communication between fundraiser and donor should be keyed in to the needs and expectations of the donor on an individual level. Secondly, there should be lots of communication (especially in the form of updates) between these two parties. If all of the above criteria are met, the donor should have a good feeling about their gifts, feel appreciated for giving, and continue to give. 

-Caitlin Garrett, Statistical Analyst at Rapid Insight

On Target: Predicting Pregnancy

Call me biased, but I think creative uses for predictive analytics are pretty cool.  Target’s “pregnancy-prediction model”, explained Thursday in a The New York Times Magazine article, is a great example.  It should inspire all of us to take a fresh look at our data and consider what more we can accomplish with a powerful predictive analysis tool (like RI Analytics) and a little bit of creative thinking.

Target’s journey to predicting pregnancy started with an idea conceived by its marketing department. The department had previously conducted surveys which indicated that once a consumer’s shopping habits are ingrained, it can be hard to change them – except during certain brief periods of a person’s life, like after a marriage or the birth of a child, where shopping patterns and brand loyalties often change.  The birth of a child represents a new grocery and household goods list for new parents, as well as the opportunity for Target to sell things like cribs, rugs, furniture, car seats, and other items that a person or couple would not usually buy. Because birth records are public information it was already common practice for companies to send promotional items to new parents; so, to stay one step ahead of competitors, marketers at Target wanted to see if there was a way to predict pregnancy during the second trimester.

Target reviewed the shopping habits of women who had a baby-shower registry as they approached their due dates. Eventually they were able to identify about 25 different products that were indicators of pregnancy, including items like unscented lotion, vitamin supplements, hand sanitizers and washcloths. By treating the purchase of each item as a variable, they were able to create a model that assigned each shopper a pregnancy prediction score based on their purchases. This score was then used to send out relevant coupons and advertisements tailored to each woman at a specific point in her pregnancy – before other retailers even knew she was pregnant. Needless to say, sales in Target’s Mom and Baby department skyrocketed.

This is one way that a creative use of data, combined with some predictive analytics, yields some pretty cool results. Target had the data they needed all along –they just needed the right person to ask the right question. 

-Caitlin Garrett, Statistical Analyst at Rapid Insight