Survey Software Email Deliverability Issues

Most survey software systems offer the ability to send out invitations to your potential respondents on your behalf. But how many of the e-mails actually get through, and how many end up in the folder? If there is one thing I’ve learned over the years, you can’t assume that your survey software provider will offer the same level of e-mail management as an e-mail vendor.

If you’re like most people, you probably only have a limited number of potential respondents for your survey. You may have a list of your club members, or a list of fellow students, or even a list of customers you’ve painstakenly collected over a period of months if not years. You’re all set to send them a survey, and figure you’re all set to go — after all, your survey software provider has made it a snap to upload your e-mail addresses and send out personalized invitations on your behalf. Right?

Stop right there! How do you know that e-mails sent by your survey software company aren’t going to end up in the bulk mail folder or spam folder of your recipients?

If there is one thing I’ve become painfully aware of as I’ve looked at the various survey software packages out there, it is that very few survey systems put much effort into how they manage the e-mail they are sending. And since these systems send out such a high volume of e-mail, many of them are blacklisted by internet service providers or blocked using standard spam filters.

Here are some things to ask about if you plan to use your survey software provider to send e-mail on your behalf or manage your e-mail list.

See if your survey software provider uses e-mail

There are several companies that offer something called e-mail . These companies work closely with and companies to guarantee that anyone using their service will automatically be “trusted” by the e-mail providers. Getting certified is somewhat of a complicated process, but in my experience using Certified, it is worth it since just about every e-mail you send out makes it to its destination and doesn’t end up in the spam box.

There are four big companies that offer e-mail certification. These are Habeas, SenderScore Certified, Goodmail, and SuretyMail.

Ordinarily, you as a survey system client wouldn’t need to work directly with any of the companies I just named — optimally, it would be the survey system itself who bonds their e-mail. If your company is bonded separately, then it would behoove you to send your e-mail through your own e-mail system and not through your survey software provider.

Incidentally, the way these certified e-mail providers work is that the client pays an annual fee or a per- fee for each message that is sent (per recipient). If any of the e-mails are reported as spam (in certain ways) the sender is charged a penalty. Back in the day when I was using Sender Score Certified, every e-mail that was reported as spam cost us $20.

Now, to be fair — survey software providers don’t have a lot of control over what e-mails their clients are uploaded and sending to so it might be very risky for them to participate in such a program. Plus these programs cost several thousand dollars a year to belong to. However, if you’re serious about your e-mail surveys, or if you make your living this way, you may want to investigate sending out your invitations through a separate e-mail service that does offer certified e-mail.

What relationships does the survey vendor have with the ISPs?

Even if a survey software provider doesn’t certify their mail, they can at least develop relationships with the major ISPs such as Google, Hotmail, Yahoo, etc. to help ensure that as many e-mails get through as possible. Getting “white listed” among the major e-mail systems isn’t a terribly complicated thing to do, and it doesn’t cost very much. It will also help to ensure that the vast majority of e-mails make it to their destination.

Is the survey software provider on any blacklists? And is there a way to tell how likely my invitations are to go through?

There are tools on the internet that make it possible for you to get a sense of whether or not your invitations sent through an survey software provider are likely to go through. These tools can also help you know whether or not your survey software provider is on any blacklists. 

An e-mail blacklist is a service that compiles lists of domains and IP addresses that regularly send out spam. These lists are used by companies and ISPs to filter mail before it even makes it to your recipient’s e-mail software. There are dozens of these lists, and once a company is on one it is very difficult to get off.

Here’s what I would do to check on the deliverability of e-mails from a survey software provider. I use a free tool called SenderScore.

1. Go to http://www.senderscore.org

2. Punch in the domain name of the survey system. (Try “.com”). You’ll be shown some basic information about how the provider sends out e-mail, such as whether or not they even send out their own e-mail (if they have an MX record), whether or not they have an SPF record (good for deliverability), and whether or not they have a Sender ID record (also good for deliverability).

Zoomerang SenderScore Info Page

 You can see on the list above that Zoomerang sends out their e-mail using two different domains — “mailsender.zoomerang.com” and “mailsender2.zoomerang.com.’ You’ll also note that these are both clickable links — so try clicking them. The first link, you’ll notice, indicates absolute no volume of mail being sent, while the second link (“mailsender2.zoomerang.com”) has a very high volume of e-mail. Note the clickable IP address on the left side. Click it.

Zoomerang E-mail Info 2

The page that appears (specifically, the SenderScore information for contains a lot of information that you can use to decide whether or not this survey software provider can get your e-mails through.

Zoomerang SenderScore Data 3

Starting from the top: the Sender Score is a summary number that you can use as a measure of this IP address’s “reputation” on the Internet. This number can range from 0 to 100. Let’s just say that if you sent out good e-mail from your own server all of the time you shouldn’t have a problem keeping this number above 90. If you send out to a lot of big e-mail lists where people come and go a lot, you could be at around 60-70. If you send out a lot of e-mail from clients who upload their own lists (like a survey software company, a 40 isn’t surprising.

Some other survey companies and their ratings:

  • InsightExpress.com: 98
  • ConfirmIt: 80
  • : 80 [not 25]
  • SurveyMonkey: 75
  • Qualtrics: 50

Working down the page, you’ll see that next to “blacklists” there is one listing for blackholes.five-ten-sg.com, a fairly aggressive blacklist that lists e-mail senders for many reasons (actually, if you do a search on google you’ll see that this particular blacklists lists so many people that most e-mail servers are not encouraged to takes the list seriously).

Next to “Sender Score Certified” it says no, indicating that Zoomerang does nog subscribe to this particular e-mail certification program.

Under deliverability it says that about 91% of the e-mail sent through this system gets through, and that e-mail sent through this system has a “medium” risk of deliverability issues. Medium isn’t bad and isn’t something you should be worried about. You should definitely be worried if the system you are using says “High.”

Reputiation measures cover a variety of facotrs that ultimatley contribute to the overall score. Higher numbers are always better.

What if I want to improve my deliverability?

Even if you don’t want to switch survey software providers in order to send out your e-mails, you can still get good deliverability by sending your e-mails through a separate service that focuses on sending out e-mails such as CheetahMail, Constant Contact, ExactTarget, Lyris and several others.

2 comments for “Survey Software Email Deliverability Issues

  1. August 28, 2010 at 3:02 pm

    There’s a typo error in this part of your article “Reputiation measures…” , though I’m really impressed with the information you have shared. It seems that you are really professional to this subject matter. I guess you could help me with my email marketing campaign by sharing more tips and lessons.


  2. Kyle
    March 4, 2011 at 10:01 am

    It should be noted that although Qualtrics has a very high volume of emails going out due to its exponential growth, it currently has a stellar Sender Score of 92!

Leave a Reply