By now everyone’s heard of the new “Render Rate” proposed by the eec’s Measurement Accuracy Roundtable, and the request for comments to the proposal. While there were the obvious questions as to “why?”, the answer is found in the comments – everybody has a different way of measuring things and use the common terminology loosely.
What that boils down to is the the definition of “Open” depends on who you ask. That doesn’t work out too well for an organization moving from one ESP to the other – one ESP counts an “Open” as the rendering (key word, here) of a tracking pixel, while another counts an “Open” as someone clicking a link. For those using the latter method, what do you call it when someone clicks a link? And then wouldn’t Opens and Clicks always be the same making one redundant or irrelevent? I’m just askin’….
The Roundtable discussed and debated these kinds of questions for several months before coming up with our proposed terminology. The combined email channel experience of the Roundtable is around 100 years, so it’s not like it we were a few college buddies sitting around making this stuff up one night over a couple of beers.
My fellow committee members have already addressed some of the comments regarding the Render Rate measurement at the eec blog. But there’s one that I’d like to address on here that came from Chris Bryan of EmailDirect since he was kind enough to offer a link to his more detailed comments.
Let’s start with his second paragraph:
I’ve always understood Opens to be calculated in two ways: Unique Opens and Total Opens. Unique Opens would be the number of unique people who “saw” your email, captured using a tracking image. Total Opens would therefore be the total number of times your email was opened (captured using a tracking image.)
Just a simple question here, in order for a tracking pixel to be identified with the recipient, doesn’t it first have to “render” to the recipient’s Inbox? And just because the tracking pixel renders, does that mean the message was opened?
Total Opens gives you a good idea of the ‘staying power’ your email had in your subscribers’ inbox while unique opens tells you how many different people saw your email (saw meaning the number of people who viewed your email with the images loaded.)
The total number of messages “Opened” isn’t an indication of the “staying power” (or shelf life) of your message. If you really want to drill down on shelf life you would have to measure between the time the message was first deployed and the last time a tracking image rendered, and you’d want to put a stake in the ground for an end-time – let’s call it a week for this example. Then chart from the deployment to the end-time and identify the fall-off point in between. In the example chart, the shelf life is around 3 – 4 days.
“Unique Opens” may or may not tell you how many different people rendered a tracking pixel, or how many unique people rendered a tracking pixel. Generally speaking, tracking pixels are tied to an email address so that the render can credited to each unique recipient to which the message was sent. If the recipient renders a tracking pixel once, then forwards the message to 3 friends, and two of the friends cause the tracking pixel to be rendered, the email address to which the pixel is attached will show three pixel renders associated to the original recipient’s address. How many people “viewed” the message? Three, of course, but by Chris’ calculations – or explanation, anyway, it only counts as one unique render even though the message was viewed by additional people.
Now if one really wanted to get geeky they could log the IP address of the rendered pixel and tie it to the render of that pixel by the recipient. Assuming that the first render was the recipient, the IP address could be logged and then if the pixel is rendered a second time from a different IP address make the assumption that the original recipient forwarded the message to someone else at a different IP address. But that’s not always an accurate assumption as the original recipient may have glanced at the message at work and then again at home, thereby creating two unique renders from two different IP addresses for a single recipient. To infer a forwarded message, one would have to put a stake in the ground based on assumptions that X number of IP addresses calling the unique pixel tied to the original recipient presumes a forwarded message within a given time frame. And the same logic will apply to clicked links in the message.
Again, one can see why it is important that the Email Industry establish standards of measurement.
Moving on the the forth paragraph, Chris says:
A small caveat to to the way the EmailDirect system counts opens is: we include the people who clicked on your email but did not view the email with images loaded. That’s just common-sense, right?
No. Common sense suggests that not everyone reading the message will find the value proposition or call-to-action compelling enough to render an image or click a link. Let me give an example, when I subscribed to Allstate’s eBill program I was sent a message confirming my enrollment. There weren’t any links that I cared to click in the message, and I didn’t need to render images to get value in receiving the message. Obviously Allstate has absolutely no way of knowing if I read the message or not. The best that they can do is assume that I received the message as Sent logs show the message was sent to me, and the bounce logs show that it didn’t come back. Other than that, Allstate knows nothing about what I may or may not have done with the message, or even if it made it past any filters to be delivered to my Inbox.
Imagine getting a mostly-text email with a small image at the top (such as a logo.) Taking the time to click the “load images” button isn’t always worth it, even when you’re interested enough to click on a link. So, common-sense tells us: If the subscriber doesn’t load the images but clicks on a link, they should be counted as “opening” the email.
While this is true, this doesn’t address whether or not the message sent was Text or HTML. What we don’t know here is if EmailDirect lumps all “opens” from a single campaign together or shows a delineation between HTML messages and Text messages. If EmailDirect was certified to be using standard metrics we would know how they measure these statistics without having to ask. Think, “transparency”…. and yet another good reason for standard metrics.
We can drill down a littler further and compare those HTML messages that didn’t render images, but clicked a link to those HTML messages that rendered a tracking pixel, to those HTML messages that rendered a pixel and clicked a link. What’s the point of that? Well, we might assume that those recipients that render images, whether they click a link or not, may have added the sender to their Safe Sender list so that images always render. From there we may make assumptions about our message content, value propositons, and call-to-action. Or we may make a variety of other assumptions to test against to improve our messaging.
And then there’s the whole matter of Multipart MIME messaging. Multipart MIME messaging is where the message sent contains both HTML and Text, and the recipients email client determines whether to show the HTML verion or the Text version. Filtering systems like Postini will display the Text version of a MIME message when available, and some recipients set their email client to read Text instead of the HTML. Lazy marketers don’t bother to include MIME, and half-way lazy marketers just slap some brief text with maybe a couple of links to a Web-hosted version and hope that the recipient is so enamored by the sender that they’ll click over.
Since I happen to use Postini on my main email address, let’s see how it looks. And just for fun, let’s look at one of EmailDirect’s clients, Big 5 Sporting Goods, since only maybe 1 out of every 10 or 12 messages makes it past Postini. Here we see the main screen of Postini. Big 5 is in good company this time with PetCo and WilsonWeb, but let’s see how they handle MIME.
Since Dr. Wilson sends messages in Text format it would not be fair to use him as an example, however his newsletter is an excellent example of how to format a Text message. When no Text message is sent as part of MIME, then the HTML is displayed, and sometimes as just a bunch of unassembled HTML code, but I’m getting off track a bit here. So for this example, I’ll use a message from Strongmail that was filtered by Postini. This is a fairly well formatted Text message, although they should have used a carriage return for a hard line break, but that’s another story. Now let’s look at the MIME version of the Big 5 message. Tsk, tsk…. lazy….
Now getting back to the topic at hand, if I were to cut and paste the URL from the Big 5 message to a browser, would it register as a Text “Open” or a Multipart MIME open for EmailDirect? Or do they lump all “Opens” togther so it doesn’t matter anyway. I hope by now that you’re getting the idea. We could go deeper into this, but I’m already at about 1,600 words and we still have a little more to go….
Chris goes on to state;
My issue with this whole Render Rate proposal is it’s adding a layer of complexity that isn’t necessary.
Hopefully by now you’ve come to see that the proposed Render Rate metric doesn’t add a layer of complexity, but a layer of clarification. And I hope by now you’ve also come to see how that without standardized metrics a relatively young company in the ESP space might just kind of make up their own calculations and attach them to common Industry terminology.
Im [sic] all for somebody defining standard metrics but let’s create the formal definition for Open Rate. Why do we need to leave a supposedly broken term to it’s brokenness [sic] and create a new one? Why not just fix the broken term?
Well, Chris, in our Roundtable discussions about how to address this we came to the conclusion that the “open” term couldn’t be “fixed” because to do so would still leave the term descriptively inaccurate. Just because an image or tracking pixel is rendered doesn’t mean that the message was “opened”, and just because a link wasn’t clicked doesn’t mean that the message wasn’t opened. And while you’ve pointed out that the eec lists a definition for “Open” in a Glossary of Terms, you seem to have missed the bigger point in that is what the Render Rate metric is designed to “fix” under a more accurately descriptive term.
Now, for those that want to take the measurement one level higher and incorporate Deliverabilty metrics into the Render Rate metric for an even more accurate measurement, the formula would be:
Total Render / ((Sent – Bounced) x 3rd Party Deliverability) = Total Render Rate
While I appreciate the thoughtful and constructive comments regardng the proposed terminology, particularly those made by Mark Brownlow, I kind of look at this as you walking in when Travis is about to shoot Ol’ Yeller, and having never seen the movie, ask why everyone is crying over a kid shooting a dog to save his mother and brother (and pause the movie to bring you up to speed) because nobody cried when Cujo died….