Have you ever wondered which type of content, say a ‘pdf’, web content (a web page) or video is best to use as a lead magnet on your landing pages?
We conducted an experiment to explore this. The idea is that by testing offer prototypes for opt-in probability before the campaign launch marketers can gain insight as to what media type to use.
Does the medium of your lead magnet affect email opt-in rate conversions?
To answer that question the CXL Institute research team set the goal of developing a testing strategy to help marketers test to see if different content formats affect opt-in rates.
But how do you measure or know if any given media type is the best fit for any said product? A guiding principle to keep in mind while exploring this concept with us is that when all is said and done, product satisfaction and preference is contextual.
While we cannot guarantee a one-size fits all answer to this problem, the results presented in this study are intended to suggest ways teams can test their content early on for marketing optimization.
Study Design: Data Collection Methods and Operations
We hypothesized that if we compare the probability of users opting-in across content marketing formats then we can make an informed decision about which media type is best-suited to release.
Scenario & Methodology
To keep the focus general, we chose a scenario that asked survey participants to imagine they were looking for more information on email marketing and were then directed to a landing page offer for “Email Marketing 101.” Of all the various content marketing formats that exist we chose to test the following three: a pdf, a website, and video.
After exposing participants to the given scenario, they were then directed to a single variation of the landing page of product offers (seen below) and asked to select the probability they would submit their email address using an 11-point Juster Scale. Juster scales are used to measure purchase probability. In our case, we are using email opt-ins as a form of purchase behavior.
To achieve statistical significance in our results we tested a total of 1206 participants (n=1206). The distribution of participants, randomly assigned, were 419 for Video, 398 for Website, and 390 for PDF respectively.
The semantic responses captured in the survey were translated into probability percentages according to the Juster Scale and were then transformed to allow for an Analysis of Variance (ANOVA).
Looking at the early percentages for the probability a visitor would opt-in for the email marketing product offer would be misleading if not tested for statistical significance.
Result: web content and pdf were more attractive offers than video content. PDFs content was 34% more probable to convert than video and regular web content (content served on a web page) was 24% more probable to convert than video.
The numbers were then crunched and the following graph (intended for visualization purposes only) illustrates the effect of the ANOVA results
- We are 100% confident that the means are different with Video across the three variables. Therefore, Video is less probable to convert than web content or PDF.
- There is no significant difference for opt-in between pdf and website as they fall within the same normal distribution.
- This is a great place where qualitative information can supplement teams for further evaluation.
Conclusions we can draw from this type of study is if media type is a determining factor for conversions or not. In our particular study, we can derive from the data that video is not the optimal method of product delivery. As we saw with Website and PDF, it is not the media type that will affect conversion since they fall within the same normal distribution.
The intent of the study is to help provide readers a framework in which they can inform their own content marketing strategies. Things still left to consider in our tested scenario of training for an Email Marketing 101 offer is frequency of use, opportunity costs, actuals of true opt-in conversion rates to benchmark against and so on.
Not all media types were tested. Typical content users find behind an email gate can be but are not limited to the following: e-books, videos, websites, coupon codes or discounts, pdfs, trials, podcasts, templates and other downloadable assets. To keep a focus on the study we chose to test three media types as our dependent variables: a website, pdf, and video in the scenario of a user learning more about the topic of email marketing.
Probability percentages do not reflect expected conversion rates. It is a known bias that study participants over report intended purchase behavior. The idea of measuring for probability in this study is to give indicators about future opt-in behavior.
In our study design, the data showed that
- PDFs are 34% more probable to convert than video.
- Websites are 24% more probable to convert than video.
- No difference between PDF and web content.