Building a Brand Value Proposition through Statistically Designed Experimentation

By Alison Burnham, CIO, ScoreBig

Alison Burnham, CIO, ScoreBig

To give this article some context I will start with a description of ScoreBig. We are a completely new kind of live event ticketing company. We work directly with the biggest names in sports and entertainment to get exclusive access to great seats to the biggest events in your city - sports, concerts, theater and family shows. Our Name a Ticket Price feature lets you score great savings up to 60 % off box office price with no surprise fees. Your tickets are guaranteed to be authentic and will arrive on time for the game or show.

"We have long been applying analytics to our business problems and we use it for our pricing algorithms, managing our inventory, and website A/B testing"

This description is relevant both to explain what we do and also because the shaping of this description is the subject of this article. At ScoreBig we have long been applying analytics to our business problems. We use it for our pricing algorithms, managing our inventory, and website A/B testing. We also use it for analysis of marketing research to drive our overall value proposition. However, in 2015 we had a bigger brand proposition challenge than ever before. The business was evolving–for example we had expanded our site offering to include new ways to buy tickets. We were also experiencing a very fast changing landscape within ticketing in general as primary (straight from the venue) ticketing sites such as Ticketmaster and secondary (re-sale and aggregator sites) such as SeatGeek and Vivid Seats competing for market attention. Our own role in this landscape is primary that of the value player but we felt that we had a stronger and more unique value proposition than something like ‘Save on Tickets’ could express. In this case we had several aspects to combine to come up with a single value proposition. Because of this the usual discrete variable testing was not going to work for us. This drove us to leverage factorial experimentation to test multiple factors at the same time.

We were working on a construct of four brand pillars–Value, Convenience, Selection and Trust. Within each of these we had multiple ‘proof points’ such as for Convenience where we had five different proof points to test:

• Easy to use

• Has an interactive mobile app

• Has dedicated, local, live customer service 7 days a week

• Has interactive seat maps

• Where you can buy tickets days, weeks, months in advance

We also had 5 proof points for Selection, 6 for Value and 5 for Trust. We wanted to not only know how they were received but what combinations drove the best response. To test all of that required 5 x 5 x 6 x 5 = 750 combinations. We ended up with a test for 3,000 participants or approximately 4 observations per combination. However, for any given proof point we had observations (with a variety of the other value pillars) for 1/5 or 1/6 of the sample. So each value pillar had a robust sample of 500 or 600 participants. To deploy the survey quickly and easily we used Survey Monkey. Our value pillars were shown as bullet points for a company description using their A/B testing capability (each value pillar was its own 5 or 6 factor A/B test). Our response metrics were based on the answers to three questions—one around willingness to try the site, on around the uniqueness of the site and the other on the believability of the description. We are looking for a robust, defendable brand proposition that we can use to inform our marketing and product development going forwards.

Our learnings from this test were very illuminating. We found that overall the company brand proposition was strong. It was particularly strong for consumers who also showed price sensitivity for live event tickets (a descriptive question asked in the survey). It also showed that for 3 of the brand pillars—Convenience, Selection and Trust, the proof points were mostly interchangeable with few small exceptions that helped us shape our messaging. This was great news as it means we can vary our brand messaging with different reasons to believe to keep the message fresh without losing its impact.

For Value however (which is arguably our most important brand pillar) it became clear that we did have some differences and that we also need to do some more testing around how we discuss value. For example - some of the reasons-to-believe (RTBs) scored high on one aspect but low on another. Also, the ‘no fees’ message scored higher on appeal (likelihood to visit) but lower on believability. This suggested we needed a more convincing way to express this feature. We are currently going out to market with a further test to refine our value messaging.

In closing–this ended up being a very fast (test deployed and analyzed in under 4 days) and informative test for us. We also felt comfortable in the results because each had been tested with a variety of the other brand pillar messages and shown a consistent result. We are excited to continue to refine our marketing message over 2016. In today’s very crowded online market, the need to get our points across in a compelling and memorable way is very high. We believe that once people come and experience the site, that the events and deals available will speak for themselves. With multiple and expanding ticketing options for the American event-goers, getting that initial trial is the challenge and the opportunity.