For the past several months, the AdHatchery team has been working furiously to build our product. Judging success and progress in the product development cycle is fairly binary. The features you build either work or they don’t. Simple as that. As we move out of this initial development phase and into beta testing however, life gets a little more complicated. The question is no longer so much whether features technically work (although that still keeps me up at night), but whether they are actually useful and easy to use.
Yesterday I spent the day chatting with other entrepreneurs and venture investors about the best way to run a beta test and gather input to continually improve the product.
Saar Gur from Charles River and
Bob Geiman at Polaris were particularly insightful, but everyone I spoke with seemed to share the same three pieces of advice:
1) Pick beta testers that share your core visionAt this point, we have a number of agencies that have raised their hands to participate in the AdHatchery beta program. Most of these agencies found their way to the AdHatchery website over the last month or so, saw only a log-in box, looked carefully for a tiny light-grey “Contact” link at the bottom of the page, sent me an email and asked to get involved. This tells me these people are committed to solving a real pain point.
We hope that all of these early users will engage deeply with the system, provide rich feedback and put up with the inevitable bumps in the road because they share our vision and view of the world – that more openness and transparency can improve our industry. If you fit this profile and want to be a champion within your agency, please reach out to me at
clay@adhatchery.com.
2) Set clear metrics
We’re focused on proving three things in the beta: 1) that we can easily on-board agencies and engage planners to fill out client profiles, 2) that sellers with access to these profiles will create and submit thoughtful, targeted ideas through AdHatchery and 3) that planners find value in judging sellers through targeted pitches on the system. As such, we’ll be looking at a number of hard metrics in particular:
- # of users (planners and sellers)
-
# of buyer “accounts” – ie. Liberty Mutual or Dunkin Donuts.
-
# of categories (brands or business lines or objectives, etc.) per account – ie. Liberty Mutual’s Responsibility Project or Dunkin’ Donuts’ Regular Refills program.
-
# of sellers that receive access to each account-
# of ideas submitted to each account-
# of ideas “starred” by each account-
Frequency of log-in and time spent on the site by both planners and sellers - As we roll out more buyer feedback tools, we’ll also be looking hard at
how often buyers give specific feedback to sellers on their ideas through the system.
3) Create feedback loops We’re hungry for feedback. We want to know exactly what people like and, more importantly, what needs improvement. We’re not afraid of criticism – in fact, we thrive on it. We’re spending time in person gathering input, but we’re also implementing tools in the application to provide for real-time feedback as users are working. The first is
Zendesk – you’ll notice a large “Support” tab on the left side of the screen throughout AdHatchery. Simply fill out your question or problem, big or small, and we’ll get to work on it immediately. We’re also looking at implementing various survey tools like
SurveyMonkey and
Kampyle to gauge user satisfaction, so you may be prompted to fill out quick survey from time to time. We'll keep them short and on point, and will treat your input like gold.
