Direct Integration with Operational Data
Use host application’s operational data for qualifying and targeting your users. Leverage your established technology stacks to access operational data in real time, securely and with very low latency. No more operational overhead and legal risks of running experiments against extracts. No more server-to-server calls across the Internet in your application path.
Experiment Definition Model
We’ve organized all your experiments in a single human readable file called experiment schema. It uses a set of powerful abstractions, collectively known as the Experiment Definition Model (XDM), to incorporate definitions of all experiments running on a host application. Experiment schema is managed by Variant server, leaving client code completely free from experimentation lint,— the application code that is concerned with instrumentation, rather than implementation of new experiences.
Developers love Variant because Variant was created with developers in mind. There are no clunky Web consoles to click through in order to create an experiment. Every phase of a Variant experiment life cycle is programmable and therefore fits right with the established development ops. Variant server is highly extendible via a fully-featured server side API, which provides a powerful framework for enrichment of the default semantics with custom behavior.
Product owners no longer have to listen to the highest paid voice in the room. Rather, they have the power to validate and refine every new product feature through controlled experiments. When a company adopts the culture of pervasive testing, all new product features are subject to experimental validation. Product owners have the opportunity to revise feature specs in vivo, as the feature goes through successive rounds of refinement and retesting. Occasionally, when they don’t perform as initially expected, new features will have to be abandoned, saving you the embarrassment of a feature recall.
Developers love Variant because it’s built by developers and for developers. Instrumenting an experiment involves no clunky GUI to click through. Rather, experiment metadata lives in a human readable schema files, which are treated as any other source code. Variant lets you instrument actual application code, so that if and when the new code path wins, no re-implementation is required. A single Variant server can support multiple application domains via experiment schemata. Even if you only have a single application, you will find the ability to deploy separate schemata for every QA build on every code branch very attractive.
Marketers have practiced A/B testing for many decades. In the past 50 years they have experimented with physical store merchandising, direct mail and, more recently, email campaigns. But it wasn’t until 2000s that the idea of experimenting on Web applications caught on with forward looking engineers and product owners. However, in many enterprises the marketing department continues to be the energy behind the paradigm shift, and frequently foots the bill not only for the vendor service but for the marketing devs. Variant makes all this obsolete, placing experimentation right where it belongs: into the laps of the CTO.
The earliest applications of online controlled experiments are in e-commerce, where people have been experimenting with anything from the color of the BUY button to suggestions algorithms, to third party integrations. While these use cases continue to be relevant to the experimentation agenda, new applications of online commerce have generated new use cases for experimentation: online booking, consumer-to-consumer marketplaces, online lending, etc.
Interactive Voice Response
Variant architecture makes no assumptions about the host application, except that it is interactive. While Web and mobile apps are ubiquitous examples of such applications, there are many others, and most can benefit from experimentation, like Interactive Voice Response (IVR). Whenever you contemplate a change to your inbound phone menu, Variant can help you instrument that change as a controlled experiment. Furthermore, many enterprises maintain support call centers and must keep the phone and the online user experiences in sync. With Variant, you can easily experiment over a feature that spans both online and voice experiences and keep them tightly integrated.
Application User Interfaces (APIs)
APIs can be viewed as most basic interactive applications, featuring a single request/response cycle. While most APIs are of the imperative nature (they simply carry out a well defined, immutable task, like process a credit card payment), some APIs are interrogative, like a personalized product suggestion. Such APIs typically package complex algorithms that are subject to extensive tuning and constant improvement. Interrogative APIs benefit greatly from ongoing controlled experiments, which Variant makes a breeze to instrument.