Overview

Required plan

This feature or setting is available to all customers on any Recurly Engage subscription plan.

Prerequisites & limitations

• You must have Company or App Administrator permissions in Recurly Engage.
• At least one prompt must exist in draft or review status before creating an experiment.

Definition

An experiment runs multiple versions (variations) of a prompt against each other—optionally including a control group—to measure performance based on custom goals and decide which version to deploy.

Key benefits

• Data-driven decisions: Use real user interactions to choose the most effective prompt variant.
• Controlled testing: Isolate single changes (copy, design, trigger) to understand their impact.
• Seamless rollout: Promote the winning variation automatically when the experiment completes.

Key details

Guide

Experiments follow these steps:

(Optional) Add a custom goal

Define what success looks like—e.g., a click or purchase—before creating variations.

1. In Recurly Engage, go to Settings > Usage Tracker and add a new tracker.

2. Return to your prompt and attach the custom goal under Goals.

Create the experiment

1. Open your prompt and click + New experiment.

2. Give your experiment a clear name—see the overview on experiments for best practices.

Configure variations

Default setup: By default, your experiment contains the Original prompt. If you added a custom goal, you may also include a Control group to measure the impact of showing vs. hiding the prompt.

Add a variation: Click Add variation, name it to reflect the change (e.g., “New headline”), and modify one or more aspects—text, design, triggers, or actions.

Use the editor to scroll down and adjust settings as needed:

Allocate traffic

Set the percentage of users who see each variation—ensure that total equals 100%.

Launch and monitor

1. Click Start experiment and confirm.

2. Monitor performance in real time—click-through and conversion rates update within minutes, depending on traffic.

3. When ready, click Use This on the winning variation to replace the control and end the experiment.

That’s it—your experiment is live, and you can review results to make data-driven optimizations to your prompts.