A/B testing

Pro - CreatorAvailable in the Creator tier

A/B testing analysis is a Pro-only feature. Upgrade to Burst Pro

Everything in the Creator tier is also available in the Business & Agency tiers.

Burst Pro automatically detects A/B test variants in your campaign data and enriches the statistics datatable with winner and significance information. No separate test configuration is required — detection is driven entirely by naming conventions in your UTM or campaign parameters.

How it works

When Burst renders a campaign conversion datatable, it scans every row for campaign parameter values that contain the string variation-a or variation-b (case-insensitive). Rows that share the same normalized campaign key but differ only by the -a / -b suffix are grouped as a test pair.

A query only counts as a campaign query when its selected columns include at least one of the campaign parameter keys below. Both source and utm_source map to the UTM source column, so a campaigns datatable that selects either key is treated as a campaign query.

Key	Description
source	UTM source
utm_source	UTM source, as used by the campaigns datatable
medium	UTM medium
campaign	UTM campaign
term	UTM term
content	UTM content

Detection runs in one of two modes depending on the metrics selected by the query:

• Conversion mode — the query selects conversion_rate or conversions, or filters by a goal_id. The winner is decided on conversion rate.
• Sales mode — the query selects sales or sales_conversion_rate. The winner is decided on sales conversion rate.

Once a complete pair (one A variant and one B variant) is found, Burst:

1. Determines the winner based on the active mode's rate (conversion rate, or sales conversion rate in sales mode). If rates are equal, the variant with higher traffic wins.
2. Runs a two-proportion z-test at 95% confidence to assess statistical significance.
3. Annotates every matched row with winner, significant, and is_ab_test fields that the datatable UI uses to highlight results.

In sales mode, the per-variant rate is read from sales_conversion_rate and the per-variant count is read from sales; in conversion mode they are read from conversion_rate and conversions. Either value is computed from the available data when it is not already present on the row.

Naming convention

To have Burst recognize your variants, include variation-a or variation-b anywhere in a campaign parameter value.

Variant	Trigger string (case-insensitive)
A	variation-a
B	variation-b

Example UTM content values:

utm_content=hero-banner-variation-a
utm_content=hero-banner-variation-b

The two rows are grouped by stripping the trailing -a / -b, so both resolve to the key hero-banner-variation. All other campaign parameter values on the row are included in the grouping key, so tests across different campaigns are kept separate.

Significance calculation

Burst uses a two-proportion z-test. The result stored in each row's significant field is one of three string values:

Value	Meaning
significant	The difference between variants is statistically significant at the 95% confidence level (
still_running	Not enough data yet to draw a conclusion.
no_winner	Total sessions ≥ 300 but the absolute conversion-rate difference is < 2 percentage points — the test has reached a futility cutoff.

Thresholds used internally:

Threshold	Value
Z-score for 95% confidence	1.95
Minimum sessions for futility check	300
Minimum effect size (futility)	2 percentage points (0.02)

Row fields added

After processing, every row that belongs to a detected A/B test receives the following additional fields:

Field	Type	Description
is_ab_test	bool	true for both the A and B rows.
winner	bool	true on the winning row, false on the losing row.
significant	string	One of significant, still_running, or no_winner.

Hooks & filters

burst_datatable_data

Filters the flat row array returned for the statistics datatable. The A/B Tests class hooks into this filter at priority 10 to append A/B test metadata to matched rows. You can hook at a higher priority to act on the enriched data.

Parameters:

Parameter	Type	Description
$data	array	Flat array of datatable rows. Each row is an associative array of column name → value.
$query_data	Burst\Admin\Statistics\Statistics_Query	The query definition used to fetch $data, including the select columns and campaign filters.

Return value: The filtered $data array, potentially with is_ab_test, winner, and significant keys added to matched rows.

Example — reading A/B results after Burst has processed them:

Show code

add_filter( 'burst_datatable_data', function( array $data, $query_data ): array {
    foreach ( $data as $row ) {
        if ( ! empty( $row['is_ab_test'] ) ) {
            $label       = $row['winner'] ? 'WINNER' : 'loser';
            $significant = $row['significant']; // 'significant' | 'still_running' | 'no_winner'
            error_log( sprintf( '[Burst A/B] %s — %s — significance: %s', $row['utm_content'] ?? '', $label, $significant ) );
        }
    }
    return $data;
}, 20, 2 );

Example — suppressing A/B test annotations entirely:

Show code

add_filter( 'burst_datatable_data', function( array $data, $query_data ): array {
    foreach ( $data as &$row ) {
        unset( $row['is_ab_test'], $row['winner'], $row['significant'] );
    }
    return $data;
}, 20, 2 );

caution

The A/B detection runs only when the datatable query selects at least one of sessions, visitors, or pageviews, and selects at least one campaign parameter key (source, utm_source, medium, campaign, term or content), and is identified as either a campaign conversion query (selects conversion_rate or conversions, or filters by goal_id) or a campaign sales query (selects sales or sales_conversion_rate). Datatables that do not meet these conditions are returned unchanged and will not contain A/B test fields.

burst_license_is_valid

Whether Pro capabilities — including A/B test detection — are treated as active is resolved through burst_license_is_valid(), a wrapper around the burst_license_is_valid filter. The shared code defaults to false; Pro registers a callback that returns the real license state, so A/B annotations only run when the filter resolves to true.

Parameters:

Parameter	Type	Default	Description
$is_valid	bool	false	Whether a valid Pro license is active. Pro overrides the default with the resolved license state.

Return value: bool — true when a valid Pro license is active, otherwise false.

Example — reading the current license state:

Show code

if ( burst_license_is_valid() ) {
    // A valid Pro license is active, so A/B detection and other Pro
    // capabilities are available.
}

Requirements

• A valid Burst Statistics Pro license, as resolved by burst_license_is_valid().
• Campaign tracking enabled and conversion goals configured.
• UTM / campaign parameter values must follow the variation-a / variation-b naming convention.
• Both the A and B variants must appear in the same datatable result set. If only one variant is present in the selected date range, no A/B annotations are added.