How to set up AB testing
1. Add multiple variants
To use the A/B testing feature you need to work with more than one Dataflow so you can compare the different variants. To create multiple Dataflow variants you need to add the same export channel more than once at least. In Exports click on Add Export again:
Try to give it a different name. If you use the same name, the platform will automatically add a number after the name to differentiate the Dataflows.
Example:
2. Select the main variant / activate deactivate a variant
2.1. Select main variant
You can set one main variant per channel. Change the main variant by clicking on On / Off. Note: If all variants are Off then the feed won't be exported - so make sure you select one, You can edit the export settings (destination, pricing, filename...) for only the one selected main variant (Don't worry, all variants will be merged into one feed at the end before the export).
Each time you change main variant, you will need to add the destination, set the filename etc. again.
2.2. Activate / deactivate a variant
In A/B, click on On / Off to activate / deactivate a variant.
Only activated variants (in the A/B column) will be used for A/B testing.
2.3. Activate / deactivate an Export
If all variants are set on Off for upload, then the channel is completely deactivated. To export to it again, activate at least one variant.
3. Manage Dataflow variants
Activated variants are available in Dataflow.
You can add, delete and edit connections. Remember that every change between Import and Intermediate affects ALL the export channels.
The products will go through one of the Dataflow variants, and then later be merged into one feed again but keeping only their unique Dataflow optimizations.
Example: Dataflow variant 1Attributes are connected, no box
Example: Dataflow variant 2
Difference with Dataflow variant 1:
Test example 1: one extra connection: additional_image_link
Test example 2: one connection less: gtin
Test example 3: one extra box: Static Value in Title ("TEST")
Even though all 3 test changes are shown in the same screenshot below, remember to only test one change at a time so that you can accurately judge which edit caused a performance change.
Result: the product feed contains 8 products.
After exporting the feed, it will look like this:
The first 4 products go through Dataflow variant 1. additional_image_link remains empty because there is no connection in the Dataflow for this attribute.
The last 4 products go through Dataflow variant 2. gtin remains empty because there is no connection in the Dataflow for this attribute.
Title contains only "TEST" for the last 4 products because of the box in Dataflow variant 2. This does not affect products going through Dataflow variant 1.
id | title | gtin | image_link | additional_image_link | price |
|---|---|---|---|---|---|
123 | Jeans 402 | 725184712258 | URL1 | 34.95 | |
234 | Microwave Grand Prix | 88698603659 | URL2 | 21.52 | |
345 | Laptop Powerstark | 725184712135 | URL3 | 35.47 | |
456 | Sneakers ConAir | 884420588689 | URL4 | 50.00 | |
567 | TEST | URL5 | URL2 | 150.23 | |
678 | TEST | URL6 | URL3 | 69.95 | |
789 | TEST | URL7 | URL4 | 79.95 | |
891 | TEST | URL8 | URL5 | 18354 |
All attributes connected at least in one active Dataflow variant will be exported.