Productsup AB testing explained

It allows you to compare different versions of your product data structure or content to determine which one performs better.

E.g. Test a new custom label, or see what impact adding information to the product title has on the performance. If it is positive, you can edit the title accordingly for all applicable products in the future.

Yes.

You can continue using the legacy "Exports" tab without problem but if you wish to do A/B testing you need to select "Exports".

For each export channel (let's say Google Merchant Center) you can activate a certain number of Dataflow variants. Based on the number of Dataflows that you set up, our algorithm will split your products in the same amount of groups. With each Dataflow variant you can create different versions of the same feed.

E.g. If you activate 2 variants, ~50% of your products will run through one, and the other ~50% through the other. Activate 3 variants and your products will be split ~33% / ~33% / ~33%, and so on.

No. Once you have edited and prepared your different Dataflow variants and want to export your products for testing, all Dataflows will merge, meaning all products will be placed in ONE feed again so that only one final feed is exported for your selected channel. Within this same feed some products will have version A, and others version B of which ever comparison you are doing.

You will have a different Dataflow for each variant. The pre-defined export template will be the same for every variant within an export. But you can choose to add boxes for one Dataflow only, if this is the element you want to A/B test.

Similarly, you can choose to delete an attribute (or remove a connection) in a Dataflow variant.

Please note: Any data edits made between Import and Intermediate feeds will still impact ALL the export channels (as before).

When you determine the number of Dataflow variants, one of them must be activated as your "main" feed. This is just so that you can select the export destination and does not affect your final feed template. The final export feed will be a combination of all the existing attributes or connections. So if Dataflow A is your "main" export and Dataflow B has an attribute added to it, the export will still include that attribute. Products in Dataflow A will simply be blank for this attribute.

There are no technical requirements. You can even use your own tracking if you want by adding a different parameter for each Dataflow variant.

Don't do these common mistakes when using A/B testing:

To use the A/B testing feature you need to work with more than one Dataflow so you can compare the different variants. To create multiple Dataflow variants you need to add the same export channel more than once at least. In Exports click on Add Export again:

Try to give it a different name. If you use the same name, the platform will automatically add a number after the name to differentiate the Dataflows.

Example:

2.1. Select main variant

You can set one main variant per channel. Change the main variant by clicking on On / Off. Note: If all variants are Off then the feed won't be exported - so make sure you select one, You can edit the export settings (destination, pricing, filename...) for only the one selected main variant (Don't worry, all variants will be merged into one feed at the end before the export).

Each time you change main variant, you will need to add the destination, set the filename etc. again.

2.2. Activate / deactivate a variant

In A/B, click on On / Off to activate / deactivate a variant.

Only activated variants (in the A/B column) will be used for A/B testing.

2.3. Activate / deactivate an Export

If all variants are set on Off for upload, then the channel is completely deactivated. To export to it again, activate at least one variant.

Activated variants are available in Dataflow.

You can add, delete and edit connections. Remember that every change between Import and Intermediate affects ALL the export channels.

The products will go through one of the Dataflow variants, and then later be merged into one feed again but keeping only their unique Dataflow optimizations.

Example: Dataflow variant 1Attributes are connected, no box

Example: Dataflow variant 2

Difference with Dataflow variant 1:

Even though all 3 test changes are shown in the same screenshot below, remember to only test one change at a time so that you can accurately judge which edit caused a performance change.

Result: the product feed contains 8 products.

After exporting the feed, it will look like this:

All attributes connected at least in one active Dataflow variant will be exported.