Optimize your data in Dataflow

Optimize your data in Dataflow in Productsup.

You can add optimization rules (boxes) directly in the Dataflow. This can enable you to clean up your data and optimize it. This can be used in place of, or in conjunction with, adding rules in the Data View. Using the Dataflow for this task allows you a greater overview of all rules implemented, but less oversight over your actual product data.

For more information on how to box maintenance such as editing or deleting and the general functionalities of boxes, see Advanced box options.

Overview of all boxes

Each box is represented by a bubble in Dataflow. This allows you to see how many boxes are used in certain connections.

  • Green bubbles are boxes implemented between the import and intermediate stages

  • Blue bubbles are boxes implemented between the intermediate and export stages

  • Gray bubbles are paused boxes

  • Yellow bubbles are boxes containing skip rules

When you hover over a connection, you can see a list of all implemented boxes.

box_overview.gif

In order to view the configuration of the boxes in more detail:

  1. Click on the column itself or the arrow icon when hovering over a column

    • this will open a tab on the right-hand side

Adding boxes in the Dataflow

To add a rule box:

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Click on an intermediate or export column where you want to add a box

    • this will open a tab on the right-hand side

  4. Click on the add box drop-down menu to expand it

  5. Search for and select the relevant box, before clicking on it

  6. Configure the box (if necessary)

  7. Click save

adding_box_small.gif

Note

You can only enter boxes on the intermediate or export stages. This is because the import stage contains your data source as you provide it and cannot be directly changed.

Adding a box using the large view

If you wish to have more information about the boxes and greater filtering options, you may consider using the large view:

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Click on the arrow icon when hovering over a column

    • this will open a tab on the right-hand side

  4. Locate the relevant box using the filtering options on the top ribbon or finding it in the list

  5. Drag the box into the relevant stage: in the green area for import-intermediate, in the blue for intermediate-export

  6. Configure the box (if necessary)

  7. Click save

adding_box.gif

Editing box setups

You can also edit any boxes you have previously set up in the Dataflow. For more information on box maintenance such as editing or deleting, see Advanced box options.

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Click on the column itself or the arrow icon when hovering over a column

    • this will open a tab on the right-hand side

  4. Edit the current box setup as desired

  5. Click save

Filter for certain boxes in the Dataflow

Once you have implemented optimization rules through boxes, you can also filter for certain boxes in the Dataflow:

  1. Navigate to the Dataflow

  2. Click into the filter boxes field

  3. Enter the name or keyword of the box you wish to search for

  4. All other boxes will then be hidden in the dataflow, leaving only boxes matching your search term

box_search.gif

Appending a column to another column

You can append a column to another column directly in the dataflow. This could be useful when creating a description column for example, and you want to append the color and size columns to your title column.

You can, of course, also add the append/prepend column box directly.

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Map a column to an already-connected column in the next stage

  4. Click append

append_column.gif

The column is now appended. You can see this by the fact that a slightly transparent indirect connection has been created between the two stages.

Replacing a column with another column

You can also replace a column with another column, without the need to delete the connection first.

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Map a column to an already-connected column in the next stage

  4. Click replace

The column is now replaced.

Copying rules from one column to another

If you have a set of rules and wish to copy it from one column to another:

  1. Navigate to the Dataflow

  2. Select the relevant export channel (if necessary)

  3. Click on the nodeof the column which you wish to copy the rules from

    • an orange circle will appear

  4. Click on the nodeof the target column where you wish to copy the rules to

    • all columns where you can copy the rules to have a white circle appear

  5. Click copy

copy_rules.gif

Note

In the target column, you have to have no rules in order to copy it.

Implementing rules in bulk across multiple sites

If you need to add the same rules to multiple sites, you can consider using the bulk option to save you time.

Note

You need to have exactly the same column names in each site for this bulk option to function.

  1. Create the boxes in one site first

    • this site will be the master site that is used to copy the rules to the other sites

  2. Navigate to either the account level or the project level

  3. Click the bulk option button

    access_bulk.png
  4. Select the site where you created the rules, before clicking the next step

  5. Click the option add Dataflow boxes and then next step

  6. Select whether the rules were set between the import-intermediate stage, or between the intermediate-export stage

    • if you chose the intermediate-export stage, please select an export channel where the rules were set from the dropdown export menu. Please note that this export channel has to be in all sites.

  7. Select the column(s) where the rules you want to copy are found

    • a brief overview of the boxes in each connection will be shown here

  8. Select the project(s) where your target sites are

  9. Select your target site(s) or target channels

  10. Click execute