Dataflow functionalities and advanced features

Learn about Dataflow functionalities and advanced features in Productsup.

This document explains some of the functionalities and advanced features of the Dataflow, such as filtering, auto-mapping columns, and creating versions.

For a quick guide on how to map your columns, see Map your product data.

The import, intermediate, and export stages

There are three types of data stages in Productsup: import, intermediate, and export.

Import

The left section is your import data. Here, all the attributes listed come from your raw data imported from the data sources tab.

Intermediate

The middle section is where your feed gets restructured to a standardized Productsup feed: the intermediate stage. Here you make general optimizations that are relevant for all export channels. You can see the Intermediate as the base for all your export feeds.

Any adjustment made here affects all export feeds.

Export

On the right, you see the currently selected export feed. This stage contains all the attributes that are relevant for each respective export channel you have set up. The attributes are based on channel templates.

Any adjustment made here only affects the selected export feed.

data_flow.png

Filtering the Dataflow

If you have a lot of information in the Dataflow, you may consider filtering or hiding certain aspects. This can help achieve a cleaner view or locate the necessary information.

Filtering columns

If you are looking for a specific column in one of the stages:

  1. Navigate to the Dataflow

  2. Click the filter columns button on the top ribbon to activate it

    • upon doing so, a search field will appear above each stage

  3. Input the column name into the stage you wish to search

    filter_columns.png

Sort the columns in the Dataflow

This option allows you to define the order of columns as they appear in the Dataflow:

  1. Navigate to the Dataflow

  2. Click the sort by button on the top ribbon to expand the drop-down menu

  3. Click the method you would like to sort the columns by:

    • connection: displays connected columns at the top of the view

    • column name: displays columns in alphabetical order (A-Z)

    • export order: displays the export stage in the order in which the columns appear in the export file

      sort.png

Note

Sorting the columns does not affect the order of the actual columns being exported. This option purely customizes your view of the Dataflow.

Hide stages of the Dataflow

Sometimes you may find it easier to focus on certain stages/columns of the Dataflow, by excluding irrelevant information:

  1. Navigate to the Dataflow

  2. Click the hide/show button on the top ribbon to expand the drop-down menu

  3. Click one of the below options to hide:

    • hide unconnected columns: hides any unmapped columns

    • hide connected columns: hides any mapped columns

    • hide import: completely hides the import stage

    • hide export: completely hides the export stage

    • hide warning connections: hides warning connections

      hide.png

To show a hidden aspect, repeat the above process and click show for the relevant aspect.

Preview your import data

To get a quick preview of the product data for one product at the import level:

  1. Navigate to the Dataflow

  2. Click the preview button in order to activate it

    • you will see a preview of one product to the left of the import columns

    • the arrows can be used to scroll through products

      preview.png

Auto-mapping columns

Whenever you enable a channel template, certain columns on the export level are mapped automatically. The same applies to connections between Import and Intermediate stages. Column names are auto-detected and mapped onto the next stage if they align.

In case you deleted one of these connections or are missing some, you can manually trigger the auto-map for these columns connections:

  1. Navigate to the Dataflow

  2. Click the automap connections button on the top ribbon to expand the drop-down menu

  3. Select and click one of the below options to auto-map:

    • import to intermediate

    • intermediate to export

    • only box templates: this automatically adds any boxes which are attached to the channel template

    • all: auto-maps import to intermediate, intermediate to export, and box templates

    • map all import columns to export: this first attempts to auto-map the columns from import to export. If there is no match, then the column will be created as a custom column in the relevant stage.

      auto_map.png

Note

Auto-mapping connections between the intermediate and export stages are only possible for intermediate columns that have a connection to the import stage.

Saving a version of the Dataflow

Versions allow you to ‘back-up’ all columns, mapping, and boxes. This allows you to quickly roll back to a previous version of the Dataflow if needed.

To save a new version:

  1. Navigate to the Dataflow

  2. Click the versions button on the top ribbon to expand the drop-down menu

  3. Click save dataflow as a new version

  4. Input a name for your version and comment for an explanation (if desired)

  5. Click save

    version.png

Loading a saved Dataflow version

If you need to load the state of the Dataflow from an old version:

  1. Navigate to the Dataflow

  2. Click the versions button on the top ribbon to expand the dropdown menu

  3. Click on the version you wish to load

  4. You will be prompted to save your current Dataflow as a new version, click the relevant option

  5. Your version is now loaded and the loaded version is shown under the last loaded version section

    load_version.png

In the drop-down version menu, hovering over the comment icon allows you to see any comments related to saved versions.

Note

If you load an old Dataflow version, it will overwrite the current Dataflow and any changes that were made in between the last loaded version and now.

Delete connections, custom columns, or boxes from the Dataflow

You can delete boxes, connections, or custom columns in bulk:

  1. Navigate to the Dataflow

  2. Click the delete… button on the top ribbon to expand the dropdown menu

  3. Select and click one of the below to delete the relevant option:

    delete_bulk.png

Determining the source of an import column

There are three possible icons that can appear next to the import attribute’s name in the Dataflow. These indicate the origin of the attribute.

The connection icon indicates that the attribute was created by a data service.

The three dots icon shows that the attribute comes from an additional data source - by hovering over it you can see exactly which one.

The flag icon indicates that an attribute has been set as the Product Identifier.

determine_source_icons.png

Any columns without an icon come from (one of) your main source(s).

Column colors on the export level

Columns in the export stage have different colors.

Light blue columns are designated by the export channel to be mandatory.

Gray columns are designated by the export channel to be optional.

Dashed white and gray columns are custom columns you have created.

cus_colors.png