Productsup
Fundamentals

How a feed moves through Productsup

The journey your product data takes inside Productsup: in from your data sources, standardized in the middle, and out to each channel.

13 min read

Your product data doesn't go straight from your shop to a channel. Inside Productsup it travels through a few clear stages: it comes in from your data sources, gets cleaned up and standardized in the middle, and then goes out to each channel in the shape that channel wants. Once you can picture that journey, the whole platform makes a lot more sense. Let's walk it end to end.

It starts with data sources

A data source is how your product data gets into a site. You point Productsup at where your data lives, whether that's a file on an FTP server, a shop platform, a PIM, or an API, and it pulls the data in. There are a few different ways to connect, covered in Product import methods and Import data from internal systems.

Under the hood, every data source is powered by a connector: a small piece of integration code that knows how to talk to one specific system. Productsup ships with hundreds of them, and if the one you need doesn't exist yet, you (or a partner) can build it. See Connector types and Create a connector for how that works, and Productsup Connect for the bigger picture on integrations.

Merging more than one source

A site isn't limited to a single source. You can combine data, and there are two ways to do it depending on what you're adding.

  • More items? Add another main data source. Main data sources stack up rows, so each one brings more products into the site.
  • More details about the items you already have? Add an additional data source. These extend your feed sideways, adding new attributes (like pricing or stock from a separate system) to products that are already there.

The trick that makes the second one work is the unique item identifier. Productsup uses it to match the extra data to the right product, so an ID, GTIN, or SKU has to line up across your sources. That's why setting a unique item identifier matters so much, and it's the same idea covered in Understanding product data.

Three stages, each a flat list of attributes

As your data moves through the platform, it lives in stages. Here's the key thing to hold onto: every stage is just a flat list of attributes. Think columns in a table, a field paired with a value, exactly like in Understanding product data. No nesting, no hierarchy, just a clean grid of fields. The data gets reshaped from one stage to the next, but the shape itself is always that same flat list.

There are three kinds of stage, and they're where Dataflow does its work.

The import stage

This is your raw data, exactly as it arrived from the data source. You can't edit it here. It's the untouched starting point, and everything downstream builds on it.

The intermediate stage

This is the shared middle ground, and its attribute list isn't something you build by hand. The platform assembles it for you as the sum of every attribute from every export enabled in the site. Turn on a Google export and an Amazon export, and the intermediate stage holds the combined set of fields those two channels need.

Left alone, that would mean a lot of duplicates, because different channels ask for the same fact under different names. One wants title, another wants titel, a third calls it name, but they all mean the product's title. To avoid three near-identical attributes, those export fields are aliased to a single intermediate attribute. The title shows up once in the intermediate stage, you optimize it once, and each export derives its own title, titel, or name from that single source.

That's what makes the intermediate stage so useful: any optimization you make here flows through to every export. It's the place for the cleanup that's true no matter where a product is headed, like fixing a title format or normalizing a category. Doing the work once here, instead of repeating it per channel, is one of the simplest ways to keep a site fast (more on that in Data processing performance).

The export stages

Here's where it branches. Each channel you sell on gets its own export stage, holding the exact attributes that channel requires. Sell on five channels and you've got five export stages, each with its own rules and its own required fields. You pick which export you're looking at from the dropdown, and optimizations you make in one export stay in that export. This is how the same products end up correctly formatted for Google, Amazon, and a retailer all at once, which is the whole reason data has to be transformed in the first place.

Mapping and optimizing in Dataflow

So how does data actually move from one stage to the next? Through Dataflow, and the core action is attribute mapping: you connect an attribute in one stage to an attribute in the next, and the data flows along that link. Map your import title to the intermediate title, then on to each export's title field, and you've built a path from source to channel. See Use Dataflow to map your attributes from import to export for the full walkthrough.

Mapping moves the data. Rule boxes change it along the way. A rule box is a small transformation you drop onto an attribute: trim whitespace, swap a value, build a new field from two others, exclude a product. You can apply them in Dataflow or in Data View, and there's a whole library to pick from in Add a rule box. Apply them in the intermediate stage to hit every export, or in a single export stage to fix just that one. When you want to see the effect on real values, Data View shows you the data at any stage.

Data services for the heavier lifting

Rule boxes handle the everyday changes. When you need something bigger, that's a job for data services. A data service is a tool that enriches or restructures your feed in ways a single rule box can't: generating unique IDs, crawling landing pages, converting currencies, splitting one product into variants or merging variants into one, even AI tasks like removing image backgrounds. See Enhance your import data through Data Services and Restructure your data in Data Services for what's on offer.

Like data sources and exports, data services run on connectors, so the same Connect tooling lets you build a custom one. See the data service connector type if you've got a transformation that needs to live in code.

Export

At the end of the line, the prepared data leaves the platform. Each export sends its stage out through an export connector, in the format and to the destination that channel expects, whether that's a file dropped on a server or a live API push. You add channels in Add export, point them at a destination in Set up an export destination, and if no template fits you can build your export from scratch. Some channels take the full feed every run; others support delta updates that send only what changed. Either way, the relevant export connector handles the handoff. See Export data feeds for the full picture, and the export connector type if you're building your own.

In short

Data comes in through data sources (powered by connectors), where multiple main sources add items and additional sources add attributes matched by a unique identifier. It then flows through three kinds of stage, each a flat list of attributes: a read-only import stage, a shared intermediate stage, and one export stage per channel. Dataflow maps attributes from stage to stage, rule boxes and data services transform them, and export connectors send the finished feed out to each channel.

On this page

Still stuck?

Reach out to our support team and we’ll help you get unstuck.

Contact support