LLM.txt
Generate and host an LLM.txt file via Productsup for organic AI discovery and web scraping guidance.
Overview
Productsup enables you to source data from your product data feed and output two key elements required for llm.txt:
- An llm.txt file for your web development team to embed on your website.
- A JSON file hosted by Productsup, accessible only to OpenAI IP addresses.
llm.txt is a temporary method that allows LLMs to crawl or scrape data from your website. It will be replaced as the industry adopts designated agentic commerce feed formats. Until then, it is a practical way to ensure your product data is available to AI agents that search your website directly.
Steps in Productsup
Add the OpenAI export.
Add the OpenAI (llm.txt) destination.
Configure the Dataflow.
There is no required or standard schema for the product data included in the file. Connect the columns that contain the data your organization wants available for AI agents.
When configuration is complete, process the site. Two events occur:
- Productsup generates the LLM.txt template, embedded in the export page.
- Productsup sends the product data to OpenAI private IP endpoints.
Steps for your web development team
Retrieve the LLM.txt template from the export page and provide it to your web development team to embed on your website.
Your web development team can find more details in the primary llm.txt documentation.
How is this guide?