LLM.txt
Generate and host an LLM.txt file via Productsup for organic AI discovery and web scraping guidance.
Overview
Productsup enables you to source data seamlessly from your product data feed, outputting two key elements that are essential for llm.txt:
- An llm.txt file which your WebDev team will embed within your website.
- A JSON file, hosted by Productsup, which is accessible only to the OpenAI IP address.
llm.txt is a temporary method developed to allow LLMs to crawl or scrape data from your website. Ultimately llm.txt will be replaced as the industry catches up via designated agentic-commerce feed formats (think UPC). Until then, this is a great way to insure your product data is available for an AI agents who directly search your website.
Steps in Productsup
- Add the 'OpenAI' export.
- Add the OpenAI (llm.txt) destination.
- Configure the Dataflow.
There is no required or standard schema for the product data contents passed in the file. Connect columns that contain the data your org wishes to be available for AI agents.
When the configurations are complete, process the site. Two events will occur:
- Productsup will generate the LLM.txt template, embedded in the export page.
- PUP sends the product data to OpenAI private IP endpoints.
Steps for your WebDev Team
- Your team must retrieve the LLM.txt template from the export page, and provide it to your WebDev team to embed on your website.
- Your WebDev team can get more details via the primary llm.txt documentation.
How is this guide?