Build a Real Time Product Feed with OpenClaw for Workflow Automation

TL;DR

This tutorial walks you through building a production grade product feed that powers Google and Meta catalogs using OpenClaw. You will define a clean product schema, transform it into platform specific columns, and publish a single stable URL that ad platforms can fetch on a schedule for workflow automation. By the end, you will have validations, monitoring, and a rollback plan so the catalog stays fresh without manual exports.

What you will build

You will create a reproducible pipeline that reads product and variant data from your source of truth, renders a CSV feed for each destination, and uploads the files to cloud storage on a schedule. Merchant Center and Meta Catalog will pull from that URL automatically. We will also add guardrails so a bad deploy does not ship empty feeds.

Internal resources to help you plan and extend this build include the overview of AI marketing automation features and the hosted ButterGrow environment if you prefer managed infrastructure. For context on why feeds matter to paid media performance, see the related piece on why creative feeds are reshaping paid media.

Prerequisites

• An OpenClaw workspace with access to source data
• Object storage such as S3 or GCS, plus a public read bucket or CDN
• A SQL source or API that exposes product, variant, inventory, and image URLs
• Access to Google Merchant Center and Meta Catalog

Architecture overview

At a high level, the pipeline has four stages: extract from your source of truth, transform into destination specific schemas, publish feed files to a stable HTTPS URL, and monitor for correctness. OpenClaw playbooks coordinate the flow, retries, and alerting as automated workflows so you do not hand run tasks. The same core dataset drives both Google and Meta with small mapping differences in an agentic workflow.

Platform field differences

The two platforms share many fields but treat some attributes differently. Use this quick map when planning your transforms.

Concept	Google column	Meta column	Notes
Unique ID	id	retailer_id	Prefer stable SKU. Do not reuse IDs for new products
Title	title	name	Append variant descriptors for clarity
Description	description	description	Plain text, no HTML
Link	link	url	Product detail page with tracking params added server side
Image	image_link	image_link	Additional images as image_link_additional and additional_image_link
Price	price	price	Use currency format like 19.99 USD
Sale price	sale_price	sale_price	Include sale_price_effective_date for Google
Availability	availability	availability	in stock, out of stock, preorder
Brand	brand	brand	Prefer manufacturer brand, not store name
GTIN	gtin	gtin	Use GS1 valid values when available
MPN	mpn	mpn	Required if GTIN missing in some categories
Condition	condition	condition	new, refurbished, used
Google product category	google_product_category	google_product_category	Meta accepts it but does not require
Item group	item_group_id	item_group_id	Parent identifier for variants

For authoritative definitions, review the Google Merchant Center product data specification and the Meta Catalog data feed guide.

Build the pipeline

We will implement the pipeline as a single OpenClaw playbook with three jobs: fetch source rows, transform rows into per destination frames, and publish files to storage. Then we will add scheduling, validation, and alerts.

Step 1Define a clean product schema

Start with a normalized model that captures both parent products and sellable variants. Keep source naming consistent and typed so transforms are simple.

// src/models/product.ts
export interface ProductVariant {
  sku: string;
  parent_id: string; // stable product ID
  title: string; // base title without size or color
  variant_title: string; // example: "Tee Shirt - Large - Red"
  description: string;
  currency: string; // USD, EUR
  price: number; // 19.99
  sale_price?: number;
  sale_start?: string; // ISO 8601
  sale_end?: string; // ISO 8601
  brand: string;
  gtin?: string;
  mpn?: string;
  condition: 'new' | 'refurbished' | 'used';
  availability: 'in stock' | 'out of stock' | 'preorder';
  product_url: string; // canonical PDP URL
  image_url: string; // primary image
  additional_images?: string[];
  google_product_category?: string;
  updated_at: string; // ISO timestamp from source
}

If your source is a warehouse, create a SQL view that emits one row per variant with the above columns. If your source is an ecommerce API, hydrate missing attributes like brand or google_product_category from a lookup table so the transform stage does not branch on source quirks.

Step 2Configure the OpenClaw playbook

Create a playbook that reads from SQL and writes files. This example uses a parameterized date filter so you can run full and incremental builds.

# playbooks/catalog-feed.yaml
version: 1
name: catalog-feed
vars:
  incremental_since: "{{ env.INCREMENTAL_SINCE | default('1970-01-01T00:00:00Z') }}"
jobs:
  fetch:
    uses: sql.query
    with:
      connection: prod_warehouse
      sql: |
        SELECT sku, parent_id, title, variant_title, description,
               currency, price, sale_price, sale_start, sale_end, brand,
               gtin, mpn, condition, availability, product_url, image_url,
               additional_images, google_product_category, updated_at
        FROM mart.product_variants
        WHERE updated_at >= TIMESTAMP '{{ vars.incremental_since }}'
  transform_google:
    needs: [fetch]
    uses: node.run
    with:
      entry: scripts/transform-google.ts
      input: "{{ steps.fetch.rows }}"
  transform_meta:
    needs: [fetch]
    uses: node.run
    with:
      entry: scripts/transform-meta.ts
      input: "{{ steps.fetch.rows }}"
  publish:
    needs: [transform_google, transform_meta]
    uses: storage.upload
    with:
      provider: s3
      bucket: "marketing-feeds"
      region: "us-east-1"
      files:
        - path: "out/google_products.csv"
          dest: "feeds/google/products.csv"
          cache_control: "public, max-age=900"
          etag_from_content: true
        - path: "out/meta_products.csv"
          dest: "feeds/meta/products.csv"
          cache_control: "public, max-age=900"
          etag_from_content: true
  validate:
    needs: [publish]
    uses: node.run
    with:
      entry: scripts/validate-feeds.ts
  notify:
    if: failure()
    uses: slack.post
    with:
      channel: "#growth-alerts"
      message: "Catalog feed pipeline failed. Check run {{ run.id }}."
schedule:
  - cron: "15 * * * *" # run at minute 15 past every hour

The storage.upload step sets cache_control and derives an ETag from file content so downstream fetchers can avoid full downloads on small edits.

Step 3Implement transforms for Google

Transform the normalized model into Google columns and write a CSV file. Keep formatting strict to avoid ingestion errors.

// scripts/transform-google.ts
import { ProductVariant } from '../src/models/product';
import { createWriteStream } from 'fs';
import { stringify } from 'csv-stringify/sync';

function toMoney(n: number, currency: string): string {
  return `${n.toFixed(2)} ${currency}`;
}

export default async function main(input: ProductVariant[]) {
  const rows = input.map(v => ({
    id: v.sku,
    title: v.variant_title || v.title,
    description: v.description,
    link: v.product_url,
    image_link: v.image_url,
    additional_image_link: (v.additional_images || []).join(','),
    price: toMoney(v.price, v.currency),
    sale_price: v.sale_price ? toMoney(v.sale_price, v.currency) : '',
    sale_price_effective_date: v.sale_start && v.sale_end ? `${v.sale_start}/${v.sale_end}` : '',
    availability: v.availability,
    brand: v.brand,
    gtin: v.gtin || '',
    mpn: v.mpn || '',
    condition: v.condition,
    google_product_category: v.google_product_category || '',
    item_group_id: v.parent_id,
  }));

  const csv = stringify(rows, { header: true });
  createWriteStream('out/google_products.csv', { flags: 'w' }).write(csv);
}

Step 4Implement transforms for Meta

Meta uses nearly the same columns but names the unique ID retailer_id and the product URL url. Keep the money format the same for consistency.

// scripts/transform-meta.ts
import { ProductVariant } from '../src/models/product';
import { createWriteStream } from 'fs';
import { stringify } from 'csv-stringify/sync';

function toMoney(n: number, currency: string): string {
  return `${n.toFixed(2)} ${currency}`;
}

export default async function main(input: ProductVariant[]) {
  const rows = input.map(v => ({
    retailer_id: v.sku,
    name: v.variant_title || v.title,
    description: v.description,
    url: v.product_url,
    image_link: v.image_url,
    additional_image_link: (v.additional_images || []).join(','),
    price: toMoney(v.price, v.currency),
    sale_price: v.sale_price ? toMoney(v.sale_price, v.currency) : '',
    availability: v.availability,
    brand: v.brand,
    gtin: v.gtin || '',
    mpn: v.mpn || '',
    condition: v.condition,
    google_product_category: v.google_product_category || '',
    item_group_id: v.parent_id,
  }));

  const csv = stringify(rows, { header: true });
  createWriteStream('out/meta_products.csv', { flags: 'w' }).write(csv);
}

Step 5Prepare the storage bucket and URLs

Create a bucket such as marketing-feeds with a feeds/google/products.csv and feeds/meta/products.csv path. Use a public read policy for those objects or route through a CDN. The path must remain stable because platforms pin to the URL, not to a changing file name.

Add caching headers and ETags so Google and Meta can skip full downloads when content has not changed. See the MDN reference on the ETag header for details on conditional requests.

Step 6Add validation before publish

Prevent empty or malformed feeds from shipping. Basic checks catch the majority of ingestion failures.

// scripts/validate-feeds.ts
import { parse } from 'csv-parse/sync';
import { readFileSync } from 'fs';

function assert(condition: boolean, msg: string) {
  if (!condition) throw new Error(msg);
}

function nonEmptyString(s: string) { return s && s.trim().length > 0; }

function validate(path: string) {
  const data = readFileSync(path, 'utf8');
  const rows = parse(data, { columns: true });
  assert(rows.length > 0, `${path} has no rows`);
  const sample = rows[0];
  const required = Object.keys(sample).filter(k => ['id','retailer_id'].includes(k) || ['title','name','description','price','availability','link','url','image_link'].includes(k));
  required.forEach(k => {
    const missing = rows.filter((r: any) => !nonEmptyString(r[k] || '')).length;
    assert(missing === 0, `${path} missing values in ${k}`);
  });
}

validate('out/google_products.csv');
validate('out/meta_products.csv');

Step 7Schedule full and incremental runs

Run a small hourly incremental job and a daily full rebuild. The incremental run filters by updated_at so only changed rows are processed. The daily job regenerates the entire file for consistency.

# playbooks/catalog-feed-schedules.yaml
version: 1
name: catalog-feed-schedules
triggers:
  hourly_incremental:
    cron: "15 * * * *"
    vars:
      INCREMENTAL_SINCE: "{{ now.subtract(65,'minutes').toISOString() }}"
    run: { use: catalog-feed }
  daily_full:
    cron: "30 3 * * *"
    vars:
      INCREMENTAL_SINCE: "1970-01-01T00:00:00Z"
    run: { use: catalog-feed }

Step 8Wire platforms to the feed URL

In Google Merchant Center, add a scheduled fetch for the public products.csv URL and set fetch times slightly after your pipeline runs. In Meta Commerce Manager, create a data feed and point it to the Meta file URL. Both platforms will reprocess on the cadence you choose and surface diagnostics if rows fail validation.

For exact field definitions and required values, reference the official Google Merchant Center product data specification and the Meta Catalog data feed guide.

Step 9Handle images, availability, and pricing

Images must be reachable and large enough to pass quality checks. Prefer 1024 px on the shortest side and use HTTPS. For availability, update quickly when popular items go out of stock so ads stop promoting them. For pricing, keep currency codes consistent with your target country.

Consider a small guardrail that pauses ad sets when the number of in stock top sellers drops below a threshold. You can reuse the notification job to post an alert in Slack when this happens.

Step 10Add analytics friendly URLs and tracking

Add campaign parameters to the product detail page URL in the feed so you can attribute clicks. Maintain a function that appends UTM tags in a consistent order to avoid duplicate landing pages in analytics dashboards. This is a good place to insert a long tail target such as how to build a product feed for Google Merchant Center without breaking attribution.

function addTracking(url: string, source: 'google' | 'meta') {
  const u = new URL(url);
  u.searchParams.set('utm_source', source);
  u.searchParams.set('utm_medium', 'catalog');
  u.searchParams.set('utm_campaign', 'pmax' );
  return u.toString();
}

Use addTracking inside your transform scripts so every URL includes consistent parameters. This keeps reporting clean when you later join click data to orders in your warehouse.

Step 11Introduce a delta mode for scale

Large catalogs benefit from deltas. Track the maximum updated_at value from the last successful run, read only changed rows, and write a compact delta file alongside the full file. Some teams configure Meta to ingest deltas more frequently while Google ingests the daily full file. This pattern is a common query shaped phrase to target such as server side product catalog pipeline with OpenClaw for teams that are scaling.

Step 12Monitor and alert

Add three simple monitors that cover most real world failures.

• Row count: compare the latest run to a rolling 7 day average. Alert on large drops or spikes.
• File size: catch near empty files that might slip past row count checks for small catalogs.
• HTTP reachability: curl the public feed URL from an external runner and alert on non 2xx responses.

curl -sSIL -o /dev/null -w "%{http_code}\n" https://cdn.example.com/feeds/google/products.csv

Step 13Test with a sandbox first

Before you switch production ad sets to use the new catalog, test with a small subset of SKUs. Create a label or tag such as catalog_test = true and filter your SQL view to only include those items. Once the platform diagnostics show zero errors, expand the scope to the full catalog.

Step 14Rollback plan

Keep the last known good feed file in your bucket under a static backup path. If a bad deploy ships empty or invalid rows, swap the pointer in your CDN to the backup file while you fix the transform. This avoids a prolonged outage in product listings.

Common issues and fixes

Here are frequent failure modes and quick remediations.

• Disallowed HTML in descriptions. Strip tags during transform and keep descriptions under the platform limits.
• Image 404s. Validate before publish by fetching a random sample of image URLs and failing the run if the error rate exceeds a threshold.
• Incorrect currency formats. Always render amounts as 12.34 USD rather than $12.34.
• Duplicate IDs across parents and variants. Ensure each row has a unique SKU level identifier and use item_group_id to group variants.
• Mismatched availability strings. Normalize to lower case values the platforms expect.

Extend the pipeline

Once your base feeds are stable, you can extend the pipeline in three practical ways.

• Add an enrichment step that merges review counts or badges into titles for high intent campaigns. Keep copy brief and informative.
• Generate a second feed for clearance items with separate sale pricing and a distinct campaign structure.
• Export a JSON version of the feed to power a dynamic site search or landing page modules, which helps organic discovery. For a broader perspective, browse more from the ButterGrow blog.

The same playbook patterns apply to other catalog types such as service listings or course libraries. If you plan to expand beyond ads, the product page for AI marketing automation features can help you decide which modules to wire in next.

Your options include agent driven optimization, automated QA, and safe rollout controls. If you want a hosted path that removes undifferentiated plumbing, the ButterGrow platform runs OpenClaw under the hood and provides simple tenants for data connectors, schedules, and audit logs.

The final long tail query to consider is schedule daily catalog updates without manual exports, which this pipeline solves with a clear schedule and no human steps.

To learn what to build after catalogs, consider exploring dynamic creative optimization or lifecycle triggered offers that use the same catalog foundation.

Your next step is to add light QA and experiment with title templates by category so search terms match. Keep changes small and measurable.

ButterGrow runs OpenClaw for you with sane defaults. If you want this pipeline deployed without managing infrastructure, follow the onboarding flow and import the sample playbook to get started in minutes. If questions come up, browse answers to common questions or open a support ticket from your workspace.

References

• Google Merchant Center product data specification - Official field definitions and policy notes for Google feeds.
• Meta Catalog data feed guide - Official guidance for Meta feed files and scheduling.
• MDN ETag header - How conditional requests and caching validators work.