Skip to content

Seed Data & Catalog Management

This guide covers managing seed data for development and staging, including strategies for incremental catalog updates. Production is live since 2026-09-04; its catalog comes from a one-time staging export (see Production).

Overview

The seed data system allows you to version-control your complete webshop catalog. It is used for local development setup (./manage.sh setup) and for CI testing of the export/import pipeline.

Policy: never sync local dev seed data to staging or production

Local dev contains test/junk data that must not pollute staging. Staging has its own curated catalog (managed via the staging Django admin) plus QA test data seeded with python manage.py seed_staging_data. Never load a local export_seed_data dump into staging.

What Gets Exported/Imported

Database Models (25 Total)

The whitelist (ALLOWED_MODELS) and the export/import order (EXPORT_ORDER) live in backend/apps/products/management/commands/_base.py. The key in the first column of the JSON file is the name used by --skip and by update_seed_data --models. TeamDiscount (team_discounts) left the whitelist with the team-order rework in v1.53 (migration pricing/0003_delete_teamdiscount); re-export rather than import a dump from before that.

App Model (JSON key) Description
core SiteSettings (site_settings) Store settings, VAT, shipping, footer info
core Page (pages) Static pages (Terms, Privacy, FAQ, About, Returns)
products ColorCategory (color_categories) Color groupings for the color picker
products SizeType (size_types) Size type groupings (clothing, caps, ...)
products Size (sizes) Available sizes (clothing, caps, accessories)
products Color (colors) Available product colors
products Category (categories) Product categories (hierarchical with MPTT)
products Brand (brands) Product brands with logos
products Product (products) Products with specifications
products ProductColorImage (product_color_images) Multi-view images per color
products TwoColorCombination (two_color_combinations) Two-color product combinations
products ProductVariant (product_variants) Size/color combinations with stock
pricing DiscountCode (discount_codes) Promotional discount codes
pricing VolumeDiscount (volume_discounts) Bulk purchase discounts
pricing VolumeDiscountTier (volume_discount_tiers) Volume discount tiers
pricing SpecialOffer (special_offers) Time-limited promotions
pricing PricingRule (pricing_rules) Dynamic pricing rules
pricing DecorationMethod (decoration_methods) Decoration (print) methods, the engine behind every print price
pricing DecorationAreaTier (decoration_area_tiers) Per-zone price tiers of a decoration method
products ProductPrintZone (product_print_zones) Print zones per product and view; without them the Designer has nothing to design on
designs Font (fonts) Designer fonts (uploaded + Google)
designs TextColor (text_colors) Designer text colors with pricing
designs DesignTemplate (design_templates) Pre-made design templates
clipart ClipartCategory (clipart_categories) Clipart categories
clipart ClipartItem (clipart_items) Individual clipart SVGs

Media Files

Directory Contents
products/ Product main images
categories/ Category thumbnails
brands/ Brand logos
variants/ Variant-specific images
fonts/ Uploaded font files (.ttf, .otf, .woff, .woff2)
clipart/ SVG files and thumbnails
templates/ Design template thumbnails

What's NOT Included (Runtime Data)

  • Orders, OrderItems, CartItems
  • CustomDesigns (user-created)
  • User accounts
  • Payment transactions

Seed data is stored in backend/fixtures/seed/ and version-controlled:

backend/fixtures/seed/
├── seed_data.json      # All catalog data
├── metadata.json       # Export metadata
└── media/              # Only the subdirectories in ALLOWED_IMAGE_SUBDIRS are imported:
    ├── products/       # products, categories, brands, variants,
    ├── categories/     # fonts, clipart, templates
    ├── fonts/
    └── clipart/

Commands

Export Seed Data

# Using helper script (recommended)
./scripts/export-seed-data.sh

# Direct Django command
cd backend
python manage.py export_seed_data

# Export without images (JSON only)
python manage.py export_seed_data --skip-images

# Export to custom location
python manage.py export_seed_data --output /tmp/my_export

# Leave whole models out (repeatable): absent from seed_data.json, so an import
# never touches them on the target — not even with --clear
python manage.py export_seed_data --skip pages --skip site_settings

# Leave out test products, including their variants, colour images, two-colour
# combinations and print zones (repeatable; an unknown SKU fails)
python manage.py export_seed_data --exclude-sku QA-HOODIE-001

# Drop per-zone price tiers that no exported print zone uses any more
# (the method SiteSettings points to is always kept)
python manage.py export_seed_data --drop-orphan-decoration-methods

Import Seed Data

# Standard import (merge mode - updates existing, creates new)
python manage.py import_seed_data

# Import new records only, skip existing
python manage.py import_seed_data --mode skip

# Import with detailed logging
python manage.py import_seed_data --mode overwrite

# Clear the catalog models that are in the export, then import (dangerous!)
python manage.py import_seed_data --clear --force

# Database only, no media files
python manage.py import_seed_data --skip-images

# Import from custom location (default: ./fixtures/seed)
python manage.py import_seed_data --input /path/to/seed_data

--force skips the confirmation prompts (CI and ./manage.sh setup use it). The file is validated against the model whitelist before anything is written, and the whole database import runs in one transaction.

What --clear deletes

--clear deletes only the models whose key is in seed_data.json. A model left out with export_seed_data --skip (for example pages or site_settings) stays untouched on the target. The file is read and validated before anything is deleted, so a broken file deletes nothing. Before #334 (2026-09-26) --clear always wiped Page, and validated only after clearing.

--clear still deletes all decoration methods and categories when those are in the export. SiteSettings.default_decoration_method and homepage_collection_category are SET_NULL, so they end up empty when the settings are not imported. The import then warns if there is no active default decoration method; set it in /admin/settings.

Once real orders exist, --clear stops with a ProtectedError (OrderItem protects product, variant and design): after go-live, use merge.

Import Modes

Mode Behavior
merge (default) Update existing records, create new ones
skip Only create new records, skip existing
overwrite Same as merge with detailed logging

Selective Export (Specific Models)

For updating specific parts of the catalog without a full export:

# Export only products
./manage.sh update-catalog products

# Export specific product (by SKU)
./manage.sh update-catalog products 'sku=HOODIE-BLUE-001'

# Export only pricing rules
./manage.sh update-catalog 'discount_codes,volume_discounts'

# Export a category and its products
./manage.sh update-catalog 'categories,products' 'category__slug=hoodies'

./manage.sh update-catalog wraps python manage.py update_seed_data --models <models> [--filter <expr>] --output <dir> --include-related (default output ./seed_data_update/). Filters are restricted to the fields in ALLOWED_FILTER_FIELDS (_base.py), e.g. sku, slug, category__slug, is_active.

Available models for selective export: site_settings, pages, color_categories, size_types, sizes, colors, categories, brands, products, product_color_images, two_color_combinations, product_variants, discount_codes, volume_discounts, volume_discount_tiers, special_offers, pricing_rules, decoration_methods, decoration_area_tiers, product_print_zones, fonts, text_colors, design_templates, clipart_categories, clipart_items

Catalog Update Strategies

After the initial catalog import, use the appropriate strategy for ongoing updates.

Strategy 1: Merge Import (Multiple Updates)

Best for updating multiple products, refreshing pricing, or adding new products alongside existing ones.

The import command uses update_or_create() by default:

  • Existing records (matched by primary key) are updated
  • New records are created
  • Existing records not in the import are preserved
# Make changes in the local Django admin, then export the catalog
./scripts/export-seed-data.sh

# On another dev machine: import (merges with existing)
python manage.py import_seed_data

Warning

Merge imports are a development-to-development mechanism. Do not import local exports into staging — see the policy note in the Overview.

Strategy 2: Selective Export (Specific Updates)

Best for updating a single product, specific pricing rules, or a specific category.

# Update single product description
./manage.sh update-catalog products 'sku=TSHIRT-BASIC-001'

# Update all discount codes
./manage.sh update-catalog 'discount_codes,volume_discounts'

# Add new color to existing product
./manage.sh update-catalog 'colors,products,product_color_images,product_variants' 'product__sku=HOODIE-001'

Tip

./manage.sh update-catalog always passes --include-related (related records such as variants come along). When calling update_seed_data directly, add it yourself or list the related models explicitly.

Strategy 3: Direct Django Admin (Single Changes)

Best for fixing a typo, adjusting one price, or toggling a flag. Make changes directly in the target environment's Django admin -- no export/import needed.

Comparison

Strategy Use Case Speed Safety Complexity
Merge Import Multiple updates, new products Slow High Low
Selective Export Specific products/models Fast High Medium
Direct Admin Single field changes Instant Medium Low

Workflow by Environment

Development

# Make catalog changes in Django admin
# Then export to version control
./scripts/export-seed-data.sh
git add backend/fixtures/seed/
git commit -m "feat(catalog): add new product line"

Staging

Seed data is not imported automatically on deploy. Staging deploys run as the deploy job inside ci.yml on every push to the staging branch (deploy-staging.yml is invoked via workflow_call); the deploy does not touch catalog data.

Staging has its own curated catalog, managed via the staging Django admin. Since production became the source of the catalog, staging falls behind; bring it level now and then with scripts/catalogus-prod-naar-staging.sh (production export, catalog media via rclone copy, import on staging, then seed_staging_data). The usual run replaces the staging catalog (--wis-testgegevens, after a --proef-import), because a merge on id trips over records that have different ids on both servers. See the runbook Catalogus van productie naar staging. QA test data (users with @qa-test.nl emails, products with qa- slug prefix) is seeded with:

python manage.py seed_staging_data           # idempotent QA test data
python manage.py seed_staging_data --clear   # delete QA test data first

Production

Production is live (first deploy 2026-09-04). Since 2026-09-04 production is the source for the legal pages and site settings: they were corrected there, and the migration leaves them out. The catalog moves once from staging to production:

# On staging: fresh export, without images, pages, settings and test products
python manage.py export_seed_data --skip-images \
  --skip pages --skip site_settings \
  --exclude-sku <test-SKU> --drop-orphan-decoration-methods

# Media: sync the staging bucket to freeze-production-media (rclone / s3cmd),
# instead of exporting images

# On production, after a manual database backup: import
python manage.py import_seed_data --clear --force

Afterwards, compare the site settings with staging (surcharges, team orders, default decoration method) in /admin/settings, price a few designs in Proefontwerp on both environments, and from the export onwards edit the catalog only on production (the Django/Next.js admin); changes made on staging after the export do not carry over.

Category texts for findability (description, SEO title and description, intro, FAQ; admin section "Vindbaarheid in Google", v1.52) are edited in the admin. The one-off command python manage.py vul_categorieteksten [--dry-run] (v1.57, feedback #65) fills only the fields that are still empty from categorieteksten.json, per category slug; what Eefje already wrote stays.

CI/CD Integration

Automatic Testing (ci.yml)

Two steps at the end of the backend-test job test the seed data export/import on the weekly scheduled run (Monday 03:00 UTC) or when ci.yml is dispatched manually with run_extended_checks: true — not on every push:

  1. Creates a minimal test fixture (python manage.py create_test_fixture, now in apps/core/management/commands/: a handful of products, categories, colors, sizes, variants and discounts)
  2. Exports to /tmp/ci_test_export with --skip-images and checks that seed_data.json and metadata.json exist
  3. Flushes the database
  4. Imports with --force and asserts the counts (3 products, 5 colors, 6 sizes)
  5. Imports the git-tracked fixtures/seed/seed_data.json with --mode skip --force (if it exists)

E2E Fixtures (ci.yml, chromium-admin)

The seed data is not used by the Playwright jobs. e2e-setup.yml with seed-admin: true runs python manage.py ensure_e2e_admin (the TOTP admin for admin.setup.ts) and python manage.py seed_e2e_extras (apps/core/management/commands/): a product with a real colour image and several named print zones, a default decoration method with area tiers, a product with an active personalisation template, and a clipart category with a single-colour figure and a figure with editable text (feedback #60, #61). It is idempotent and prints a JSON summary the specs read. Add new E2E fixtures there, not to the seed export.

Staging Deployment (deploy-staging.yml)

The staging deploy (the deploy job in ci.yml, which calls deploy-staging.yml via workflow_call) does not copy or import seed data. Catalog data on staging is managed independently (see "Workflow by Environment" above).

Adding New Models to CI Testing

  1. Update create_test_fixture.py -- add 1 instance of the new model
  2. Update CI verification assertions in .github/workflows/ci.yml
  3. Test locally before committing

Best Practices

  1. Always export after local admin changes -- Keep git-tracked seed data current
  2. Review before committing -- git diff fixtures/seed/seed_data.json
  3. Never import dev exports into staging -- Staging has its own curated data (see policy in Overview)
  4. Staging via admin -- Use the staging Django admin for staging catalog changes
  5. Version large updates -- Create git tags for major catalog changes
  6. Backup before importing -- Export existing data before major catalog changes
  7. Use selective exports for efficiency -- For large catalogs (1000+ products), don't export/import everything for small changes

Troubleshooting

Import Fails with Foreign Key Error

Ensure full export is being imported. Dependencies are imported in order. If using selective export, include all related models:

# Wrong: Export only products
./manage.sh update-catalog products

# Right: Export products AND their dependencies
./manage.sh update-catalog 'categories,brands,products'

Images Not Showing

Check MEDIA_ROOT and MEDIA_URL settings. Ensure web server can serve media files. If using DigitalOcean Spaces, upload paths should be relative (e.g., products/image.png not media/products/image.png).

Import Overwrites Custom Production Changes

Production-specific changes (like stock levels) get overwritten. Solution: Use Django admin for those fields, or manage them via selective export excluding those fields.

Large Media Files

Product images and media files are stored on DigitalOcean Spaces (S3-compatible object storage). Seed data media files in backend/fixtures/seed/media/ are version-controlled in the repository. During import, files are uploaded to the configured storage backend (local filesystem in development, DigitalOcean Spaces in staging/production).

CI Test Failures

Error Cause Solution
"Expected X records, got Y" Fixture/import logic changed Check create_test_fixture.py and CI assertions
"JSON serialization error" New field type not handled Update DecimalEncoder in export_seed_data.py
"Foreign key constraint" Import order issue Check import_order list in import_database_data()

Technical Details

Security Features

  • Model whitelist prevents importing unauthorized models
  • Path traversal protection on image imports
  • File size limits (100MB max)
  • Symlink following disabled

Export Format

  • Database: Django JSON serialization with natural foreign keys
  • Images: Direct file copy preserving directory structure
  • Metadata: Export timestamp and record counts