Seed Data & Catalog Management¶

This guide covers managing seed data for development and staging, including strategies for incremental catalog updates. (There is no production environment yet, status 2026-06-10.)

Overview¶

The seed data system allows you to version-control your complete webshop catalog. It is used for local development setup (./manage.sh setup) and for CI testing of the export/import pipeline.

Policy: never sync local dev seed data to staging or production

Local dev contains test/junk data that must not pollute staging. Staging has its own curated catalog (managed via the staging Django admin) plus QA test data seeded with python manage.py seed_staging_data. Never load a local export_seed_data dump into staging.

What Gets Exported/Imported¶

Database Models (23 Total)¶

App	Model	Description
core	`SiteSettings`	Store settings, VAT, shipping, footer info
core	`Page`	Static pages (Terms, Privacy, FAQ, About, Returns)
products	`Category`	Product categories (hierarchical with MPTT)
products	`Brand`	Product brands with logos
products	`Product`	Products with specifications
products	`Color`	Available product colors
products	`ColorCategory`	Color groupings for the color picker
products	`Size`	Available sizes (clothing, caps, accessories)
products	`SizeType`	Size type groupings (clothing, caps, ...)
products	`ProductColorImage`	Multi-view images per color
products	`TwoColorCombination`	Two-color product combinations
products	`ProductVariant`	Size/color combinations with stock
pricing	`DiscountCode`	Promotional discount codes
pricing	`VolumeDiscount`	Bulk purchase discounts
pricing	`VolumeDiscountTier`	Volume discount tiers
pricing	`TeamDiscount`	Team order discounts
pricing	`SpecialOffer`	Time-limited promotions
pricing	`PricingRule`	Dynamic pricing rules
designs	`Font`	Designer fonts (uploaded + Google)
designs	`TextColor`	Designer text colors with pricing
designs	`DesignTemplate`	Pre-made design templates
clipart	`ClipartCategory`	Clipart categories
clipart	`ClipartItem`	Individual clipart SVGs

Media Files¶

Directory	Contents
`products/`	Product main images
`categories/`	Category thumbnails
`brands/`	Brand logos
`variants/`	Variant-specific images
`fonts/`	Uploaded font files (.ttf, .otf, .woff, .woff2)
`clipart/`	SVG files and thumbnails
`templates/`	Design template thumbnails

What's NOT Included (Runtime Data)¶

Orders, OrderItems, CartItems
CustomDesigns (user-created)
User accounts
Payment transactions

Git-Tracked Seed Data (Recommended)¶

Seed data is stored in backend/fixtures/seed/ and version-controlled:

backend/fixtures/seed/
├── seed_data.json      # All catalog data
├── metadata.json       # Export metadata
└── media/              # Product images, fonts, clipart
    ├── products/
    ├── categories/
    ├── fonts/
    └── clipart/

Commands¶

Export Seed Data¶

# Using helper script (recommended)
./scripts/export-seed-data.sh

# Direct Django command
cd backend
python manage.py export_seed_data

# Export without images (JSON only)
python manage.py export_seed_data --skip-images

# Export to custom location
python manage.py export_seed_data --output /tmp/my_export

Import Seed Data¶

# Standard import (merge mode - updates existing, creates new)
python manage.py import_seed_data

# Import new records only, skip existing
python manage.py import_seed_data --mode skip

# Import with detailed logging
python manage.py import_seed_data --mode overwrite

# Clear all catalog data before import (dangerous!)
python manage.py import_seed_data --clear --force

# Import from custom location
python manage.py import_seed_data --input /path/to/seed_data

Import Modes¶

Mode	Behavior
`merge` (default)	Update existing records, create new ones
`skip`	Only create new records, skip existing
`overwrite`	Same as merge with detailed logging

Selective Export (Specific Models)¶

For updating specific parts of the catalog without a full export:

# Export only products
./manage.sh update-catalog products

# Export specific product (by SKU)
./manage.sh update-catalog products 'sku=HOODIE-BLUE-001'

# Export only pricing rules
./manage.sh update-catalog 'discount_codes,volume_discounts'

# Export a category and its products
./manage.sh update-catalog 'categories,products' 'category__slug=hoodies'

Available models for selective export: site_settings, color_categories, size_types, sizes, colors, categories, brands, products, product_color_images, two_color_combinations, product_variants, discount_codes, volume_discounts, volume_discount_tiers, team_discounts, special_offers, pricing_rules

Catalog Update Strategies¶

After the initial catalog import, use the appropriate strategy for ongoing updates.

Strategy 1: Merge Import (Multiple Updates)¶

Best for updating multiple products, refreshing pricing, or adding new products alongside existing ones.

The import command uses update_or_create() by default:

Existing records (matched by primary key) are updated
New records are created
Existing records not in the import are preserved

# Make changes in the local Django admin, then export the catalog
./scripts/export-seed-data.sh

# On another dev machine: import (merges with existing)
python manage.py import_seed_data

Warning

Merge imports are a development-to-development mechanism. Do not import local exports into staging — see the policy note in the Overview.

Strategy 2: Selective Export (Specific Updates)¶

Best for updating a single product, specific pricing rules, or a specific category.

# Update single product description
./manage.sh update-catalog products 'sku=TSHIRT-BASIC-001'

# Update all discount codes
./manage.sh update-catalog 'discount_codes,volume_discounts,team_discounts'

# Add new color to existing product
./manage.sh update-catalog 'colors,products,product_color_images,product_variants' 'product__sku=HOODIE-001'

Tip

Use --include-related to automatically include related records, or explicitly list related models.

Strategy 3: Direct Django Admin (Single Changes)¶

Best for fixing a typo, adjusting one price, or toggling a flag. Make changes directly in the target environment's Django admin -- no export/import needed.

Comparison¶

Strategy	Use Case	Speed	Safety	Complexity
Merge Import	Multiple updates, new products	Slow	High	Low
Selective Export	Specific products/models	Fast	High	Medium
Direct Admin	Single field changes	Instant	Medium	Low

Workflow by Environment¶

Development¶

# Make catalog changes in Django admin
# Then export to version control
./scripts/export-seed-data.sh
git add backend/fixtures/seed/
git commit -m "feat(catalog): add new product line"

Staging¶

Seed data is not imported automatically on deploy. Staging deploys run as the deploy job inside ci.yml on every push to the staging branch (deploy-staging.yml is invoked via workflow_call); the deploy does not touch catalog data.

Staging has its own curated catalog, managed via the staging Django admin. QA test data (users with @qa-test.nl emails, products with qa- slug prefix) is seeded with:

python manage.py seed_staging_data           # idempotent QA test data
python manage.py seed_staging_data --clear   # delete QA test data first

Production¶

There is no production environment yet (status 2026-06-10). When it exists, the intended approach is:

# Initial setup: Import a curated baseline catalog (NOT a local dev export)
python manage.py import_seed_data --mode skip --force

# Ongoing: Use Django admin for catalog changes

CI/CD Integration¶

Automatic Testing (ci.yml)¶

The CI pipeline tests seed data export/import on scheduled (nightly) runs, or when manually dispatched with run_extended_checks: true — not on every push:

Creates a minimal test fixture (python manage.py create_test_fixture: a handful of products, categories, colors, sizes, and variants)
Exports to /tmp with --skip-images
Flushes database
Imports and validates data integrity
Tests git-tracked seed data import (if exists)

Staging Deployment (deploy-staging.yml)¶

The staging deploy (the deploy job in ci.yml, which calls deploy-staging.yml via workflow_call) does not copy or import seed data. Catalog data on staging is managed independently (see "Workflow by Environment" above).

Adding New Models to CI Testing¶

Update create_test_fixture.py -- add 1 instance of the new model
Update CI verification assertions in .github/workflows/ci.yml
Test locally before committing

Best Practices¶

Always export after local admin changes -- Keep git-tracked seed data current
Review before committing -- git diff fixtures/seed/seed_data.json
Never import dev exports into staging -- Staging has its own curated data (see policy in Overview)
Staging via admin -- Use the staging Django admin for staging catalog changes
Version large updates -- Create git tags for major catalog changes
Backup before importing -- Export existing data before major catalog changes
Use selective exports for efficiency -- For large catalogs (1000+ products), don't export/import everything for small changes

Troubleshooting¶

Import Fails with Foreign Key Error¶

Ensure full export is being imported. Dependencies are imported in order. If using selective export, include all related models:

# Wrong: Export only products
./manage.sh update-catalog products

# Right: Export products AND their dependencies
./manage.sh update-catalog 'categories,brands,products'

Images Not Showing¶

Check MEDIA_ROOT and MEDIA_URL settings. Ensure web server can serve media files. If using DigitalOcean Spaces, upload paths should be relative (e.g., products/image.png not media/products/image.png).

Import Overwrites Custom Production Changes¶

Production-specific changes (like stock levels) get overwritten. Solution: Use Django admin for those fields, or manage them via selective export excluding those fields.

Large Media Files¶

Product images and media files are stored on DigitalOcean Spaces (S3-compatible object storage). Seed data media files in backend/fixtures/seed/media/ are version-controlled in the repository. During import, files are uploaded to the configured storage backend (local filesystem in development, DigitalOcean Spaces in staging/production).

CI Test Failures¶

Error	Cause	Solution
"Expected X records, got Y"	Fixture/import logic changed	Check `create_test_fixture.py` and CI assertions
"JSON serialization error"	New field type not handled	Update `DecimalEncoder` in `export_seed_data.py`
"Foreign key constraint"	Import order issue	Check `import_order` list in `import_database_data()`