Seed Data & Catalog Management¶
This guide covers managing seed data for development and staging, including strategies for incremental catalog updates. Production is live since 2026-09-04; its catalog comes from a one-time staging export (see Production).
Overview¶
The seed data system allows you to version-control your complete webshop catalog. It is used for local development setup (./manage.sh setup) and for CI testing of the export/import pipeline.
Policy: never sync local dev seed data to staging or production
Local dev contains test/junk data that must not pollute staging. Staging
has its own curated catalog (managed via the staging Django admin) plus QA
test data seeded with python manage.py seed_staging_data. Never load a
local export_seed_data dump into staging.
What Gets Exported/Imported¶
Database Models (25 Total)¶
The whitelist (ALLOWED_MODELS) and the export/import order (EXPORT_ORDER) live in backend/apps/products/management/commands/_base.py. The key in the first column of the JSON file is the name used by --skip and by update_seed_data --models. TeamDiscount (team_discounts) left the whitelist with the team-order rework in v1.53 (migration pricing/0003_delete_teamdiscount); re-export rather than import a dump from before that.
| App | Model (JSON key) | Description |
|---|---|---|
| core | SiteSettings (site_settings) |
Store settings, VAT, shipping, footer info |
| core | Page (pages) |
Static pages (Terms, Privacy, FAQ, About, Returns) |
| products | ColorCategory (color_categories) |
Color groupings for the color picker |
| products | SizeType (size_types) |
Size type groupings (clothing, caps, ...) |
| products | Size (sizes) |
Available sizes (clothing, caps, accessories) |
| products | Color (colors) |
Available product colors |
| products | Category (categories) |
Product categories (hierarchical with MPTT) |
| products | Brand (brands) |
Product brands with logos |
| products | Product (products) |
Products with specifications |
| products | ProductColorImage (product_color_images) |
Multi-view images per color |
| products | TwoColorCombination (two_color_combinations) |
Two-color product combinations |
| products | ProductVariant (product_variants) |
Size/color combinations with stock |
| pricing | DiscountCode (discount_codes) |
Promotional discount codes |
| pricing | VolumeDiscount (volume_discounts) |
Bulk purchase discounts |
| pricing | VolumeDiscountTier (volume_discount_tiers) |
Volume discount tiers |
| pricing | SpecialOffer (special_offers) |
Time-limited promotions |
| pricing | PricingRule (pricing_rules) |
Dynamic pricing rules |
| pricing | DecorationMethod (decoration_methods) |
Decoration (print) methods, the engine behind every print price |
| pricing | DecorationAreaTier (decoration_area_tiers) |
Per-zone price tiers of a decoration method |
| products | ProductPrintZone (product_print_zones) |
Print zones per product and view; without them the Designer has nothing to design on |
| designs | Font (fonts) |
Designer fonts (uploaded + Google) |
| designs | TextColor (text_colors) |
Designer text colors with pricing |
| designs | DesignTemplate (design_templates) |
Pre-made design templates |
| clipart | ClipartCategory (clipart_categories) |
Clipart categories |
| clipart | ClipartItem (clipart_items) |
Individual clipart SVGs |
Media Files¶
| Directory | Contents |
|---|---|
products/ |
Product main images |
categories/ |
Category thumbnails |
brands/ |
Brand logos |
variants/ |
Variant-specific images |
fonts/ |
Uploaded font files (.ttf, .otf, .woff, .woff2) |
clipart/ |
SVG files and thumbnails |
templates/ |
Design template thumbnails |
What's NOT Included (Runtime Data)¶
- Orders, OrderItems, CartItems
- CustomDesigns (user-created)
- User accounts
- Payment transactions
Git-Tracked Seed Data (Recommended)¶
Seed data is stored in backend/fixtures/seed/ and version-controlled:
backend/fixtures/seed/
├── seed_data.json # All catalog data
├── metadata.json # Export metadata
└── media/ # Only the subdirectories in ALLOWED_IMAGE_SUBDIRS are imported:
├── products/ # products, categories, brands, variants,
├── categories/ # fonts, clipart, templates
├── fonts/
└── clipart/
Commands¶
Export Seed Data¶
# Using helper script (recommended)
./scripts/export-seed-data.sh
# Direct Django command
cd backend
python manage.py export_seed_data
# Export without images (JSON only)
python manage.py export_seed_data --skip-images
# Export to custom location
python manage.py export_seed_data --output /tmp/my_export
# Leave whole models out (repeatable): absent from seed_data.json, so an import
# never touches them on the target — not even with --clear
python manage.py export_seed_data --skip pages --skip site_settings
# Leave out test products, including their variants, colour images, two-colour
# combinations and print zones (repeatable; an unknown SKU fails)
python manage.py export_seed_data --exclude-sku QA-HOODIE-001
# Drop per-zone price tiers that no exported print zone uses any more
# (the method SiteSettings points to is always kept)
python manage.py export_seed_data --drop-orphan-decoration-methods
Import Seed Data¶
# Standard import (merge mode - updates existing, creates new)
python manage.py import_seed_data
# Import new records only, skip existing
python manage.py import_seed_data --mode skip
# Import with detailed logging
python manage.py import_seed_data --mode overwrite
# Clear the catalog models that are in the export, then import (dangerous!)
python manage.py import_seed_data --clear --force
# Database only, no media files
python manage.py import_seed_data --skip-images
# Import from custom location (default: ./fixtures/seed)
python manage.py import_seed_data --input /path/to/seed_data
--force skips the confirmation prompts (CI and ./manage.sh setup use it). The file is validated against the model whitelist before anything is written, and the whole database import runs in one transaction.
What --clear deletes
--clear deletes only the models whose key is in seed_data.json. A model left out
with export_seed_data --skip (for example pages or site_settings) stays untouched
on the target. The file is read and validated before anything is deleted, so a broken
file deletes nothing. Before #334 (2026-09-26) --clear always wiped Page, and
validated only after clearing.
--clear still deletes all decoration methods and categories when those are in the
export. SiteSettings.default_decoration_method and homepage_collection_category
are SET_NULL, so they end up empty when the settings are not imported. The import
then warns if there is no active default decoration method; set it in
/admin/settings.
Once real orders exist, --clear stops with a ProtectedError (OrderItem protects
product, variant and design): after go-live, use merge.
Import Modes¶
| Mode | Behavior |
|---|---|
merge (default) |
Update existing records, create new ones |
skip |
Only create new records, skip existing |
overwrite |
Same as merge with detailed logging |
Selective Export (Specific Models)¶
For updating specific parts of the catalog without a full export:
# Export only products
./manage.sh update-catalog products
# Export specific product (by SKU)
./manage.sh update-catalog products 'sku=HOODIE-BLUE-001'
# Export only pricing rules
./manage.sh update-catalog 'discount_codes,volume_discounts'
# Export a category and its products
./manage.sh update-catalog 'categories,products' 'category__slug=hoodies'
./manage.sh update-catalog wraps python manage.py update_seed_data --models <models> [--filter <expr>] --output <dir> --include-related (default output ./seed_data_update/). Filters are restricted to the fields in ALLOWED_FILTER_FIELDS (_base.py), e.g. sku, slug, category__slug, is_active.
Available models for selective export: site_settings, pages, color_categories, size_types, sizes, colors, categories, brands, products, product_color_images, two_color_combinations, product_variants, discount_codes, volume_discounts, volume_discount_tiers, special_offers, pricing_rules, decoration_methods, decoration_area_tiers, product_print_zones, fonts, text_colors, design_templates, clipart_categories, clipart_items
Catalog Update Strategies¶
After the initial catalog import, use the appropriate strategy for ongoing updates.
Strategy 1: Merge Import (Multiple Updates)¶
Best for updating multiple products, refreshing pricing, or adding new products alongside existing ones.
The import command uses update_or_create() by default:
- Existing records (matched by primary key) are updated
- New records are created
- Existing records not in the import are preserved
# Make changes in the local Django admin, then export the catalog
./scripts/export-seed-data.sh
# On another dev machine: import (merges with existing)
python manage.py import_seed_data
Warning
Merge imports are a development-to-development mechanism. Do not import local exports into staging — see the policy note in the Overview.
Strategy 2: Selective Export (Specific Updates)¶
Best for updating a single product, specific pricing rules, or a specific category.
# Update single product description
./manage.sh update-catalog products 'sku=TSHIRT-BASIC-001'
# Update all discount codes
./manage.sh update-catalog 'discount_codes,volume_discounts'
# Add new color to existing product
./manage.sh update-catalog 'colors,products,product_color_images,product_variants' 'product__sku=HOODIE-001'
Tip
./manage.sh update-catalog always passes --include-related (related records such as variants come along). When calling update_seed_data directly, add it yourself or list the related models explicitly.
Strategy 3: Direct Django Admin (Single Changes)¶
Best for fixing a typo, adjusting one price, or toggling a flag. Make changes directly in the target environment's Django admin -- no export/import needed.
Comparison¶
| Strategy | Use Case | Speed | Safety | Complexity |
|---|---|---|---|---|
| Merge Import | Multiple updates, new products | Slow | High | Low |
| Selective Export | Specific products/models | Fast | High | Medium |
| Direct Admin | Single field changes | Instant | Medium | Low |
Workflow by Environment¶
Development¶
# Make catalog changes in Django admin
# Then export to version control
./scripts/export-seed-data.sh
git add backend/fixtures/seed/
git commit -m "feat(catalog): add new product line"
Staging¶
Seed data is not imported automatically on deploy. Staging deploys run as
the deploy job inside ci.yml on every push to the staging branch
(deploy-staging.yml is invoked via workflow_call); the deploy does not
touch catalog data.
Staging has its own curated catalog, managed via the staging Django admin.
Since production became the source of the catalog, staging falls behind; bring it
level now and then with scripts/catalogus-prod-naar-staging.sh (production
export, catalog media via rclone copy, import on staging, then seed_staging_data).
The usual run replaces the staging catalog (--wis-testgegevens, after a --proef-import),
because a merge on id trips over records that have different ids on both servers. See the runbook
Catalogus van productie naar staging.
QA test data (users with @qa-test.nl emails, products with qa- slug
prefix) is seeded with:
python manage.py seed_staging_data # idempotent QA test data
python manage.py seed_staging_data --clear # delete QA test data first
Production¶
Production is live (first deploy 2026-09-04). Since 2026-09-04 production is the source for the legal pages and site settings: they were corrected there, and the migration leaves them out. The catalog moves once from staging to production:
# On staging: fresh export, without images, pages, settings and test products
python manage.py export_seed_data --skip-images \
--skip pages --skip site_settings \
--exclude-sku <test-SKU> --drop-orphan-decoration-methods
# Media: sync the staging bucket to freeze-production-media (rclone / s3cmd),
# instead of exporting images
# On production, after a manual database backup: import
python manage.py import_seed_data --clear --force
Afterwards, compare the site settings with staging (surcharges, team orders, default
decoration method) in /admin/settings, price a few designs in Proefontwerp on both
environments, and from the export onwards edit the catalog only on production (the
Django/Next.js admin); changes made on staging after the export do not carry over.
Category texts for findability (description, SEO title and description, intro, FAQ; admin
section "Vindbaarheid in Google", v1.52) are edited in the admin. The one-off command
python manage.py vul_categorieteksten [--dry-run] (v1.57, feedback #65) fills only the
fields that are still empty from categorieteksten.json, per category slug; what Eefje
already wrote stays.
CI/CD Integration¶
Automatic Testing (ci.yml)¶
Two steps at the end of the backend-test job test the seed data export/import
on the weekly scheduled run (Monday 03:00 UTC) or when ci.yml is dispatched
manually with run_extended_checks: true — not on every push:
- Creates a minimal test fixture (
python manage.py create_test_fixture, now inapps/core/management/commands/: a handful of products, categories, colors, sizes, variants and discounts) - Exports to
/tmp/ci_test_exportwith--skip-imagesand checks thatseed_data.jsonandmetadata.jsonexist - Flushes the database
- Imports with
--forceand asserts the counts (3 products, 5 colors, 6 sizes) - Imports the git-tracked
fixtures/seed/seed_data.jsonwith--mode skip --force(if it exists)
E2E Fixtures (ci.yml, chromium-admin)¶
The seed data is not used by the Playwright jobs. e2e-setup.yml with seed-admin: true
runs python manage.py ensure_e2e_admin (the TOTP admin for admin.setup.ts) and
python manage.py seed_e2e_extras (apps/core/management/commands/): a product with a
real colour image and several named print zones, a default decoration method with area
tiers, a product with an active personalisation template, and a clipart category with a
single-colour figure and a figure with editable text (feedback #60, #61). It is idempotent
and prints a JSON summary the specs read. Add new E2E fixtures there, not to the seed export.
Staging Deployment (deploy-staging.yml)¶
The staging deploy (the deploy job in ci.yml, which calls
deploy-staging.yml via workflow_call) does not copy or import seed
data. Catalog data on staging is managed independently (see "Workflow by
Environment" above).
Adding New Models to CI Testing¶
- Update
create_test_fixture.py-- add 1 instance of the new model - Update CI verification assertions in
.github/workflows/ci.yml - Test locally before committing
Best Practices¶
- Always export after local admin changes -- Keep git-tracked seed data current
- Review before committing --
git diff fixtures/seed/seed_data.json - Never import dev exports into staging -- Staging has its own curated data (see policy in Overview)
- Staging via admin -- Use the staging Django admin for staging catalog changes
- Version large updates -- Create git tags for major catalog changes
- Backup before importing -- Export existing data before major catalog changes
- Use selective exports for efficiency -- For large catalogs (1000+ products), don't export/import everything for small changes
Troubleshooting¶
Import Fails with Foreign Key Error¶
Ensure full export is being imported. Dependencies are imported in order. If using selective export, include all related models:
# Wrong: Export only products
./manage.sh update-catalog products
# Right: Export products AND their dependencies
./manage.sh update-catalog 'categories,brands,products'
Images Not Showing¶
Check MEDIA_ROOT and MEDIA_URL settings. Ensure web server can serve media files. If using DigitalOcean Spaces, upload paths should be relative (e.g., products/image.png not media/products/image.png).
Import Overwrites Custom Production Changes¶
Production-specific changes (like stock levels) get overwritten. Solution: Use Django admin for those fields, or manage them via selective export excluding those fields.
Large Media Files¶
Product images and media files are stored on DigitalOcean Spaces (S3-compatible object storage). Seed data media files in backend/fixtures/seed/media/ are version-controlled in the repository. During import, files are uploaded to the configured storage backend (local filesystem in development, DigitalOcean Spaces in staging/production).
CI Test Failures¶
| Error | Cause | Solution |
|---|---|---|
| "Expected X records, got Y" | Fixture/import logic changed | Check create_test_fixture.py and CI assertions |
| "JSON serialization error" | New field type not handled | Update DecimalEncoder in export_seed_data.py |
| "Foreign key constraint" | Import order issue | Check import_order list in import_database_data() |
Technical Details¶
Security Features¶
- Model whitelist prevents importing unauthorized models
- Path traversal protection on image imports
- File size limits (100MB max)
- Symlink following disabled
Export Format¶
- Database: Django JSON serialization with natural foreign keys
- Images: Direct file copy preserving directory structure
- Metadata: Export timestamp and record counts