System Patterns — Paper Surplus Marketplace

Architecture Overview

The system follows a 4-layer architecture (from V1 System Diagram):

┌─────────────────────────────────────────┐
│           OUTPUT LAYER                   │
│  Newsletters, Container Proposals,       │
│  Reports, Notifications                  │
├─────────────────────────────────────────┤
│           ACTION LAYER                   │
│  Matching, Container Assembly,           │
│  Exclusivity, Newsletter Generation,     │
│  Excel Ingestion, Visibility Config      │
├─────────────────────────────────────────┤
│           ENTITY LAYER                   │
│  Products, Surplus, Mills,               │
│  Buyers, Buyer Specifics,                │
│  Mills Surplus Visibility                │
├─────────────────────────────────────────┤
│       DATA BOOTSTRAP LAYER               │
│  Initial Data Collection, Onboarding,    │
│  Excel Import, Email Ingestion           │
└─────────────────────────────────────────┘

6 Core Entities

1. Products

• Master catalog of paper product types
• Fields: type, GSM range, standard widths, primary use, characteristics
• Reference data — rarely changes

2. Surplus Availability

• Current mill surplus inventory
• Fields: mill, paper type, GSM, width, diameter, quantity, grade, price, currency, incoterm, available_from, location
• High-frequency updates (new surplus daily)

3. Mills

• Paper manufacturing facilities
• Fields: name, location, country, product types, capacity, contact info, onboarding status
• Relationship: one-to-many with Surplus Availability

4. Mills Surplus Visibility

• Geographic visibility configuration per mill
• Controls which regions can see a mill's surplus
• NOT per-buyer — per-region (Europe, Americas, Asia, etc.)
• Business logic: some mills only want to sell surplus in certain markets

5. Buyers

• Purchasing companies
• Fields: name, location, country, company type (converter, trader, end-user), volume needs
• Relationship: one-to-many with Buyer Specifics

6. Buyer Specifics

• Detailed buyer requirements / purchase preferences
• Fields: paper types, GSM ranges, width ranges, quality grades, volume ranges, preferred incoterms, delivery locations
• Used by matching algorithm to find relevant surplus

8 Core Actions

1. Extract Buyers

• Onboard and profile buyer companies
• Collect specifications and preferences
• Manual + automated (from inquiry emails)

2. Extract Suppliers (Mills)

• Onboard and profile mill suppliers
• Map product capabilities and typical surplus patterns
• Manual + automated (from historical data)

3. Exclusivity

• 48-hour exclusive window for matched buyers
• When surplus matches buyer specs, buyer gets first right of refusal
• Timer-based: auto-expires after 48 hours if no response
• Only one exclusivity per surplus item at a time

4. Pre-Production

• Track surplus that will be available before production completes
• Mills can list planned overruns, trial runs
• Early matching before surplus physically exists

5. Newsletter

• Automated newsletter generation for matched buyers
• Contains: matched surplus items, specs, pricing, availability
• Frequency: weekly or on-demand when high-relevance matches found
• HTML email with structured data

6. Container Assembly

• Optimize container loading from multiple surplus lots
• Combine partial rolls from different lots into full containers
• Respect weight limits (20ft: 18-22MT, 40ft: 24-26MT, 40ft HC: 24-28MT)
• Consider roll dimensions, stacking, dunnage requirements

7. Excel Ingestion

• Parse mill Excel surplus spreadsheets
• AI-powered extraction (varying formats per mill)
• Map to structured surplus data model
• Handle: different column names, units, formats, languages

8. Visibility Config

• Configure geographic visibility rules for mill surplus
• Regional controls (not per-buyer)
• Business logic: market protection, price differentiation by region

Key Design Patterns

Spec-Based Matching

• All matching is on technical specifications, not free text or product names
• Match dimensions: paper type, GSM (±tolerance), width (±tolerance), grade, geography
• Scoring: each match gets a relevance score based on spec closeness
• Threshold: only surface matches above minimum relevance score

Geographic Visibility (NOT per-buyer)

• Surplus visibility controlled at the regional level
• Regions: Europe, North America, South America, Asia, Middle East, Africa
• A mill can choose to show surplus only in specific regions
• All buyers in a visible region can see the surplus

48-Hour Exclusivity Window

• Triggered when a high-relevance match is found
• The buyer gets 48 hours of exclusive access
• During exclusivity: surplus not shown to other buyers
• After expiry: surplus becomes visible to all matched buyers
• Only one active exclusivity per surplus item

The Three Pillars Pattern

Every feature touching transactions must consider:

1. Credit/Payment — Who pays, when, what terms?
2. Documentation — What paperwork is needed?
3. Trust/Relationships — Does this build or erode trust?

Service Layer Pattern (NEW — first introduced in containers app)

• Business logic lives in apps/<app>/services.py, not in views or serializers
• Services are pure functions (stateless, testable without HTTP)
• Views call services, not the other way around
• Example: containers/services.py has 4 core functions for fill optimization:
  • calculate_container_gap() — pure math, no DB
  • find_fill_suggestions() — queries DB, scores candidates, returns dicts
  • calculate_freight_comparison() — DB lookup with fallback to hardcoded rates
  • build_fill_proposal() — orchestrates the above + creates model instances
• Tests: service tests in tests/test_services/, API tests in tests/test_api/
• Pattern should be replicated for matching algorithm, ingestion parsing, etc.

Zero-Friction Mill Pattern

• Mills don't change their workflow
• They email Excel files (as they do today)
• AI handles all parsing and structuring
• Mill doesn't need to log in to a dashboard (nice-to-have, not required)