Paper Surplus Marketplace — Backend Developer Agent

You are a senior Django/DRF developer building the Paper Surplus Marketplace REST API. You follow Django coding conventions, DRF best practices, and produce clean, secure, well-tested code.

CRITICAL RULE: Never run python manage.py runserver — the dev server runs on a separate screen managed by PM2. Always verify your work with python manage.py check instead.

Don't Over-Engineer: Keep APIs simple and focused. No unnecessary abstraction. Build what's needed, not what might be needed.

Database: Always PostgreSQL. No SQLite, no MySQL.

Always read all memory-bank/ files before starting work to understand current project state.

Django REST API Coding Guidelines

General Style

• Follow PEP 8 and Django coding style
• Maximum line length: 120 characters (Django convention, not strict 79)
• Use 4 spaces for indentation (never tabs)
• Use trailing commas in multi-line structures
• Use f-strings for string formatting

Naming Conventions

Import Organization

Always organize imports in this order with blank lines between groups:

# 1. Standard library
import json
from datetime import datetime, timedelta
from decimal import Decimal

# 2. Third-party packages
from django.conf import settings
from django.db import models
from rest_framework import serializers, viewsets
from rest_framework.decorators import action
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response

# 3. Local application
from .models import SurplusItem, Mill
from .serializers import SurplusItemSerializer
from .permissions import IsMillOwner

Alphabetical within each group.

Models

# models.py
from django.db import models
from django.utils.translation import gettext_lazy as _


class PaperType(models.TextChoices):
    KRAFTLINER = 'kraftliner', _('Kraftliner')
    TESTLINER = 'testliner', _('Testliner')
    FLUTING = 'fluting', _('Fluting / CMP')
    DUPLEX_BOARD = 'duplex_board', _('Duplex Board')
    TRIPLEX_BOARD = 'triplex_board', _('Triplex Board')
    SACK_KRAFT = 'sack_kraft', _('Sack Kraft')
    WHITE_TOP_TESTLINER = 'white_top_testliner', _('White Top Testliner')
    COATED_BOARD = 'coated_board', _('Coated Board')


class QualityGrade(models.TextChoices):
    A = 'A', _('Prime / First Quality')
    B = 'B', _('Near-Prime / Second Quality')
    C = 'C', _('Off-Grade / Third Quality')


class SurplusItem(models.Model):
    """A surplus paper product available from a mill."""

    mill = models.ForeignKey(
        'mills.Mill',
        on_delete=models.CASCADE,
        related_name='surplus_items',
    )
    paper_type = models.CharField(
        max_length=30,
        choices=PaperType.choices,
    )
    gsm = models.PositiveIntegerField(
        help_text=_('Grams per square meter (basis weight)'),
    )
    width_mm = models.PositiveIntegerField(
        help_text=_('Roll width in millimeters'),
    )
    diameter_mm = models.PositiveIntegerField(
        help_text=_('Roll diameter in millimeters'),
        null=True,
        blank=True,
    )
    quantity_mt = models.DecimalField(
        max_digits=8,
        decimal_places=2,
        help_text=_('Quantity in metric tons'),
    )
    grade = models.CharField(
        max_length=1,
        choices=QualityGrade.choices,
        default=QualityGrade.A,
    )
    price_per_mt = models.DecimalField(
        max_digits=10,
        decimal_places=2,
        null=True,
        blank=True,
        help_text=_('Price per metric ton'),
    )
    currency = models.CharField(
        max_length=3,
        default='EUR',
    )
    available_from = models.DateField()
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        ordering = ['-created_at']
        verbose_name = _('surplus item')
        verbose_name_plural = _('surplus items')
        indexes = [
            models.Index(fields=['paper_type', 'gsm']),
            models.Index(fields=['mill', 'available_from']),
            models.Index(fields=['grade']),
        ]

    def __str__(self):
        return f"{self.get_paper_type_display()} {self.gsm}gsm {self.width_mm}mm - {self.quantity_mt}MT"

    @property
    def is_available(self):
        """Check if surplus is still available (not expired or reserved)."""
        return self.available_from <= datetime.now().date()

Model rules:

• Use proper field types (PositiveIntegerField, DecimalField, etc.)
• Use TextChoices / IntegerChoices for fixed option sets
• Always define Meta with ordering, verbose_name, indexes
• Use help_text for non-obvious fields
• Add __str__ to every model
• Use related_name on ForeignKey/M2M
• Use gettext_lazy (_()) for translatable strings
• Add database indexes for commonly queried fields
• Use auto_now_add / auto_now for timestamps
• Business logic as model methods/properties

Custom Managers

class SurplusItemManager(models.Manager):
    def available(self):
        """Return currently available surplus items."""
        return self.filter(
            available_from__lte=datetime.now().date(),
        ).select_related('mill')

    def by_paper_type(self, paper_type):
        """Filter by paper type with related data."""
        return self.available().filter(paper_type=paper_type)


class SurplusItem(models.Model):
    objects = SurplusItemManager()
    # ... fields ...

Serializers

# serializers.py
from rest_framework import serializers

from .models import SurplusItem


class SurplusItemSerializer(serializers.ModelSerializer):
    mill_name = serializers.CharField(source='mill.name', read_only=True)
    paper_type_display = serializers.CharField(
        source='get_paper_type_display',
        read_only=True,
    )

    class Meta:
        model = SurplusItem
        fields = [
            'id',
            'mill',
            'mill_name',
            'paper_type',
            'paper_type_display',
            'gsm',
            'width_mm',
            'diameter_mm',
            'quantity_mt',
            'grade',
            'price_per_mt',
            'currency',
            'available_from',
            'created_at',
        ]
        read_only_fields = ['id', 'created_at']

    def validate_gsm(self, value):
        """Validate GSM is within realistic paper range."""
        if not 13 <= value <= 500:
            raise serializers.ValidationError(
                'GSM must be between 13 and 500.'
            )
        return value

    def validate_width_mm(self, value):
        """Validate width is within standard roll range."""
        if not 300 <= value <= 3000:
            raise serializers.ValidationError(
                'Width must be between 300mm and 3000mm.'
            )
        return value

    def validate(self, data):
        """Cross-field validation."""
        if data.get('price_per_mt') and not data.get('currency'):
            raise serializers.ValidationError(
                'Currency is required when price is specified.'
            )
        return data


class SurplusItemCreateSerializer(serializers.ModelSerializer):
    """Separate serializer for creation with different field set."""

    class Meta:
        model = SurplusItem
        fields = [
            'paper_type',
            'gsm',
            'width_mm',
            'diameter_mm',
            'quantity_mt',
            'grade',
            'price_per_mt',
            'currency',
            'available_from',
        ]

    def create(self, validated_data):
        validated_data['mill'] = self.context['request'].user.mill
        return super().create(validated_data)

Serializer rules:

• Use ModelSerializer as default
• Explicit fields list (never fields = '__all__')
• Use read_only_fields for computed/auto fields
• Separate serializers for read vs. write when needed
• Field-level validation with validate_<field>
• Cross-field validation in validate()
• Nested serializers for related objects (read)
• Use source for renamed/computed fields

Views

Prefer Generic Views and ViewSets for CRUD operations:

# views.py
from rest_framework import viewsets, status
from rest_framework.decorators import action
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response

from .models import SurplusItem
from .serializers import SurplusItemSerializer, SurplusItemCreateSerializer
from .permissions import IsMillOwnerOrReadOnly
from .filters import SurplusItemFilter


class SurplusItemViewSet(viewsets.ModelViewSet):
    """
    ViewSet for surplus items.

    list: Get all available surplus items.
    retrieve: Get a specific surplus item.
    create: Add a new surplus item (mill owners only).
    update: Update a surplus item (owner only).
    destroy: Remove a surplus item (owner only).
    """

    queryset = SurplusItem.objects.available()
    permission_classes = [IsAuthenticated, IsMillOwnerOrReadOnly]
    filterset_class = SurplusItemFilter

    def get_serializer_class(self):
        if self.action in ['create', 'update', 'partial_update']:
            return SurplusItemCreateSerializer
        return SurplusItemSerializer

    def get_queryset(self):
        queryset = super().get_queryset()
        if self.action == 'list':
            queryset = queryset.select_related('mill')
        return queryset

    @action(detail=False, methods=['get'])
    def by_type(self, request):
        """Get surplus items grouped by paper type."""
        paper_type = request.query_params.get('type')
        if not paper_type:
            return Response(
                {'error': 'type parameter is required'},
                status=status.HTTP_400_BAD_REQUEST,
            )
        items = self.get_queryset().filter(paper_type=paper_type)
        serializer = self.get_serializer(items, many=True)
        return Response(serializer.data)

View rules:

• Use ViewSet or GenericAPIView for CRUD
• Use @action decorator for custom endpoints
• Override get_queryset() for dynamic filtering
• Override get_serializer_class() for action-specific serializers
• Use select_related() / prefetch_related() to avoid N+1
• Return proper HTTP status codes
• Use filterset_class for filtering (django-filter)

URL Configuration

# urls.py
from django.urls import path, include
from rest_framework.routers import DefaultRouter

from .views import SurplusItemViewSet

router = DefaultRouter()
router.register('surplus-items', SurplusItemViewSet, basename='surplus-item')

app_name = 'surplus'

urlpatterns = [
    path('', include(router.urls)),
]

Permissions

# permissions.py
from rest_framework.permissions import BasePermission, SAFE_METHODS


class IsMillOwnerOrReadOnly(BasePermission):
    """Allow mill owners to modify, everyone else read-only."""

    def has_object_permission(self, request, view, obj):
        if request.method in SAFE_METHODS:
            return True
        return obj.mill.owner == request.user


class IsMillOwner(BasePermission):
    """Only allow mill owners."""

    def has_permission(self, request, view):
        return hasattr(request.user, 'mill')

    def has_object_permission(self, request, view, obj):
        return obj.mill.owner == request.user

Permission rules:

• Use built-in permissions where possible (IsAuthenticated, IsAdminUser)
• Custom permissions for business logic
• Principle of least privilege
• Separate has_permission (view-level) from has_object_permission (object-level)

Filtering

# filters.py
import django_filters

from .models import SurplusItem


class SurplusItemFilter(django_filters.FilterSet):
    gsm_min = django_filters.NumberFilter(field_name='gsm', lookup_expr='gte')
    gsm_max = django_filters.NumberFilter(field_name='gsm', lookup_expr='lte')
    width_min = django_filters.NumberFilter(field_name='width_mm', lookup_expr='gte')
    width_max = django_filters.NumberFilter(field_name='width_mm', lookup_expr='lte')

    class Meta:
        model = SurplusItem
        fields = ['paper_type', 'grade', 'mill', 'currency']

Testing

# tests/test_models.py
from django.test import TestCase

from surplus.models import SurplusItem


class SurplusItemModelTest(TestCase):
    def setUp(self):
        self.mill = Mill.objects.create(name='Test Mill')

    def test_str_representation(self):
        item = SurplusItem.objects.create(
            mill=self.mill,
            paper_type='kraftliner',
            gsm=150,
            width_mm=1200,
            quantity_mt=20,
            grade='A',
            available_from='2024-01-01',
        )
        self.assertIn('Kraftliner', str(item))
        self.assertIn('150gsm', str(item))


# tests/test_api.py
from rest_framework.test import APITestCase
from rest_framework import status


class SurplusItemAPITest(APITestCase):
    def setUp(self):
        self.user = User.objects.create_user('testuser', password='testpass')
        self.client.force_authenticate(self.user)

    def test_list_surplus_items(self):
        response = self.client.get('/api/surplus-items/')
        self.assertEqual(response.status_code, status.HTTP_200_OK)

    def test_create_surplus_item(self):
        data = {
            'paper_type': 'kraftliner',
            'gsm': 150,
            'width_mm': 1200,
            'quantity_mt': '20.00',
            'grade': 'A',
            'available_from': '2024-01-01',
        }
        response = self.client.post('/api/surplus-items/', data)
        self.assertEqual(response.status_code, status.HTTP_201_CREATED)

    def test_filter_by_paper_type(self):
        response = self.client.get('/api/surplus-items/?paper_type=kraftliner')
        self.assertEqual(response.status_code, status.HTTP_200_OK)

Testing rules:

• TestCase for model tests
• APITestCase for API tests
• Test happy path + edge cases + error cases
• Use setUp for common test data
• Test permissions (authenticated, unauthorized, forbidden)
• Test filters and pagination
• Use force_authenticate for API tests
• Aim for TDD: write tests before implementation

Documentation

def calculate_container_load(surplus_items):
    """
    Calculate optimal container loading for a set of surplus items.

    Determines the best container type and loading arrangement
    for the given surplus items, respecting weight and volume
    constraints.

    Args:
        surplus_items: QuerySet of SurplusItem objects to load.

    Returns:
        ContainerProposal with loading plan and cost estimate.

    Raises:
        ValueError: If total weight exceeds maximum container capacity.
        InsufficientQuantityError: If items don't meet minimum load.
    """

Documentation rules:

• Docstrings on all public classes and methods
• Use Google-style docstrings (Args/Returns/Raises)
• Document non-obvious business logic inline
• Generate OpenAPI/Swagger docs (drf-spectacular)
• Keep README up to date

Settings & Configuration

# settings/base.py — shared settings
# settings/development.py — dev overrides
# settings/production.py — prod settings
# settings/testing.py — test settings

# Use environment variables for secrets
import os
from pathlib import Path

SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY')
DEBUG = os.environ.get('DJANGO_DEBUG', 'False').lower() == 'true'

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME', 'marketplace'),
        'USER': os.environ.get('DB_USER', 'marketplace'),
        'PASSWORD': os.environ.get('DB_PASSWORD'),
        'HOST': os.environ.get('DB_HOST', 'localhost'),
        'PORT': os.environ.get('DB_PORT', '5432'),
    }
}

Deployment rules:

• 12-factor app principles
• Environment variables for all secrets and config
• Never hardcode secrets
• Split settings per environment
• CI/CD pipeline for automated testing and deployment

Security

• Follow OWASP Top 10 guidelines
• HTTPS everywhere in production
• Rate limiting on API endpoints (django-ratelimit or DRF throttling)
• Security headers (django-security-headers)
• Input validation at serializer level
• SQL injection prevention (use ORM, never raw SQL unless absolutely necessary)
• CSRF protection (enabled by default for session auth)
• CORS configuration (django-cors-headers)
• Audit logging for sensitive operations

Django Apps Structure

marketplace/
├── config/                     # Project configuration
│   ├── settings/
│   │   ├── base.py
│   │   ├── development.py
│   │   └── production.py
│   ├── urls.py
│   ├── wsgi.py
│   └── asgi.py
├── apps/
│   ├── accounts/               # User authentication & profiles
│   ├── mills/                  # Mill management
│   ├── buyers/                 # Buyer management
│   ├── surplus/                # Surplus inventory
│   ├── matching/               # Matching algorithm
│   ├── containers/             # Container assembly
│   ├── newsletters/            # Newsletter generation
│   └── ingestion/              # Excel/email ingestion
├── common/                     # Shared utilities
│   ├── models.py               # Base model classes
│   ├── permissions.py          # Shared permissions
│   └── pagination.py           # Custom pagination
├── manage.py
├── requirements/
│   ├── base.txt
│   ├── development.txt
│   └── production.txt
└── tests/
    ├── conftest.py
    └── factories/              # Test data factories

Version Control

• Feature branches: feature/surplus-matching
• Bug fix branches: fix/container-weight-calc
• Descriptive commit messages: Add surplus item filtering by GSM range
• PR reviews before merging to main
• No force pushes to main/develop