Skip to content

System Architecture

RotaCC is a Django 5.2 monolith that generates and manages on-call rotas for primary care (general practice) clinicians. It uses a service-layer pattern where business logic lives in dedicated service classes rather than in views or models, and Celery for asynchronous rota generation and email delivery.

System Diagram

graph TB
    subgraph Client
        Browser[Browser]
    end

    subgraph Production
        Nginx[Nginx<br/>Reverse Proxy]
        Django[Django 5.2<br/>Gunicorn 4 workers]
        CeleryW[Celery Worker<br/>+ Beat Scheduler]
    end

    subgraph Data
        PG[(PostgreSQL 15)]
        Redis[(Redis 7)]
    end

    subgraph External
        SMTP[SMTP / Email Provider]
    end

    Browser --> Nginx --> Django
    Django --> PG
    Django --> Redis
    CeleryW --> Redis
    CeleryW --> PG
    CeleryW --> SMTP

Django serves HTTP requests via Gunicorn (4 workers, 120s timeout). Celery workers consume tasks from Redis, which acts as both the Celery broker and the Django cache backend. PostgreSQL is the primary data store. Nginx sits in front as a reverse proxy in production deployments.

Django Apps

The project is organised into local apps and utility packages. Apps with an AppConfig class are registered in INSTALLED_APPS; utility packages provide shared code without Django app registration.

Registered Apps (in INSTALLED_APPS)

App Purpose Key Models / Services
config Core models and system configuration Clinician, Shift, WorkingTerm, LeaveRequest, SystemConfiguration, ClinicianGroup
frontend User-facing views, templates, and calendar feed CalendarFeedService
auth_app Authentication, invitation-based signup Invitation, SupervisionRelationship
audit_trail Middleware-based audit logging of model changes AuditLog model, AuditMiddleware
rota_generation Multi-phase rota generation engine RotaGenerationTask, OptimizationEngine, CoverageSimulationService, phase modules
leave_processing Leave request validation and balance tracking LeaveRequestProcessor
tasks Celery app configuration and task definitions celery_app, task modules (rota_tasks, email_tasks, notification_tasks, shift_swap_tasks, backup_tasks)
backup_restore Database backup and restore management pg_dump/restore commands, admin views
shift_swap Clinician-to-clinician shift swap workflow ShiftSwapService, ShiftSwapRequest
reports Statistical reporting and forecasting ClinicianStatsService, ClinicianTotalsService, StaffingForecastService, WorkloadReportService
shift_management Shift lifecycle, undo, and batch operations UndoService, Shift, BulkShiftOperation

Utility Packages (not in INSTALLED_APPS)

Package Purpose Key Components
calculations Date utilities, WTE calculations, and shared mathematical helpers Date helpers, WTE calculation functions
manual_adjustments Admin override services for shifts ManualAdjustmentService, ShiftManagementService, ManualAdjustment model
rebalancing Rota rebalancing after leave or staffing changes RebalancingService
alerts Alert generation for staffing issues and validation warnings AlertService, Alert model
api REST API viewsets and versioned URL routing (v1, v2) DRF viewsets for clinicians, shifts, leave, statistics
validators Staffing and coverage validation StaffingValidator

Rota Generation Pipeline

Rota generation is a six-phase pipeline. Each phase runs as an independent Celery task, with a pre-generation database backup triggered automatically. The RotaGenerationTask model provides application-level locking to prevent concurrent generation for the same date range.

flowchart LR
    P1[Phase 1<br/>Fixed Shifts] --> P2[Phase 2<br/>Alternate Weeks]
    P2 --> P3[Phase 3<br/>Deficit Fill]
    P3 --> P4[Phase 4<br/>Partner Minimums]
    P4 --> P5[Phase 5<br/>WTE Optimisation]
    P5 --> P6[Phase 6<br/>Duty Doctors]

Phase 1 -- Salaried and Partner Fixed Shifts. Creates shifts from fixed working days configured on each clinician's WorkingTerm. Covers salaried doctors' regular working days, partner must-work days, and locum bookings. All shifts are pinned (is_pinned=True).

Phase 2 -- Alternate Week Patterns. Applies Week A / Week B alternate working patterns for salaried clinicians. Starts from the first full Monday in the date range, skips bank holidays and approved leave, and pins all created shifts.

Phase 3 -- Deficit-Based Allocation. Calculates staffing deficits per day (minimum required minus current shifts) and per clinician (target shifts minus current shifts). Allocates shifts one at a time to the day with the highest deficit, selecting the most under-worked clinician. This ensures balanced distribution relative to each clinician's WTE.

Phase 4 -- Partner Minimum Shifts. Ensures every partner meets their weekly minimum shift requirement. Adds shifts on days with the highest remaining deficit. Only applies to partner-type clinicians.

Phase 5 -- WTE Weight Tuning Optimisation. Post-processes the rota using Google OR-Tools' CP-SAT constraint solver. Rearranges and adds partner shifts to satisfy multiple competing objectives: WTE fairness across clinicians, Monday balance, weekly evenness, and staffing levels. Only unpinned partner STANDARD shifts are decision variables; salaried, locum, pinned, and DUTY shifts remain untouched. Supports configurable optimisation weights via OptimizationWeights.

Phase 6 -- Duty Doctor Assignment. Assigns duty shifts (on-call, extended hours) prioritising salaried doctors. Ensures no salaried doctor has more than one duty shift per week and avoids consecutive-week duty assignments. Remaining duty shifts are distributed to partners to maintain balance. Uses a rotating system for fair distribution.

Data Flow

Rota Generation Flow

  1. Admin triggers generation from the frontend (selects date range and clinicians)
  2. API endpoint receives the request and dispatches a Celery task
  3. A pre-generation pg_dump backup is taken automatically
  4. RotaGenerationTask record is created with RUNNING status (prevents concurrent generation)
  5. Phases 1 through 6 execute sequentially, each creating shifts in the database
  6. After each phase completes, alert generation runs to flag any staffing issues
  7. On completion, the task status is set to COMPLETED and audit log entries are created
  8. Optional email notification is sent to the admin who triggered generation
  9. If any phase fails, the task is marked FAILED and a failure notification email is sent

Leave Request Flow

  1. Clinician submits a leave request via the frontend or API
  2. LeaveRequestProcessor validates the request (date range, leave balance, type rules)
  3. Leave request is created with PENDING status
  4. Admin reviews and approves or rejects the request
  5. On approval, any overlapping shifts are cancelled or flagged for reassignment
  6. Leave balance is updated on the clinician's working term
  7. A rebalancing task may be triggered to redistribute shifts affected by the leave
  8. Notification emails are sent to the clinician

Core Data Model

erDiagram
    Clinician ||--o{ WorkingTerm : "has contracts"
    Clinician ||--o{ Shift : "assigned to"
    Clinician ||--o{ LeaveRequest : "submits"
    Clinician ||--o{ ShiftSwapRequest : "requests"
    Clinician }o--o{ ClinicianGroup : "belongs to"

    WorkingTerm {
        string type "SALARIED, PARTNER, LOCUM, FY_DOCTOR, ST_DOCTOR"
        date start_date
        date end_date
        decimal wte_percentage
        json fixed_working_days
        json alternate_week_pattern
    }

    Shift {
        date date
        string type "STANDARD, DUTY, STUDY_LEAVE, CORONERS"
        string duration "FULL, HALF"
        string status "SCHEDULED, COMPLETED, CANCELLED"
        boolean is_pinned
    }

    LeaveRequest {
        string type "Annual, Study, Maternity, Paternity, etc."
        string status "REQUESTED, APPROVED, DENIED, CANCELLED"
        date start_date
        date end_date
        boolean impacts_balance
    }

    SystemConfiguration {
        decimal target_wte
        int minimum_staffing_per_day
        int fixed_period_weeks
    }

Key Relationships

  • Clinician to WorkingTerm -- One-to-many. A clinician can have multiple working terms over time (e.g., changing from FY Doctor to Salaried). Each term defines the contract type, WTE percentage, and fixed working days.
  • Clinician to Shift -- One-to-many. Each shift belongs to one clinician and one date. Shifts have a type (standard or duty), a duration (9h or 12h), and a status (scheduled, cancelled, replaced).
  • Clinician to LeaveRequest -- One-to-many. Leave requests track type, status, date range, and balance impact.
  • SystemConfiguration -- Singleton model (get_or_create()) holding system-wide settings: target WTE, minimum staffing per day, duty requirements, fixed period configuration, and Christmas restricted periods.
  • Shift.is_pinned -- Pinned shifts (from Phases 1 and 2) are not modified by later phases. This protects fixed commitments like salaried doctor working days.

Key Design Patterns

Service Layer Pattern Business logic is encapsulated in service classes (*Service, *Processor, *Engine) rather than in views or model methods. Views handle HTTP concerns and delegate to services. This keeps models focused on data definition and makes business logic testable in isolation.

Singleton Configuration SystemConfiguration uses get_or_create() to ensure exactly one row holds all system-wide settings. This avoids scattered configuration and provides a single source of truth for parameters like minimum staffing levels and target WTE.

Deficit-Based Allocation Phases 3 and 4 use a greedy deficit algorithm: for each unmet staffing day, the system allocates a shift to the clinician furthest below their WTE-proportional target. This produces balanced rotas without requiring a global optimisation pass.

CP-SAT Constraint Optimisation Phase 5 uses Google OR-Tools' CP-SAT solver for multi-objective optimisation. Only unpinned partner STANDARD shifts are treated as decision variables, keeping the search space manageable while respecting hard constraints from earlier phases.

Fixed vs Flexible Periods The system divides the rota timeline into a fixed period (default 6 weeks) where shifts are visible and committed, and a flexible period where shifts can be regenerated. This gives clinicians certainty about their near-term schedule while allowing adjustments further out.

Audit Trail The AuditMiddleware in audit_trail logs all model changes (create, update, delete) with previous and new state, the acting user, and a reason. This provides a complete history for compliance and debugging.

Concurrent Task Protection RotaGenerationTask records with RUNNING status prevent parallel rota generation for the same date range. Tasks are marked COMPLETED or FAILED on finish, releasing the lock.

Infrastructure

The application is deployed as a Docker Compose stack with four services:

Service Image / Build Role
web Custom Dockerfile Django application served by Gunicorn on port 8000
celery_worker Custom Dockerfile Celery worker with beat scheduler for async tasks and periodic jobs
postgres postgres:15-alpine Primary database
redis redis:7-alpine Celery broker and Django cache backend

Persistent Volumes

  • pgdata -- PostgreSQL data directory
  • static_data -- Collected static files served by Nginx
  • media_data -- User-uploaded files
  • backup_data -- Database backup files (shared between web and celery_worker)

Periodic Tasks (Celery Beat)

Task Schedule Description
daily_pg_dump_backup 03:00 daily Full database backup via pg_dump
cleanup_old_backups 04:00 daily Removes aged backup files

URL Structure

Path Handler
/ Frontend views (main application)
/accounts/ django-allauth authentication flow
/calendar-feed/<key>/rota.ics ICS calendar feed for clinicians
/django-admin/ Unfold admin interface
/api/v1/, /api/v2/ Versioned REST API
/api/rota/generate/ Rota generation endpoint
/api/rebalance/ Rebalancing endpoint
/api/statistics/ Statistics and forecast endpoints
/reports/ Report pages
/admin/backup/ Backup management

Settings Architecture

Django settings are split into a package at rota/settings/:

  • base.py -- Common settings using python-decouple for environment variable loading
  • prod.py -- Production overrides (PostgreSQL, Redis cache, security headers)
  • dev.py -- Development overrides (DEBUG, SQLite, LocMem cache, debug toolbar)

The active settings module is selected via the DJANGO_SETTINGS_MODULE environment variable, defaulting to rota.settings.prod.