DiunaBI/.claude/project-context.md

# DiunaBI Project Context

> This file is auto-generated for Claude Code to quickly understand the project structure.
> Last updated: 2025-12-06

## RECENT CHANGES (This Session)

**API Key Authorization Fix for Cron Jobs (Dec 6, 2025):**
- ✅ **Fixed 401 Unauthorized on API Key Endpoints** - Cron jobs calling `/jobs/schedule` endpoints were getting rejected despite valid API keys
- ✅ **Added [AllowAnonymous] Attribute** - Bypasses controller-level `[Authorize]` to allow `[ApiKeyAuth]` filter to handle authorization
- ✅ **Three Endpoints Fixed** - Applied fix to all job scheduling endpoints:
  - `POST /jobs/schedule` - Schedule all jobs (imports + processes)
  - `POST /jobs/schedule/imports` - Schedule import jobs only
  - `POST /jobs/schedule/processes` - Schedule process jobs only
- Root cause: Controller-level `[Authorize]` attribute required JWT Bearer auth for all endpoints, blocking API key authentication
- Solution: Add `[AllowAnonymous]` to allow `[ApiKeyAuth]` filter to validate X-API-Key header
- Files modified: [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs)
- Status: Cron jobs can now authenticate with API key via X-API-Key header

**SignalR Authentication Token Flow Fix (Dec 6, 2025):**
- ✅ **TokenProvider Population** - Fixed `TokenProvider.Token` never being set with JWT, causing 401 Unauthorized on SignalR connections
- ✅ **AuthService Token Management** - Injected `TokenProvider` into `AuthService` and set token in 3 key places:
  - `ValidateWithBackendAsync()` - on fresh Google login
  - `CheckAuthenticationAsync()` - on session restore from localStorage
  - `ClearAuthenticationAsync()` - clear token on logout
- ✅ **SignalR Initialization Timing** - Moved SignalR initialization from `MainLayout.OnInitializedAsync` to after authentication completes
- ✅ **Event-Driven Architecture** - `MainLayout` now subscribes to `AuthenticationStateChanged` event to initialize SignalR when user authenticates
- ✅ **Session Restore Support** - `CheckAuthenticationAsync()` now fires `AuthenticationStateChanged` event to initialize SignalR on page refresh
- Root cause: SignalR was initialized before authentication, so JWT token was empty during connection setup
- Solution: Initialize SignalR only after token is available via event subscription
- Files modified: [AuthService.cs](DiunaBI.UI.Shared/Services/AuthService.cs), [MainLayout.razor](DiunaBI.UI.Shared/Components/Layout/MainLayout.razor)
- Status: SignalR authentication working for both fresh login and restored sessions

**SignalR Authentication DI Fix (Dec 6, 2025):**
- ✅ **TokenProvider Registration** - Added missing `TokenProvider` service registration in DI container
- ✅ **EntityChangeHubService Scope Fix** - Changed from singleton to scoped to support user-specific JWT tokens
- ✅ **Bug Fix** - Resolved `InvalidOperationException` preventing app from starting after SignalR authentication was added
- Root cause: Singleton service (`EntityChangeHubService`) cannot depend on scoped service (`TokenProvider`) in DI
- Solution: Made `EntityChangeHubService` scoped so each user session has its own authenticated SignalR connection
- Files modified: [ServiceCollectionExtensions.cs](DiunaBI.UI.Shared/Extensions/ServiceCollectionExtensions.cs)

---

**Security Audit & Hardening (Dec 5, 2025):**
- ✅ **JWT Token Validation** - Enabled issuer/audience validation in [Program.cs](DiunaBI.API/Program.cs), fixed config key mismatch in [JwtTokenService.cs](DiunaBI.API/Services/JwtTokenService.cs)
- ✅ **API Key Security** - Created [ApiKeyAuthAttribute.cs](DiunaBI.API/Attributes/ApiKeyAuthAttribute.cs) with X-API-Key header auth, constant-time comparison
- ✅ **Job Endpoints** - Migrated 3 job scheduling endpoints in [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs) from URL-based to header-based API keys
- ✅ **Stack Trace Exposure** - Fixed 20 instances across 3 controllers ([JobsController.cs](DiunaBI.API/Controllers/JobsController.cs), [LayersController.cs](DiunaBI.API/Controllers/LayersController.cs), [DataInboxController.cs](DiunaBI.API/Controllers/DataInboxController.cs)) - now returns generic error messages
- ✅ **SignalR Authentication** - Added [Authorize] to [EntityChangeHub.cs](DiunaBI.API/Hubs/EntityChangeHub.cs), configured JWT token in [EntityChangeHubService.cs](DiunaBI.UI.Shared/Services/EntityChangeHubService.cs)
- ✅ **Rate Limiting** - Implemented ASP.NET Core rate limiting: 100 req/min general, 10 req/min auth in [Program.cs](DiunaBI.API/Program.cs)
- ✅ **Security Headers** - Added XSS, clickjacking, MIME sniffing protection middleware in [Program.cs](DiunaBI.API/Program.cs)
- ✅ **Input Validation** - Added pagination limits (1-1000) to GetAll endpoints in 3 controllers
- ✅ **User Enumeration** - Fixed generic auth error in [GoogleAuthService.cs](DiunaBI.API/Services/GoogleAuthService.cs)
- ✅ **Sensitive Data Logging** - Made conditional on development only in [Program.cs](DiunaBI.API/Program.cs)
- ✅ **Base64 Size Limit** - Added 10MB limit to DataInbox in [DataInboxController.cs](DiunaBI.API/Controllers/DataInboxController.cs)
- Files modified: 12 files (API: Program.cs, 4 controllers, 3 services, 1 hub, 1 new attribute; UI: EntityChangeHubService.cs, ServiceCollectionExtensions.cs)
- Security status: 5/5 CRITICAL fixed, 3/3 HIGH fixed, 4/4 MEDIUM fixed

**Seq Removal - Logging Cleanup (Dec 5, 2025):**
- ✅ Removed Seq logging sink to eliminate commercial licensing concerns
- ✅ Removed `Serilog.Sinks.Seq` NuGet package from DiunaBI.API.csproj
- ✅ Removed Seq sink configuration from appsettings.Development.json
- ✅ Kept Serilog (free, open-source) with Console + File sinks for production-ready logging
- ✅ Build verified - no errors after Seq removal
- Files modified: [DiunaBI.API.csproj](DiunaBI.API/DiunaBI.API.csproj), [appsettings.Development.json](DiunaBI.API/appsettings.Development.json)
- Manual step required: Remove `seq` service from docker-compose.yml and add Docker log rotation config

**UI Reorganization (Dec 5, 2025):**
- ✅ Moved pages to feature-based folders: `Pages/Layers/`, `Pages/Jobs/`, `Pages/DataInbox/`
- ✅ Organized components: `Components/Layout/` (MainLayout, EmptyLayout, Routes), `Components/Auth/` (AuthGuard, LoginCard)
- ✅ Removed obsolete wrapper files (LayerListPage, JobListPage, DataInboxListPage, etc.)
- ✅ Removed duplicate component files (LayerListComponent, JobListComponent, DataInboxListComponent)
- ✅ Standardized code-behind: `.razor.cs` for complex logic, inline `@code` for simple pages
- ✅ Updated `_Imports.razor` with new namespaces: `DiunaBI.UI.Shared.Components.Layout`, `DiunaBI.UI.Shared.Components.Auth`
- ✅ All routes unchanged - backward compatible

---

## PROJECT TYPE & TECH STACK

**Application Type:** Full-stack Business Intelligence (BI) platform with multi-tier architecture, real-time capabilities, and plugin system

**Core Stack:**
- Backend: ASP.NET Core 10.0 Web API
- Frontend: Blazor Server + MAUI Mobile
- Database: SQL Server + EF Core 10.0
- UI: MudBlazor 8.0
- Real-time: SignalR (EntityChangeHub)
- Google: Sheets API, Drive API, OAuth
- Logging: Serilog (Console, File)
- Auth: JWT Bearer + Google OAuth

---

## SOLUTION STRUCTURE (10 Projects)

```
DiunaBI.API (Web API)
├── Controllers: Auth, Layers, Jobs, DataInbox
├── Hubs: EntityChangeHub (SignalR real-time updates)
└── Services: GoogleAuth, JwtToken

DiunaBI.Domain (Entities)
└── User, Layer, Record, RecordHistory, QueueJob, DataInbox, ProcessSource

DiunaBI.Application (DTOs)
└── LayerDto, RecordDto, UserDto, RecordHistoryDto, PagedResult, JobDto

DiunaBI.Infrastructure (Data + Services)
├── Data: AppDbContext, Migrations (47 total)
├── Interceptors: EntityChangeInterceptor (auto-broadcasts DB changes)
├── Services: PluginManager, JobScheduler, JobWorker, GoogleSheets/Drive
├── Plugins: BaseDataImporter, BaseDataProcessor, BaseDataExporter
└── Interfaces: IPlugin, IDataProcessor, IDataImporter, IDataExporter

DiunaBI.UI.Web (Blazor Server)
└── Server-side Blazor web application

DiunaBI.UI.Mobile (MAUI)
└── iOS, Android, Windows, macOS support

DiunaBI.UI.Shared (Blazor Component Library - Reorganized)
├── Pages/
│   ├── Layers/ (Index.razor, Details.razor)
│   ├── Jobs/ (Index.razor, Details.razor)
│   ├── DataInbox/ (Index.razor, Details.razor)
│   ├── Dashboard.razor, Login.razor, Index.razor
├── Components/
│   ├── Layout/ (MainLayout, EmptyLayout, Routes)
│   └── Auth/ (AuthGuard, LoginCard)
└── Services/
    ├── LayerService, JobService, DataInboxService
    ├── EntityChangeHubService (SignalR client)
    ├── FilterStateServices (remember filters)
    └── AuthService, TokenProvider

DiunaBI.Plugins.Morska (Feature Plugin)
├── Importers: Standard, D1, D3, FK2 (4 total)
├── Processors: D6, T1, T3, T4, T5 variants (12 total)
└── Exporters: Google Sheets export (1)

DiunaBI.Plugins.PedrolloPL (Feature Plugin - NEW)
└── Importers: B3 (1 total)

DiunaBI.Tests (Testing)
└── Unit and integration tests
```

---

## CORE FUNCTIONALITY

**Purpose:** BI platform for data import, processing, transformation via modular plugin architecture. Multi-layer workflows with audit trails, real-time notifications, scheduled job processing.

**Main Features:**
1. **Layer Management** - 4 types (Import/Processed/Admin/Dictionary), parent-child relationships, soft deletes
2. **Data Records** - 32 numeric columns (Value1-32) + description, hierarchical, full audit trail
3. **Plugin Architecture** - Dynamic assembly loading, base classes in Infrastructure, 3 types (Importers/Processors/Exporters)
4. **Job Queue System** - Background worker with retry logic (30s → 2m → 5m), priority-based, auto-scheduling
5. **External Data** - DataInbox API, Google Sheets read/write, Google Drive integration
6. **Real-time Updates** - SignalR broadcasts entity changes (create/update/delete) to all connected clients
7. **Audit Trail** - RecordHistory tracks all record changes with field-level diffs and JSON summaries
8. **Filter Persistence** - UI filter states saved across sessions (LayerFilterStateService, DataInboxFilterStateService)

---

## KEY ENTITIES

**Layer**
- Id, Number, Name, Type (Import/Processed/Administration/Dictionary)
- CreatedAt/ModifiedAt, CreatedBy/ModifiedBy (with user relations)
- IsDeleted (soft delete), IsCancelled (processing control), ParentId
- Relations: Records (1-to-many), ProcessSources (1-to-many)

**Record**
- Id, Code (unique identifier), LayerId
- Value1-Value32 (double?), Desc1 (string, max 10000 chars)
- CreatedAt/ModifiedAt, CreatedBy/ModifiedBy, IsDeleted
- Audit: Full history tracked in RecordHistory table

**RecordHistory** (NEW - Migration 47)
- RecordId, LayerId, ChangedAt, ChangedById
- ChangeType (Created/Updated/Deleted)
- Code, Desc1 (snapshot at time of change)
- ChangedFields (comma-separated field names)
- ChangesSummary (JSON with old/new values)
- Indexes: (RecordId, ChangedAt), (LayerId, ChangedAt) for performance

**QueueJob**
- LayerId, LayerName, PluginName
- JobType (Import/Process)
- Priority (0 = highest), Status (Pending/Running/Completed/Failed/Retrying)
- RetryCount, MaxRetries (default 5)
- CreatedAt, LastAttemptAt, CompletedAt
- LastError (detailed error message)

**DataInbox**
- Id, Name, Source (identifiers)
- Data (base64-encoded JSON array)
- CreatedAt
- Used by importers to stage incoming data

**User**
- Id (Guid), Email, UserName
- CreatedAt, LastLoginAt
- Google OAuth identity

**ProcessSource**
- Id, SourceLayerId, TargetLayerId
- Defines layer processing relationships

---

## API ENDPOINTS

**Base:** `/` (ApiController routes)

### AuthController (/auth)
- `POST /auth/apiToken` - Exchange Google ID token for JWT (AllowAnonymous)
- `POST /auth/refresh` - Refresh expired JWT token

### LayersController (/layers)
- `GET /layers?page=1&pageSize=10&search=&type=` - List layers (paged, filterable)
- `GET /layers/{id}` - Get layer details with records
- `POST /layers` - Create new layer
- `PUT /layers/{id}` - Update layer
- `DELETE /layers/{id}` - Soft delete layer
- `POST /layers/{id}/records` - Add/update records
- `PUT /layers/{layerId}/records/{recordId}` - Update specific record
- `DELETE /layers/{layerId}/records/{recordId}` - Delete record
- `GET /layers/{layerId}/records/{recordId}/history` - Get record history
- `GET /layers/{layerId}/deleted-records` - Get deleted records with history

### JobsController (/jobs) - NEW
- `GET /jobs?page=1&pageSize=50&status=&jobType=` - List jobs (paged, filterable)
- `GET /jobs/{id}` - Get job details
- `GET /jobs/stats` - Get job statistics (counts by status)
- `POST /jobs/schedule/{apiKey}` - Schedule all jobs from layer configs
- `POST /jobs/schedule/imports/{apiKey}` - Schedule import jobs only
- `POST /jobs/schedule/processes/{apiKey}` - Schedule process jobs only
- `POST /jobs/create-for-layer/{layerId}` - Create job for specific layer (manual trigger)
- `POST /jobs/{id}/retry` - Retry failed job (resets to Pending)
- `DELETE /jobs/{id}` - Cancel pending/retrying job

### DataInboxController (/datainbox)
- `GET /datainbox?page=1&pageSize=10&search=` - List inbox items (paged, filterable)
- `GET /datainbox/{id}` - Get inbox item with decoded data
- `POST /datainbox` - Create inbox item
- `PUT /datainbox/Add/{apiKey}` - Add data (API key + Basic Auth)
- `DELETE /datainbox/{id}` - Delete inbox item

### SignalR Hub
- `/hubs/entitychanges` - SignalR hub for real-time entity change notifications
  - Event: `EntityChanged(module, id, operation)` - broadcasts to all clients
  - Modules: QueueJobs, Layers, Records, RecordHistory

---

## AUTHENTICATION & SECURITY

**Flow:**
1. Client exchanges Google ID token → `/auth/apiToken`
2. GoogleAuthService validates token with Google, maps to internal User
3. Returns JWT (7-day expiration, HS256 signing)
4. JWT required on all protected endpoints (except /auth/apiToken, /health)
5. UserId extraction middleware sets X-UserId header for audit trails

**Security:**
- Google OAuth 2.0 for identity verification
- JWT Bearer tokens for API access
- API key + Basic Auth for DataInbox external endpoints
- CORS configured for:
  - http://localhost:4200
  - https://diuna.bim-it.pl
  - https://morska.diunabi.com

---

## KEY SERVICES

### Infrastructure Services

**PluginManager**
- Location: `DiunaBI.Infrastructure/Services/PluginManager.cs`
- Loads plugin assemblies from `bin/Plugins/` directory at startup
- Registers IDataProcessor, IDataImporter, IDataExporter implementations
- Provides plugin discovery and execution

**JobSchedulerService**
- Location: `DiunaBI.Infrastructure/Services/JobSchedulerService.cs`
- Creates QueueJob entries from Administration layer configs
- Reads layer.Records with Code="Plugin", Code="Priority", Code="MaxRetries"
- Methods: ScheduleImportJobsAsync, ScheduleProcessJobsAsync, ScheduleAllJobsAsync

**JobWorkerService** (BackgroundService)
- Location: `DiunaBI.Infrastructure/Services/JobWorkerService.cs`
- Polls QueueJobs table every 10 seconds
- Executes jobs via PluginManager (Import/Process)
- Retry logic with exponential backoff: 30s → 2m → 5m delays
- Rate limiting: 5-second delay after imports (Google Sheets API quota)
- Updates job status in real-time (triggers SignalR broadcasts)

**EntityChangeInterceptor**
- Location: `DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs`
- EF Core SaveChangesInterceptor
- Captures entity changes: Added, Modified, Deleted
- Broadcasts changes via SignalR EntityChangeHub after successful save
- Uses reflection to avoid circular dependencies with IHubContext

**GoogleSheetsHelper**
- Location: `DiunaBI.Infrastructure/Helpers/GoogleSheetsHelper.cs`
- Google Sheets API v4 integration
- Methods: ReadRange, WriteRange, CreateSpreadsheet, UpdateSpreadsheet

**GoogleDriveHelper**
- Location: `DiunaBI.Infrastructure/Helpers/GoogleDriveHelper.cs`
- Google Drive API v3 integration
- Methods: UploadFile, ListFiles, MoveFile

**GoogleAuthService / JwtTokenService**
- Authentication and token management
- JWT generation and validation

### UI Services

**EntityChangeHubService**
- Location: `DiunaBI.UI.Shared/Services/EntityChangeHubService.cs`
- Singleton service for SignalR client connection
- Auto-reconnect enabled
- Event: `EntityChanged` - UI components subscribe for real-time updates
- Initialized in MainLayout.OnInitializedAsync

**LayerService / JobService / DataInboxService**
- HTTP clients for API communication
- DTOs serialization/deserialization
- Paged result handling

**LayerFilterStateService / DataInboxFilterStateService**
- Persist filter state across navigation
- Singleton services remember search, type, page selections

---

## DATABASE SCHEMA

**Total Migrations:** 47

**Latest Migrations:**

**Migration 47: RecordHistory (Dec 1, 2025)**
- **NEW Table: RecordHistory**
- Tracks all record changes (Created, Updated, Deleted)
- Fields: Id, RecordId, LayerId, ChangedAt, ChangedById, ChangeType, Code, Desc1, ChangedFields, ChangesSummary
- Indexes: IX_RecordHistory_RecordId_ChangedAt, IX_RecordHistory_LayerId_ChangedAt
- Foreign key: RecordHistory.ChangedById → Users.Id

**Migration 46: FixLayerDefaultValues (Nov 20, 2025)**
- Set default value: Layers.IsDeleted = false

**Migration 45: UpdateModel (Nov 19, 2025)**
- Added GETUTCDATE() defaults for all timestamp fields
- Changed foreign key constraints from CASCADE to RESTRICT:
  - Layers → Users (CreatedById, ModifiedById)
  - Records → Users (CreatedById, ModifiedById)
- Added FK_ProcessSources_Layers_LayerId

**Core Tables:**
- Users (authentication, audit)
- Layers (4 types, soft deletes, parent-child)
- Records (32 Value fields + Desc1, audit, soft deletes)
- RecordHistory (change tracking, field diffs, JSON summaries)
- QueueJobs (job queue, retry logic, status tracking)
- DataInbox (incoming data staging, base64 encoded)
- ProcessSources (layer relationships)

---

## PLUGIN SYSTEM

### Base Classes (Infrastructure/Plugins/)

**BaseDataImporter** (`DiunaBI.Infrastructure/Plugins/BaseDataImporter.cs`)
- Abstract base for all importers
- Methods: ImportAsync(layerId, jobId), ValidateConfiguration()
- Access: AppDbContext, PluginManager, GoogleSheetsHelper, GoogleDriveHelper

**BaseDataProcessor** (`DiunaBI.Infrastructure/Plugins/BaseDataProcessor.cs`)
- Abstract base for all processors
- Methods: ProcessAsync(layerId, jobId), ValidateConfiguration()
- Access: AppDbContext, PluginManager

**BaseDataExporter** (`DiunaBI.Infrastructure/Plugins/BaseDataExporter.cs`)
- Abstract base for all exporters
- Methods: ExportAsync(layerId, jobId), ValidateConfiguration()
- Access: AppDbContext, GoogleSheetsHelper, GoogleDriveHelper

### Morska Plugin (DiunaBI.Plugins.Morska)

**Importers (4):**
- MorskaStandardImporter - Generic CSV/Excel import
- MorskaD1Importer - D1 data format
- MorskaD3Importer - D3 data format
- MorskaFK2Importer - FK2 data format

**Processors (12):**
- MorskaD6Processor
- MorskaT1R1Processor
- MorskaT1R3Processor
- MorskaT3SingleSourceProcessor
- MorskaT3SourceYearSummaryProcessor
- MorskaT3MultiSourceSummaryProcessor
- MorskaT3MultiSourceYearSummaryProcessor
- MorskaT4R2Processor
- MorskaT4SingleSourceProcessor
- MorskaT5LastValuesProcessor
- MorskaT3MultiSourceCopySelectedCodesProcessor-TO_REMOVE (deprecated)
- MorskaT3MultiSourceCopySelectedCodesYearSummaryProcessor-TO_REMOVE (deprecated)

**Exporters (1):**
- googleSheet.export.cs - Google Sheets export

**Total:** ~6,566 lines of code

### PedrolloPL Plugin (DiunaBI.Plugins.PedrolloPL) - NEW

**Importers (1):**
- **PedrolloPLImportB3** (`DiunaBI.Plugins.PedrolloPL/Importers/PedrolloPLImportB3.cs`)
  - Imports B3 data from DataInbox
  - Uses L1-D-B3-CODES dictionary layer for region code mapping
  - Creates 12 monthly records per region (Value1-Value12)
  - Generates Import layers: L{Number}-I-B3-{Year}-{Timestamp}
  - Handles base64 JSON data decoding

---

## UI STRUCTURE (DiunaBI.UI.Shared)

### Reorganized Structure (Dec 5, 2025)

**Pages/** (Routable pages with @page directive)
```
Pages/
├── Layers/
│   ├── Index.razor + Index.razor.cs    - /layers (list with filters, pagination)
│   └── Details.razor + Details.razor.cs - /layers/{id} (detail, edit, history)
├── Jobs/
│   ├── Index.razor + Index.razor.cs    - /jobs (list with filters, real-time updates)
│   └── Details.razor                    - /jobs/{id} (detail, retry, cancel, real-time)
├── DataInbox/
│   ├── Index.razor + Index.razor.cs    - /datainbox (list with filters)
│   └── Details.razor + Details.razor.cs - /datainbox/{id} (detail, base64 decode)
├── Dashboard.razor                      - /dashboard (user info)
├── Login.razor                          - /login (Google OAuth)
└── Index.razor                          - / (redirects to /dashboard)
```

**Components/** (Reusable components, no routes)
```
Components/
├── Layout/
│   ├── MainLayout.razor     - Main app layout with drawer, nav menu
│   ├── EmptyLayout.razor    - Minimal layout for login page
│   └── Routes.razor         - Router configuration
└── Auth/
    ├── AuthGuard.razor      - Authentication guard wrapper
    └── LoginCard.razor      - Google login button component
```

**Navigation Menu:**
- Dashboard (/dashboard) - User profile
- Layers (/layers) - Layer management
- Data Inbox (/datainbox) - Incoming data review
- Jobs (/jobs) - Job queue monitoring (with real-time status updates)

**Code-Behind Pattern:**
- Complex pages (50+ lines logic): Separate `.razor.cs` files
- Simple pages: Inline `@code` blocks
- Namespaces: `DiunaBI.UI.Shared.Pages.{Feature}`

---

## REAL-TIME FEATURES (SignalR)

### Architecture

**Hub:** `DiunaBI.API/Hubs/EntityChangeHub.cs`
- Endpoint: `/hubs/entitychanges`
- Method: `SendEntityChange(string module, string id, string operation)`
- Broadcasts: `EntityChanged` event to all connected clients

**Interceptor:** `DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs`
- EF Core SaveChangesInterceptor
- Detects: Added, Modified, Deleted entities
- Broadcasts: After successful SaveChanges
- Modules: QueueJobs, Layers, Records, RecordHistory

**UI Service:** `DiunaBI.UI.Shared/Services/EntityChangeHubService.cs`
- Singleton initialized in MainLayout
- Auto-reconnect enabled
- Components subscribe: `HubService.EntityChanged += OnEntityChanged`

### Real-time Update Flow

1. User action → API endpoint
2. DbContext.SaveChangesAsync()
3. EntityChangeInterceptor captures changes
4. SignalR broadcast to all clients: `EntityChanged(module, id, operation)`
5. UI components receive event and refresh data
6. StateHasChanged() updates UI

**Example:** Job status changes appear instantly on JobDetailPage and JobListPage

---

## JOB QUEUE SYSTEM

### Components

**Entity:** `QueueJob` (DiunaBI.Domain/Entities/QueueJob.cs)
- JobType: Import, Process
- JobStatus: Pending, Running, Completed, Failed, Retrying
- Priority: 0 = highest priority
- Retry: 30s → 2m → 5m delays, max 5 attempts

**Scheduler:** `JobSchedulerService`
- Reads Administration layer configs (Type=ImportWorker/ProcessWorker)
- Auto-creates jobs based on layer.Records configuration
- API endpoints: `/jobs/schedule/{apiKey}`, `/jobs/schedule/imports/{apiKey}`, `/jobs/schedule/processes/{apiKey}`

**Worker:** `JobWorkerService` (BackgroundService)
- Polls every 10 seconds
- Executes via PluginManager
- Exponential backoff on failures
- Rate limiting for Google API quota
- Real-time status updates via SignalR

**UI:** `Pages/Jobs/`
- Index.razor - Job list with filters, real-time updates
- Details.razor - Job detail with retry/cancel, real-time status

### Job Lifecycle

1. **Creation** - JobSchedulerService or manual via API
2. **Queued** - Status: Pending, sorted by Priority
3. **Execution** - JobWorkerService picks up, Status: Running
4. **Completion** - Status: Completed or Failed
5. **Retry** - On failure, Status: Retrying with exponential backoff
6. **Real-time** - All status changes broadcast via SignalR

**Statistics Endpoint:** `GET /jobs/stats`
```json
{
  "pending": 5,
  "running": 2,
  "completed": 150,
  "failed": 3,
  "retrying": 1,
  "total": 161
}
```

---

## RECENT DEVELOPMENT

**Recent Commits (Dec 2-5, 2025):**
- **193127b:** SignalR for realtime entitychanges (Dec 4)
- **bf2beda, 942da18:** Build fixes (Dec 4)
- **a3fa8f9:** B3 import is working (Dec 4)
- **0e3b393:** WIP: b3 plugin (Dec 3)
- **445c07a:** Morska plugins refactor (Dec 2)
- **3f8e62f:** WIP: queue engine (Dec 2)
- **248106a:** Plugins little refactor (Dec 2)
- **587d4d6:** Pedrollo plugins (Dec 2)
- **e70a8dd:** Remember list filters (Dec 2)
- **89859cd:** Record history is working (Dec 1)

**Development Focus (Last 30 Days):**
1. ✅ Real-time updates (SignalR integration)
2. ✅ Job queue system (background worker, retry logic)
3. ✅ PedrolloPL plugin (B3 importer)
4. ✅ Record history tracking (audit trail)
5. ✅ UI reorganization (feature-based folders)
6. ✅ Plugin refactoring (base classes in Infrastructure)
7. ✅ Filter persistence (UI state management)

**Major Features Added:**
- SignalR real-time entity change notifications
- Background job processing with retry logic
- Record history with field-level diffs
- PedrolloPL B3 data importer
- UI reorganization (Pages/Layers, Pages/Jobs, Pages/DataInbox)
- Filter state persistence across sessions

---

## CONFIGURATION

**Key Settings (appsettings.Development.json):**
- ConnectionStrings:SQLDatabase - SQL Server (localhost:21433, DB: DiunaBI-PedrolloPL)
- JwtSettings:SecurityKey, ExpiryDays (7)
- GoogleAuth:ClientId, RedirectUri
- apiKey, apiUser, apiPass - DataInbox API security
- exportDirectory - Google Drive folder ID for exports
- apiLocalUrl - localhost:5400
- InstanceName - DEV/PROD environment identifier

**Logging Configuration:**
```json
"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning",
      "Microsoft.EntityFrameworkCore.Database.Command": "Warning",
      "Microsoft.EntityFrameworkCore.Infrastructure": "Warning",
      "System.Net.Http.HttpClient": "Warning",
      "Google.Apis": "Warning",
      "DiunaBI.Core.Services.PluginManager": "Information"
    }
  }
}
```

**CORS Origins:**
- http://localhost:4200 (development)
- https://diuna.bim-it.pl (production)
- https://morska.diunabi.com (production)

---

## PATTERNS & ARCHITECTURE

**Design Patterns:**
- Clean Architecture (Domain → Application → Infrastructure → API)
- Plugin Pattern (dynamic loading, base classes, interface contracts)
- Interceptor Pattern (EF Core SaveChangesInterceptor for change tracking)
- Hub Pattern (SignalR for real-time notifications)
- Service Pattern (dependency injection throughout)
- Repository Pattern (EF Core DbContext as repository)
- Background Service Pattern (JobWorkerService for async processing)

**Tech Versions:**
- .NET 10.0 (upgraded from .NET 8.0)
- EF Core 10.0
- C# 13.0
- Blazor Server (net10.0)
- MAUI (net10.0-ios/android/windows/macos)
- MudBlazor 8.0

**Architectural Decisions:**
- Plugin base classes in Infrastructure for reusability
- SignalR for real-time updates (no polling)
- Background service for job processing (no external scheduler)
- Soft deletes with audit trails
- Foreign key RESTRICT to prevent accidental cascades
- Feature-based folder structure in UI

---

## QUICK REFERENCE

**Database:**
- SQL Server with 47 EF Core migrations
- Auto-timestamps via GETUTCDATE() defaults
- Soft deletes (IsDeleted flag)
- Audit trails (CreatedBy, ModifiedBy, RecordHistory table)

**Build Process:**
- MSBuild target copies plugin DLLs to `bin/Plugins/` after build
- Plugins: DiunaBI.Plugins.Morska.dll, DiunaBI.Plugins.PedrolloPL.dll

**SignalR:**
- Hub: `/hubs/entitychanges`
- Broadcasts: `EntityChanged(module, id, operation)`
- Auto-reconnect enabled in UI
- Real-time updates for QueueJobs, Layers, Records

**Job Queue:**
- Auto-scheduling from layer configs (Type=ImportWorker/ProcessWorker)
- Background processing every 10 seconds
- Retry logic: 30s → 2m → 5m (max 5 retries)
- Priority-based execution (0 = highest)
- Real-time status updates via SignalR

**Plugins:**
- **Morska:** 4 importers, 12 processors, 1 exporter (~6,566 LOC)
- **PedrolloPL:** 1 importer (B3 data)
- Base classes: BaseDataImporter, BaseDataProcessor, BaseDataExporter
- Dynamic loading from `bin/Plugins/` at startup

**UI Structure:**
- Feature-based folders: Pages/Layers, Pages/Jobs, Pages/DataInbox
- Separate code-behind for complex logic (.razor.cs files)
- Inline @code for simple pages
- Organized components: Layout/, Auth/
- Filter state persistence across navigation

---

## FILE PATHS REFERENCE

**Key Configuration:**
- API: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/appsettings.json`
- API Startup: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Program.cs`

**SignalR:**
- Hub: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Hubs/EntityChangeHub.cs`
- Interceptor: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs`
- UI Service: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Services/EntityChangeHubService.cs`

**Job System:**
- Controller: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Controllers/JobsController.cs`
- Scheduler: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Services/JobSchedulerService.cs`
- Worker: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Services/JobWorkerService.cs`
- UI Pages: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Pages/Jobs/`

**Plugins:**
- Base Classes: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Plugins/`
- Morska: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Plugins.Morska/`
- PedrolloPL: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Plugins.PedrolloPL/`

**Migrations:**
- Latest: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Migrations/20251201165810_RecordHistory.cs`

**UI Components:**
- Pages: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Pages/`
- Components: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Components/`
- Services: `/Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Services/`