DiunaBI/.claude/project-context.md

DiunaBI Project Context

This file is auto-generated for Claude Code to quickly understand the project structure. Last updated: 2025-12-08

RECENT CHANGES (This Session)

Jobs List Sorting and Multi-Select Filtering (Dec 8, 2025):

• ✅ Fixed Job Sorting - Changed from single CreatedAt DESC to Priority ASC → JobType → CreatedAt DESC
• ✅ Multi-Select Status Filter - Replaced single status dropdown with multi-select supporting multiple JobStatus values
• ✅ Auto-Refresh on Filter Change - Filters now automatically trigger data reload without requiring manual button click
• ✅ API Updates - JobsController GetAll endpoint accepts List<JobStatus>? statuses instead of single status
• ✅ JobService Updates - Sends status values as integers in query string for multi-select support
• Files modified: JobsController.cs, JobService.cs, Index.razor, Index.razor.cs
• Status: Jobs list now sortable by priority/type/date with working multi-select filters

User Timezone Support (Dec 8, 2025):

• ✅ DateTimeHelper Service - Created JS Interop service to detect user's browser timezone
• ✅ UTC to Local Conversion - All date displays now show user's local timezone instead of UTC
• ✅ Database Consistency - Database continues to store UTC (correct), conversion only for display
• ✅ Updated Pages - Applied timezone conversion to all date fields in:
  • Jobs Index and Details pages
  • Layers Details page (CreatedAt, ModifiedAt, record history)
  • DataInbox Index page
• ✅ Service Registration - Registered DateTimeHelper as scoped service in DI container
• Files created: DateTimeHelper.cs
• Files modified: ServiceCollectionExtensions.cs, Jobs/Index.razor.cs, Jobs/Details.razor, Layers/Details.razor, Layers/Details.razor.cs, DataInbox/Index.razor.cs
• Status: All dates display in user's local timezone with format "yyyy-MM-dd HH:mm:ss"

QueueJob Model Cleanup and AutoImport User (Dec 8, 2025):

• ✅ Removed Duplicate Fields - Removed CreatedAtUtc and ModifiedAtUtc from QueueJob (were duplicates of CreatedAt/ModifiedAt)
• ✅ Added ModifiedAt Field - Was missing, now tracks job modification timestamp
• ✅ AutoImport User ID - Created User.AutoImportUserId constant: f392209e-123e-4651-a5a4-0b1d6cf9ff9d
• ✅ System Operations - All system-created/modified jobs now use AutoImportUserId for CreatedById and ModifiedById
• ✅ Database Migration - Created migration: RemoveQueueJobDuplicateUTCFields
• Files modified: QueueJob.cs, User.cs, JobWorkerService.cs, JobSchedulerService.cs, AppDbContext.cs, JobsController.cs
• Files created: 20251208205202_RemoveQueueJobDuplicateUTCFields.cs
• Status: QueueJob model cleaned up, all automated operations tracked with AutoImport user ID

Job Scheduling UI with JWT Authorization (Dec 8, 2025):

• ✅ New JWT Endpoints - Created UI-specific endpoints at /jobs/ui/schedule/* with JWT authorization (parallel to API key endpoints)
• ✅ Three Scheduling Options - MudMenu dropdown in Jobs Index with:
  • Run All Jobs - schedules all import and process jobs
  • Run All Imports - schedules import jobs only
  • Run All Processes - schedules process jobs only
• ✅ JobService Methods - Added three scheduling methods returning (success, jobsCreated, message) tuples
• ✅ Auto-Refresh - Jobs list automatically reloads after scheduling with success/failure notifications
• ✅ Dual Authorization - Existing /jobs/schedule/{apiKey} endpoints for automation, new /jobs/ui/schedule endpoints for UI users
• Files modified: JobsController.cs, JobService.cs, Index.razor, Index.razor.cs
• Status: UI users can now schedule jobs directly from Jobs page using JWT authentication

---

API Key Authorization Fix for Cron Jobs (Dec 6, 2025):

• ✅ Fixed 401 Unauthorized on API Key Endpoints - Cron jobs calling /jobs/schedule endpoints were getting rejected despite valid API keys
• ✅ Added [AllowAnonymous] Attribute - Bypasses controller-level [Authorize] to allow [ApiKeyAuth] filter to handle authorization
• ✅ Three Endpoints Fixed - Applied fix to all job scheduling endpoints:
  • POST /jobs/schedule - Schedule all jobs (imports + processes)
  • POST /jobs/schedule/imports - Schedule import jobs only
  • POST /jobs/schedule/processes - Schedule process jobs only
• Root cause: Controller-level [Authorize] attribute required JWT Bearer auth for all endpoints, blocking API key authentication
• Solution: Add [AllowAnonymous] to allow [ApiKeyAuth] filter to validate X-API-Key header
• Files modified: JobsController.cs
• Status: Cron jobs can now authenticate with API key via X-API-Key header

SignalR Authentication Token Flow Fix (Dec 6, 2025):

• ✅ TokenProvider Population - Fixed TokenProvider.Token never being set with JWT, causing 401 Unauthorized on SignalR connections
• ✅ AuthService Token Management - Injected TokenProvider into AuthService and set token in 3 key places:
  • ValidateWithBackendAsync() - on fresh Google login
  • CheckAuthenticationAsync() - on session restore from localStorage
  • ClearAuthenticationAsync() - clear token on logout
• ✅ SignalR Initialization Timing - Moved SignalR initialization from MainLayout.OnInitializedAsync to after authentication completes
• ✅ Event-Driven Architecture - MainLayout now subscribes to AuthenticationStateChanged event to initialize SignalR when user authenticates
• ✅ Session Restore Support - CheckAuthenticationAsync() now fires AuthenticationStateChanged event to initialize SignalR on page refresh
• Root cause: SignalR was initialized before authentication, so JWT token was empty during connection setup
• Solution: Initialize SignalR only after token is available via event subscription
• Files modified: AuthService.cs, MainLayout.razor
• Status: SignalR authentication working for both fresh login and restored sessions

SignalR Authentication DI Fix (Dec 6, 2025):

• ✅ TokenProvider Registration - Added missing TokenProvider service registration in DI container
• ✅ EntityChangeHubService Scope Fix - Changed from singleton to scoped to support user-specific JWT tokens
• ✅ Bug Fix - Resolved InvalidOperationException preventing app from starting after SignalR authentication was added
• Root cause: Singleton service (EntityChangeHubService) cannot depend on scoped service (TokenProvider) in DI
• Solution: Made EntityChangeHubService scoped so each user session has its own authenticated SignalR connection
• Files modified: ServiceCollectionExtensions.cs

---

Security Audit & Hardening (Dec 5, 2025):

• ✅ JWT Token Validation - Enabled issuer/audience validation in Program.cs, fixed config key mismatch in JwtTokenService.cs
• ✅ API Key Security - Created ApiKeyAuthAttribute.cs with X-API-Key header auth, constant-time comparison
• ✅ Job Endpoints - Migrated 3 job scheduling endpoints in JobsController.cs from URL-based to header-based API keys
• ✅ Stack Trace Exposure - Fixed 20 instances across 3 controllers (JobsController.cs, LayersController.cs, DataInboxController.cs) - now returns generic error messages
• ✅ SignalR Authentication - Added [Authorize] to EntityChangeHub.cs, configured JWT token in EntityChangeHubService.cs
• ✅ Rate Limiting - Implemented ASP.NET Core rate limiting: 100 req/min general, 10 req/min auth in Program.cs
• ✅ Security Headers - Added XSS, clickjacking, MIME sniffing protection middleware in Program.cs
• ✅ Input Validation - Added pagination limits (1-1000) to GetAll endpoints in 3 controllers
• ✅ User Enumeration - Fixed generic auth error in GoogleAuthService.cs
• ✅ Sensitive Data Logging - Made conditional on development only in Program.cs
• ✅ Base64 Size Limit - Added 10MB limit to DataInbox in DataInboxController.cs
• Files modified: 12 files (API: Program.cs, 4 controllers, 3 services, 1 hub, 1 new attribute; UI: EntityChangeHubService.cs, ServiceCollectionExtensions.cs)
• Security status: 5/5 CRITICAL fixed, 3/3 HIGH fixed, 4/4 MEDIUM fixed

Seq Removal - Logging Cleanup (Dec 5, 2025):

• ✅ Removed Seq logging sink to eliminate commercial licensing concerns
• ✅ Removed Serilog.Sinks.Seq NuGet package from DiunaBI.API.csproj
• ✅ Removed Seq sink configuration from appsettings.Development.json
• ✅ Kept Serilog (free, open-source) with Console + File sinks for production-ready logging
• ✅ Build verified - no errors after Seq removal
• Files modified: DiunaBI.API.csproj, appsettings.Development.json
• Manual step required: Remove seq service from docker-compose.yml and add Docker log rotation config

UI Reorganization (Dec 5, 2025):

• ✅ Moved pages to feature-based folders: Pages/Layers/, Pages/Jobs/, Pages/DataInbox/
• ✅ Organized components: Components/Layout/ (MainLayout, EmptyLayout, Routes), Components/Auth/ (AuthGuard, LoginCard)
• ✅ Removed obsolete wrapper files (LayerListPage, JobListPage, DataInboxListPage, etc.)
• ✅ Removed duplicate component files (LayerListComponent, JobListComponent, DataInboxListComponent)
• ✅ Standardized code-behind: .razor.cs for complex logic, inline @code for simple pages
• ✅ Updated _Imports.razor with new namespaces: DiunaBI.UI.Shared.Components.Layout, DiunaBI.UI.Shared.Components.Auth
• ✅ All routes unchanged - backward compatible

---

PROJECT TYPE & TECH STACK

Application Type: Full-stack Business Intelligence (BI) platform with multi-tier architecture, real-time capabilities, and plugin system

Core Stack:

• Backend: ASP.NET Core 10.0 Web API
• Frontend: Blazor Server + MAUI Mobile
• Database: SQL Server + EF Core 10.0
• UI: MudBlazor 8.0
• Real-time: SignalR (EntityChangeHub)
• Google: Sheets API, Drive API, OAuth
• Logging: Serilog (Console, File)
• Auth: JWT Bearer + Google OAuth

---

SOLUTION STRUCTURE (10 Projects)

DiunaBI.API (Web API)
├── Controllers: Auth, Layers, Jobs, DataInbox
├── Hubs: EntityChangeHub (SignalR real-time updates)
└── Services: GoogleAuth, JwtToken

DiunaBI.Domain (Entities)
└── User, Layer, Record, RecordHistory, QueueJob, DataInbox, ProcessSource

DiunaBI.Application (DTOs)
└── LayerDto, RecordDto, UserDto, RecordHistoryDto, PagedResult, JobDto

DiunaBI.Infrastructure (Data + Services)
├── Data: AppDbContext, Migrations (47 total)
├── Interceptors: EntityChangeInterceptor (auto-broadcasts DB changes)
├── Services: PluginManager, JobScheduler, JobWorker, GoogleSheets/Drive
├── Plugins: BaseDataImporter, BaseDataProcessor, BaseDataExporter
└── Interfaces: IPlugin, IDataProcessor, IDataImporter, IDataExporter

DiunaBI.UI.Web (Blazor Server)
└── Server-side Blazor web application

DiunaBI.UI.Mobile (MAUI)
└── iOS, Android, Windows, macOS support

DiunaBI.UI.Shared (Blazor Component Library - Reorganized)
├── Pages/
│   ├── Layers/ (Index.razor, Details.razor)
│   ├── Jobs/ (Index.razor, Details.razor)
│   ├── DataInbox/ (Index.razor, Details.razor)
│   ├── Dashboard.razor, Login.razor, Index.razor
├── Components/
│   ├── Layout/ (MainLayout, EmptyLayout, Routes)
│   └── Auth/ (AuthGuard, LoginCard)
└── Services/
    ├── LayerService, JobService, DataInboxService
    ├── EntityChangeHubService (SignalR client)
    ├── FilterStateServices (remember filters)
    └── AuthService, TokenProvider

DiunaBI.Plugins.Morska (Feature Plugin)
├── Importers: Standard, D1, D3, FK2 (4 total)
├── Processors: D6, T1, T3, T4, T5 variants (12 total)
└── Exporters: Google Sheets export (1)

DiunaBI.Plugins.PedrolloPL (Feature Plugin - NEW)
└── Importers: B3 (1 total)

DiunaBI.Tests (Testing)
└── Unit and integration tests

---

CORE FUNCTIONALITY

Purpose: BI platform for data import, processing, transformation via modular plugin architecture. Multi-layer workflows with audit trails, real-time notifications, scheduled job processing.

Main Features:

1. Layer Management - 4 types (Import/Processed/Admin/Dictionary), parent-child relationships, soft deletes
2. Data Records - 32 numeric columns (Value1-32) + description, hierarchical, full audit trail
3. Plugin Architecture - Dynamic assembly loading, base classes in Infrastructure, 3 types (Importers/Processors/Exporters)
4. Job Queue System - Background worker with retry logic (30s → 2m → 5m), priority-based, auto-scheduling
5. External Data - DataInbox API, Google Sheets read/write, Google Drive integration
6. Real-time Updates - SignalR broadcasts entity changes (create/update/delete) to all connected clients
7. Audit Trail - RecordHistory tracks all record changes with field-level diffs and JSON summaries
8. Filter Persistence - UI filter states saved across sessions (LayerFilterStateService, DataInboxFilterStateService)

---

KEY ENTITIES

Layer

• Id, Number, Name, Type (Import/Processed/Administration/Dictionary)
• CreatedAt/ModifiedAt, CreatedBy/ModifiedBy (with user relations)
• IsDeleted (soft delete), IsCancelled (processing control), ParentId
• Relations: Records (1-to-many), ProcessSources (1-to-many)

Record

• Id, Code (unique identifier), LayerId
• Value1-Value32 (double?), Desc1 (string, max 10000 chars)
• CreatedAt/ModifiedAt, CreatedBy/ModifiedBy, IsDeleted
• Audit: Full history tracked in RecordHistory table

RecordHistory (NEW - Migration 47)

• RecordId, LayerId, ChangedAt, ChangedById
• ChangeType (Created/Updated/Deleted)
• Code, Desc1 (snapshot at time of change)
• ChangedFields (comma-separated field names)
• ChangesSummary (JSON with old/new values)
• Indexes: (RecordId, ChangedAt), (LayerId, ChangedAt) for performance

QueueJob

• LayerId, LayerName, PluginName
• JobType (Import/Process)
• Priority (0 = highest), Status (Pending/Running/Completed/Failed/Retrying)
• RetryCount, MaxRetries (default 5)
• CreatedAt, LastAttemptAt, CompletedAt
• LastError (detailed error message)

DataInbox

• Id, Name, Source (identifiers)
• Data (base64-encoded JSON array)
• CreatedAt
• Used by importers to stage incoming data

User

• Id (Guid), Email, UserName
• CreatedAt, LastLoginAt
• Google OAuth identity

ProcessSource

• Id, SourceLayerId, TargetLayerId
• Defines layer processing relationships

---

API ENDPOINTS

Base: / (ApiController routes)

AuthController (/auth)

• POST /auth/apiToken - Exchange Google ID token for JWT (AllowAnonymous)
• POST /auth/refresh - Refresh expired JWT token

LayersController (/layers)

• GET /layers?page=1&pageSize=10&search=&type= - List layers (paged, filterable)
• GET /layers/{id} - Get layer details with records
• POST /layers - Create new layer
• PUT /layers/{id} - Update layer
• DELETE /layers/{id} - Soft delete layer
• POST /layers/{id}/records - Add/update records
• PUT /layers/{layerId}/records/{recordId} - Update specific record
• DELETE /layers/{layerId}/records/{recordId} - Delete record
• GET /layers/{layerId}/records/{recordId}/history - Get record history
• GET /layers/{layerId}/deleted-records - Get deleted records with history

JobsController (/jobs) - NEW

• GET /jobs?page=1&pageSize=50&status=&jobType= - List jobs (paged, filterable)
• GET /jobs/{id} - Get job details
• GET /jobs/stats - Get job statistics (counts by status)
• POST /jobs/schedule/{apiKey} - Schedule all jobs from layer configs
• POST /jobs/schedule/imports/{apiKey} - Schedule import jobs only
• POST /jobs/schedule/processes/{apiKey} - Schedule process jobs only
• POST /jobs/create-for-layer/{layerId} - Create job for specific layer (manual trigger)
• POST /jobs/{id}/retry - Retry failed job (resets to Pending)
• DELETE /jobs/{id} - Cancel pending/retrying job

DataInboxController (/datainbox)

• GET /datainbox?page=1&pageSize=10&search= - List inbox items (paged, filterable)
• GET /datainbox/{id} - Get inbox item with decoded data
• POST /datainbox - Create inbox item
• PUT /datainbox/Add/{apiKey} - Add data (API key + Basic Auth)
• DELETE /datainbox/{id} - Delete inbox item

SignalR Hub

• /hubs/entitychanges - SignalR hub for real-time entity change notifications
  • Event: EntityChanged(module, id, operation) - broadcasts to all clients
  • Modules: QueueJobs, Layers, Records, RecordHistory

---

AUTHENTICATION & SECURITY

Flow:

1. Client exchanges Google ID token → /auth/apiToken
2. GoogleAuthService validates token with Google, maps to internal User
3. Returns JWT (7-day expiration, HS256 signing)
4. JWT required on all protected endpoints (except /auth/apiToken, /health)
5. UserId extraction middleware sets X-UserId header for audit trails

Security:

• Google OAuth 2.0 for identity verification
• JWT Bearer tokens for API access
• API key + Basic Auth for DataInbox external endpoints
• CORS configured for:
  • http://localhost:4200
  • https://diuna.bim-it.pl
  • https://morska.diunabi.com

---

KEY SERVICES

Infrastructure Services

PluginManager

• Location: DiunaBI.Infrastructure/Services/PluginManager.cs
• Loads plugin assemblies from bin/Plugins/ directory at startup
• Registers IDataProcessor, IDataImporter, IDataExporter implementations
• Provides plugin discovery and execution

JobSchedulerService

• Location: DiunaBI.Infrastructure/Services/JobSchedulerService.cs
• Creates QueueJob entries from Administration layer configs
• Reads layer.Records with Code="Plugin", Code="Priority", Code="MaxRetries"
• Methods: ScheduleImportJobsAsync, ScheduleProcessJobsAsync, ScheduleAllJobsAsync

JobWorkerService (BackgroundService)

• Location: DiunaBI.Infrastructure/Services/JobWorkerService.cs
• Polls QueueJobs table every 10 seconds
• Executes jobs via PluginManager (Import/Process)
• Retry logic with exponential backoff: 30s → 2m → 5m delays
• Rate limiting: 5-second delay after imports (Google Sheets API quota)
• Updates job status in real-time (triggers SignalR broadcasts)

EntityChangeInterceptor

• Location: DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs
• EF Core SaveChangesInterceptor
• Captures entity changes: Added, Modified, Deleted
• Broadcasts changes via SignalR EntityChangeHub after successful save
• Uses reflection to avoid circular dependencies with IHubContext

GoogleSheetsHelper

• Location: DiunaBI.Infrastructure/Helpers/GoogleSheetsHelper.cs
• Google Sheets API v4 integration
• Methods: ReadRange, WriteRange, CreateSpreadsheet, UpdateSpreadsheet

GoogleDriveHelper

• Location: DiunaBI.Infrastructure/Helpers/GoogleDriveHelper.cs
• Google Drive API v3 integration
• Methods: UploadFile, ListFiles, MoveFile

GoogleAuthService / JwtTokenService

• Authentication and token management
• JWT generation and validation

UI Services

EntityChangeHubService

• Location: DiunaBI.UI.Shared/Services/EntityChangeHubService.cs
• Singleton service for SignalR client connection
• Auto-reconnect enabled
• Event: EntityChanged - UI components subscribe for real-time updates
• Initialized in MainLayout.OnInitializedAsync

LayerService / JobService / DataInboxService

• HTTP clients for API communication
• DTOs serialization/deserialization
• Paged result handling

LayerFilterStateService / DataInboxFilterStateService

• Persist filter state across navigation
• Singleton services remember search, type, page selections

---

DATABASE SCHEMA

Total Migrations: 47

Latest Migrations:

Migration 47: RecordHistory (Dec 1, 2025)

• NEW Table: RecordHistory
• Tracks all record changes (Created, Updated, Deleted)
• Fields: Id, RecordId, LayerId, ChangedAt, ChangedById, ChangeType, Code, Desc1, ChangedFields, ChangesSummary
• Indexes: IX_RecordHistory_RecordId_ChangedAt, IX_RecordHistory_LayerId_ChangedAt
• Foreign key: RecordHistory.ChangedById → Users.Id

Migration 46: FixLayerDefaultValues (Nov 20, 2025)

• Set default value: Layers.IsDeleted = false

Migration 45: UpdateModel (Nov 19, 2025)

• Added GETUTCDATE() defaults for all timestamp fields
• Changed foreign key constraints from CASCADE to RESTRICT:
  • Layers → Users (CreatedById, ModifiedById)
  • Records → Users (CreatedById, ModifiedById)
• Added FK_ProcessSources_Layers_LayerId

Core Tables:

• Users (authentication, audit)
• Layers (4 types, soft deletes, parent-child)
• Records (32 Value fields + Desc1, audit, soft deletes)
• RecordHistory (change tracking, field diffs, JSON summaries)
• QueueJobs (job queue, retry logic, status tracking)
• DataInbox (incoming data staging, base64 encoded)
• ProcessSources (layer relationships)

---

PLUGIN SYSTEM

Base Classes (Infrastructure/Plugins/)

BaseDataImporter (DiunaBI.Infrastructure/Plugins/BaseDataImporter.cs)

• Abstract base for all importers
• Methods: ImportAsync(layerId, jobId), ValidateConfiguration()
• Access: AppDbContext, PluginManager, GoogleSheetsHelper, GoogleDriveHelper

BaseDataProcessor (DiunaBI.Infrastructure/Plugins/BaseDataProcessor.cs)

• Abstract base for all processors
• Methods: ProcessAsync(layerId, jobId), ValidateConfiguration()
• Access: AppDbContext, PluginManager

BaseDataExporter (DiunaBI.Infrastructure/Plugins/BaseDataExporter.cs)

• Abstract base for all exporters
• Methods: ExportAsync(layerId, jobId), ValidateConfiguration()
• Access: AppDbContext, GoogleSheetsHelper, GoogleDriveHelper

Morska Plugin (DiunaBI.Plugins.Morska)

Importers (4):

• MorskaStandardImporter - Generic CSV/Excel import
• MorskaD1Importer - D1 data format
• MorskaD3Importer - D3 data format
• MorskaFK2Importer - FK2 data format

Processors (12):

• MorskaD6Processor
• MorskaT1R1Processor
• MorskaT1R3Processor
• MorskaT3SingleSourceProcessor
• MorskaT3SourceYearSummaryProcessor
• MorskaT3MultiSourceSummaryProcessor
• MorskaT3MultiSourceYearSummaryProcessor
• MorskaT4R2Processor
• MorskaT4SingleSourceProcessor
• MorskaT5LastValuesProcessor
• MorskaT3MultiSourceCopySelectedCodesProcessor-TO_REMOVE (deprecated)
• MorskaT3MultiSourceCopySelectedCodesYearSummaryProcessor-TO_REMOVE (deprecated)

Exporters (1):

• googleSheet.export.cs - Google Sheets export

Total: ~6,566 lines of code

PedrolloPL Plugin (DiunaBI.Plugins.PedrolloPL) - NEW

Importers (1):

• PedrolloPLImportB3 (DiunaBI.Plugins.PedrolloPL/Importers/PedrolloPLImportB3.cs)
  • Imports B3 data from DataInbox
  • Uses L1-D-B3-CODES dictionary layer for region code mapping
  • Creates 12 monthly records per region (Value1-Value12)
  • Generates Import layers: L{Number}-I-B3-{Year}-{Timestamp}
  • Handles base64 JSON data decoding

---

UI STRUCTURE (DiunaBI.UI.Shared)

Reorganized Structure (Dec 5, 2025)

Pages/ (Routable pages with @page directive)

Pages/
├── Layers/
│   ├── Index.razor + Index.razor.cs    - /layers (list with filters, pagination)
│   └── Details.razor + Details.razor.cs - /layers/{id} (detail, edit, history)
├── Jobs/
│   ├── Index.razor + Index.razor.cs    - /jobs (list with filters, real-time updates)
│   └── Details.razor                    - /jobs/{id} (detail, retry, cancel, real-time)
├── DataInbox/
│   ├── Index.razor + Index.razor.cs    - /datainbox (list with filters)
│   └── Details.razor + Details.razor.cs - /datainbox/{id} (detail, base64 decode)
├── Dashboard.razor                      - /dashboard (user info)
├── Login.razor                          - /login (Google OAuth)
└── Index.razor                          - / (redirects to /dashboard)

Components/ (Reusable components, no routes)

Components/
├── Layout/
│   ├── MainLayout.razor     - Main app layout with drawer, nav menu
│   ├── EmptyLayout.razor    - Minimal layout for login page
│   └── Routes.razor         - Router configuration
└── Auth/
    ├── AuthGuard.razor      - Authentication guard wrapper
    └── LoginCard.razor      - Google login button component

Navigation Menu:

• Dashboard (/dashboard) - User profile
• Layers (/layers) - Layer management
• Data Inbox (/datainbox) - Incoming data review
• Jobs (/jobs) - Job queue monitoring (with real-time status updates)

Code-Behind Pattern:

• Complex pages (50+ lines logic): Separate .razor.cs files
• Simple pages: Inline @code blocks
• Namespaces: DiunaBI.UI.Shared.Pages.{Feature}

---

REAL-TIME FEATURES (SignalR)

Architecture

Hub: DiunaBI.API/Hubs/EntityChangeHub.cs

• Endpoint: /hubs/entitychanges
• Method: SendEntityChange(string module, string id, string operation)
• Broadcasts: EntityChanged event to all connected clients

Interceptor: DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs

• EF Core SaveChangesInterceptor
• Detects: Added, Modified, Deleted entities
• Broadcasts: After successful SaveChanges
• Modules: QueueJobs, Layers, Records, RecordHistory

UI Service: DiunaBI.UI.Shared/Services/EntityChangeHubService.cs

• Singleton initialized in MainLayout
• Auto-reconnect enabled
• Components subscribe: HubService.EntityChanged += OnEntityChanged

Real-time Update Flow

1. User action → API endpoint
2. DbContext.SaveChangesAsync()
3. EntityChangeInterceptor captures changes
4. SignalR broadcast to all clients: EntityChanged(module, id, operation)
5. UI components receive event and refresh data
6. StateHasChanged() updates UI

Example: Job status changes appear instantly on JobDetailPage and JobListPage

---

JOB QUEUE SYSTEM

Components

Entity: QueueJob (DiunaBI.Domain/Entities/QueueJob.cs)

• JobType: Import, Process
• JobStatus: Pending, Running, Completed, Failed, Retrying
• Priority: 0 = highest priority
• Retry: 30s → 2m → 5m delays, max 5 attempts

Scheduler: JobSchedulerService

• Reads Administration layer configs (Type=ImportWorker/ProcessWorker)
• Auto-creates jobs based on layer.Records configuration
• API endpoints: /jobs/schedule/{apiKey}, /jobs/schedule/imports/{apiKey}, /jobs/schedule/processes/{apiKey}

Worker: JobWorkerService (BackgroundService)

• Polls every 10 seconds
• Executes via PluginManager
• Exponential backoff on failures
• Rate limiting for Google API quota
• Real-time status updates via SignalR

UI: Pages/Jobs/

• Index.razor - Job list with filters, real-time updates
• Details.razor - Job detail with retry/cancel, real-time status

Job Lifecycle

1. Creation - JobSchedulerService or manual via API
2. Queued - Status: Pending, sorted by Priority
3. Execution - JobWorkerService picks up, Status: Running
4. Completion - Status: Completed or Failed
5. Retry - On failure, Status: Retrying with exponential backoff
6. Real-time - All status changes broadcast via SignalR

Statistics Endpoint: GET /jobs/stats

{
  "pending": 5,
  "running": 2,
  "completed": 150,
  "failed": 3,
  "retrying": 1,
  "total": 161
}

---

RECENT DEVELOPMENT

Recent Commits (Dec 2-5, 2025):

• 193127b: SignalR for realtime entitychanges (Dec 4)
• bf2beda, 942da18: Build fixes (Dec 4)
• a3fa8f9: B3 import is working (Dec 4)
• 0e3b393: WIP: b3 plugin (Dec 3)
• 445c07a: Morska plugins refactor (Dec 2)
• 3f8e62f: WIP: queue engine (Dec 2)
• 248106a: Plugins little refactor (Dec 2)
• 587d4d6: Pedrollo plugins (Dec 2)
• e70a8dd: Remember list filters (Dec 2)
• 89859cd: Record history is working (Dec 1)

Development Focus (Last 30 Days):

1. ✅ Real-time updates (SignalR integration)
2. ✅ Job queue system (background worker, retry logic)
3. ✅ PedrolloPL plugin (B3 importer)
4. ✅ Record history tracking (audit trail)
5. ✅ UI reorganization (feature-based folders)
6. ✅ Plugin refactoring (base classes in Infrastructure)
7. ✅ Filter persistence (UI state management)

Major Features Added:

• SignalR real-time entity change notifications
• Background job processing with retry logic
• Record history with field-level diffs
• PedrolloPL B3 data importer
• UI reorganization (Pages/Layers, Pages/Jobs, Pages/DataInbox)
• Filter state persistence across sessions

---

CONFIGURATION

Key Settings (appsettings.Development.json):

• ConnectionStrings:SQLDatabase - SQL Server (localhost:21433, DB: DiunaBI-PedrolloPL)
• JwtSettings:SecurityKey, ExpiryDays (7)
• GoogleAuth:ClientId, RedirectUri
• apiKey, apiUser, apiPass - DataInbox API security
• exportDirectory - Google Drive folder ID for exports
• apiLocalUrl - localhost:5400
• InstanceName - DEV/PROD environment identifier

Logging Configuration:

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft.AspNetCore": "Warning",
      "Microsoft.EntityFrameworkCore.Database.Command": "Warning",
      "Microsoft.EntityFrameworkCore.Infrastructure": "Warning",
      "System.Net.Http.HttpClient": "Warning",
      "Google.Apis": "Warning",
      "DiunaBI.Core.Services.PluginManager": "Information"
    }
  }
}

CORS Origins:

• http://localhost:4200 (development)
• https://diuna.bim-it.pl (production)
• https://morska.diunabi.com (production)

---

PATTERNS & ARCHITECTURE

Design Patterns:

• Clean Architecture (Domain → Application → Infrastructure → API)
• Plugin Pattern (dynamic loading, base classes, interface contracts)
• Interceptor Pattern (EF Core SaveChangesInterceptor for change tracking)
• Hub Pattern (SignalR for real-time notifications)
• Service Pattern (dependency injection throughout)
• Repository Pattern (EF Core DbContext as repository)
• Background Service Pattern (JobWorkerService for async processing)

Tech Versions:

• .NET 10.0 (upgraded from .NET 8.0)
• EF Core 10.0
• C# 13.0
• Blazor Server (net10.0)
• MAUI (net10.0-ios/android/windows/macos)
• MudBlazor 8.0

Architectural Decisions:

• Plugin base classes in Infrastructure for reusability
• SignalR for real-time updates (no polling)
• Background service for job processing (no external scheduler)
• Soft deletes with audit trails
• Foreign key RESTRICT to prevent accidental cascades
• Feature-based folder structure in UI

---

QUICK REFERENCE

Database:

• SQL Server with 47 EF Core migrations
• Auto-timestamps via GETUTCDATE() defaults
• Soft deletes (IsDeleted flag)
• Audit trails (CreatedBy, ModifiedBy, RecordHistory table)

Build Process:

• MSBuild target copies plugin DLLs to bin/Plugins/ after build
• Plugins: DiunaBI.Plugins.Morska.dll, DiunaBI.Plugins.PedrolloPL.dll

SignalR:

• Hub: /hubs/entitychanges
• Broadcasts: EntityChanged(module, id, operation)
• Auto-reconnect enabled in UI
• Real-time updates for QueueJobs, Layers, Records

Job Queue:

• Auto-scheduling from layer configs (Type=ImportWorker/ProcessWorker)
• Background processing every 10 seconds
• Retry logic: 30s → 2m → 5m (max 5 retries)
• Priority-based execution (0 = highest)
• Real-time status updates via SignalR

Plugins:

• Morska: 4 importers, 12 processors, 1 exporter (~6,566 LOC)
• PedrolloPL: 1 importer (B3 data)
• Base classes: BaseDataImporter, BaseDataProcessor, BaseDataExporter
• Dynamic loading from bin/Plugins/ at startup

UI Structure:

• Feature-based folders: Pages/Layers, Pages/Jobs, Pages/DataInbox
• Separate code-behind for complex logic (.razor.cs files)
• Inline @code for simple pages
• Organized components: Layout/, Auth/
• Filter state persistence across navigation

---

FILE PATHS REFERENCE

Key Configuration:

• API: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/appsettings.json
• API Startup: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Program.cs

SignalR:

• Hub: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Hubs/EntityChangeHub.cs
• Interceptor: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Interceptors/EntityChangeInterceptor.cs
• UI Service: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Services/EntityChangeHubService.cs

Job System:

• Controller: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.API/Controllers/JobsController.cs
• Scheduler: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Services/JobSchedulerService.cs
• Worker: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Services/JobWorkerService.cs
• UI Pages: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Pages/Jobs/

Plugins:

• Base Classes: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Plugins/
• Morska: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Plugins.Morska/
• PedrolloPL: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Plugins.PedrolloPL/

Migrations:

• Latest: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.Infrastructure/Migrations/20251201165810_RecordHistory.cs

UI Components:

• Pages: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Pages/
• Components: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Components/
• Services: /Users/mz/Projects/Diuna/DiunaBI/DiunaBI.UI.Shared/Services/