Compare commits
19 Commits
595076033b
...
AIValidato
| Author | SHA1 | Date | |
|---|---|---|---|
| f10dfe629e | |||
| 096ff5573e | |||
| b17fdae56e | |||
| 972feb2d8c | |||
| e48cc7e1f9 | |||
| 16eb688607 | |||
| 2132c130a3 | |||
| dffbc31432 | |||
| 151ecaa98f | |||
| b917aa5077 | |||
| 24f5f91704 | |||
| 00c9584d03 | |||
| c94a3b41c9 | |||
| e25cdc4441 | |||
| 1f95d57717 | |||
| d2fb9b8071 | |||
| 08abd96751 | |||
| eb570679ba | |||
| 8713ed9686 |
@@ -1,10 +1,136 @@
|
|||||||
# DiunaBI Project Context
|
# DiunaBI Project Context
|
||||||
|
|
||||||
> This file is auto-generated for Claude Code to quickly understand the project structure.
|
> This file is auto-generated for Claude Code to quickly understand the project structure.
|
||||||
> Last updated: 2025-12-05
|
> Last updated: 2025-12-08
|
||||||
|
|
||||||
## RECENT CHANGES (This Session)
|
## RECENT CHANGES (This Session)
|
||||||
|
|
||||||
|
**SignalR Real-Time Updates & UI Consistency (Dec 8, 2025):**
|
||||||
|
- ✅ **Removed Manual Refresh Button** - Removed refresh button from Jobs/Index.razor (SignalR auto-refresh eliminates need)
|
||||||
|
- ✅ **SignalR on Layers List** - Added real-time updates to Layers/Index with EntityChangeHubService subscription
|
||||||
|
- ✅ **SignalR on DataInbox List** - Added real-time updates to DataInbox/Index with EntityChangeHubService subscription
|
||||||
|
- ✅ **SignalR on Layer Details** - Added real-time updates to Layers/Details for both layer and record changes
|
||||||
|
- ✅ **Consistent UI Behavior** - All lists now have uniform SignalR-based real-time updates
|
||||||
|
- ✅ **Proper Cleanup** - Implemented IDisposable pattern to unsubscribe from SignalR events on all pages
|
||||||
|
- ✅ **Jobs Sorting Fix** - Changed sorting from Priority→JobType→CreatedAt DESC to CreatedAt DESC→Priority ASC (newest jobs first, then by priority)
|
||||||
|
- ✅ **Faster Job Processing** - Reduced JobWorkerService poll interval from 10 seconds to 5 seconds
|
||||||
|
- Files modified:
|
||||||
|
- [Jobs/Index.razor](DiunaBI.UI.Shared/Pages/Jobs/Index.razor) - removed refresh button
|
||||||
|
- [Layers/Index.razor](DiunaBI.UI.Shared/Pages/Layers/Index.razor), [Layers/Index.razor.cs](DiunaBI.UI.Shared/Pages/Layers/Index.razor.cs) - added SignalR + IDisposable
|
||||||
|
- [DataInbox/Index.razor](DiunaBI.UI.Shared/Pages/DataInbox/Index.razor), [DataInbox/Index.razor.cs](DiunaBI.UI.Shared/Pages/DataInbox/Index.razor.cs) - added SignalR + IDisposable
|
||||||
|
- [Layers/Details.razor](DiunaBI.UI.Shared/Pages/Layers/Details.razor), [Layers/Details.razor.cs](DiunaBI.UI.Shared/Pages/Layers/Details.razor.cs) - added SignalR + IDisposable
|
||||||
|
- [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs) - fixed sorting logic
|
||||||
|
- [JobWorkerService.cs](DiunaBI.Infrastructure/Services/JobWorkerService.cs) - reduced poll interval to 5 seconds
|
||||||
|
- Status: All lists have consistent real-time behavior, no manual refresh needed, jobs sorted by date first
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Job Scheduler Race Condition Fix (Dec 8, 2025):**
|
||||||
|
- ✅ **In-Memory Deduplication** - Added `HashSet<Guid>` to track LayerIds scheduled within the same batch
|
||||||
|
- ✅ **Prevents Duplicate Jobs** - Fixed race condition where same layer could be scheduled multiple times during single "Run All Jobs" operation
|
||||||
|
- ✅ **Two-Level Protection** - In-memory check (HashSet) runs before database check for O(1) performance
|
||||||
|
- ✅ **Applied to Both Methods** - Fixed both ScheduleImportJobsAsync and ScheduleProcessJobsAsync
|
||||||
|
- ✅ **Better Logging** - Added debug log message "Job already scheduled in this batch" for transparency
|
||||||
|
- Root cause: When multiple layers had same ID in query results or import plugins created new layers during scheduling loop, database check couldn't detect duplicates added in same batch before SaveChangesAsync()
|
||||||
|
- Solution: Track scheduled LayerIds in HashSet during loop iteration to prevent within-batch duplicates
|
||||||
|
- Files modified: [JobSchedulerService.cs](DiunaBI.Infrastructure/Services/JobSchedulerService.cs)
|
||||||
|
- Status: Race condition resolved, duplicate job creation prevented
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Blazor Server Reconnection UI Customization (Dec 8, 2025):**
|
||||||
|
- ✅ **Custom Reconnection Modal** - Replaced default Blazor "Rejoin failed..." dialog with custom-styled modal
|
||||||
|
- ✅ **Theme-Matched Styling** - Changed loader and button colors from blue to app's primary red (#e7163d) matching navbar
|
||||||
|
- ✅ **Timer with Elapsed Seconds** - Added real-time timer showing elapsed reconnection time (0s, 1s, 2s...)
|
||||||
|
- ✅ **CSS Classes Integration** - Used Blazor's built-in `.components-reconnect-show/failed/rejected` classes for state management
|
||||||
|
- ✅ **MutationObserver Timer** - JavaScript watches for CSS class changes to start/stop elapsed time counter
|
||||||
|
- ✅ **Professional Design** - Modal backdrop blur, spinner animation, red reload button with hover effects
|
||||||
|
- Files modified: [App.razor](DiunaBI.UI.Web/Components/App.razor), [app.css](DiunaBI.UI.Web/wwwroot/app.css)
|
||||||
|
- Files created: [reconnect.js](DiunaBI.UI.Web/wwwroot/js/reconnect.js)
|
||||||
|
- Status: Blazor reconnection UI now matches app theme with timer indicator
|
||||||
|
|
||||||
|
**Jobs List Sorting and Multi-Select Filtering (Dec 8, 2025):**
|
||||||
|
- ✅ **Fixed Job Sorting** - Changed from single CreatedAt DESC to Priority ASC → JobType → CreatedAt DESC
|
||||||
|
- ✅ **Multi-Select Status Filter** - Replaced single status dropdown with multi-select supporting multiple JobStatus values
|
||||||
|
- ✅ **Auto-Refresh on Filter Change** - Filters now automatically trigger data reload without requiring manual button click
|
||||||
|
- ✅ **API Updates** - JobsController GetAll endpoint accepts `List<JobStatus>? statuses` instead of single status
|
||||||
|
- ✅ **JobService Updates** - Sends status values as integers in query string for multi-select support
|
||||||
|
- Files modified: [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs), [JobService.cs](DiunaBI.UI.Shared/Services/JobService.cs), [Index.razor](DiunaBI.UI.Shared/Pages/Jobs/Index.razor), [Index.razor.cs](DiunaBI.UI.Shared/Pages/Jobs/Index.razor.cs)
|
||||||
|
- Status: Jobs list now sortable by priority/type/date with working multi-select filters
|
||||||
|
|
||||||
|
**User Timezone Support (Dec 8, 2025):**
|
||||||
|
- ✅ **DateTimeHelper Service** - Created JS Interop service to detect user's browser timezone
|
||||||
|
- ✅ **UTC to Local Conversion** - All date displays now show user's local timezone instead of UTC
|
||||||
|
- ✅ **Database Consistency** - Database continues to store UTC (correct), conversion only for display
|
||||||
|
- ✅ **Updated Pages** - Applied timezone conversion to all date fields in:
|
||||||
|
- Jobs Index and Details pages
|
||||||
|
- Layers Details page (CreatedAt, ModifiedAt, record history)
|
||||||
|
- DataInbox Index page
|
||||||
|
- ✅ **Service Registration** - Registered DateTimeHelper as scoped service in DI container
|
||||||
|
- Files created: [DateTimeHelper.cs](DiunaBI.UI.Shared/Services/DateTimeHelper.cs)
|
||||||
|
- Files modified: [ServiceCollectionExtensions.cs](DiunaBI.UI.Shared/Extensions/ServiceCollectionExtensions.cs), [Jobs/Index.razor.cs](DiunaBI.UI.Shared/Pages/Jobs/Index.razor.cs), [Jobs/Details.razor](DiunaBI.UI.Shared/Pages/Jobs/Details.razor), [Layers/Details.razor](DiunaBI.UI.Shared/Pages/Layers/Details.razor), [Layers/Details.razor.cs](DiunaBI.UI.Shared/Pages/Layers/Details.razor.cs), [DataInbox/Index.razor.cs](DiunaBI.UI.Shared/Pages/DataInbox/Index.razor.cs)
|
||||||
|
- Status: All dates display in user's local timezone with format "yyyy-MM-dd HH:mm:ss"
|
||||||
|
|
||||||
|
**QueueJob Model Cleanup and AutoImport User (Dec 8, 2025):**
|
||||||
|
- ✅ **Removed Duplicate Fields** - Removed CreatedAtUtc and ModifiedAtUtc from QueueJob (were duplicates of CreatedAt/ModifiedAt)
|
||||||
|
- ✅ **Added ModifiedAt Field** - Was missing, now tracks job modification timestamp
|
||||||
|
- ✅ **AutoImport User ID** - Created User.AutoImportUserId constant: `f392209e-123e-4651-a5a4-0b1d6cf9ff9d`
|
||||||
|
- ✅ **System Operations** - All system-created/modified jobs now use AutoImportUserId for CreatedById and ModifiedById
|
||||||
|
- ✅ **Database Migration** - Created migration: RemoveQueueJobDuplicateUTCFields
|
||||||
|
- Files modified: [QueueJob.cs](DiunaBI.Domain/Entities/QueueJob.cs), [User.cs](DiunaBI.Domain/Entities/User.cs), [JobWorkerService.cs](DiunaBI.Infrastructure/Services/JobWorkerService.cs), [JobSchedulerService.cs](DiunaBI.Infrastructure/Services/JobSchedulerService.cs), [AppDbContext.cs](DiunaBI.Infrastructure/Data/AppDbContext.cs), [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs)
|
||||||
|
- Files created: [20251208205202_RemoveQueueJobDuplicateUTCFields.cs](DiunaBI.Infrastructure/Migrations/20251208205202_RemoveQueueJobDuplicateUTCFields.cs)
|
||||||
|
- Status: QueueJob model cleaned up, all automated operations tracked with AutoImport user ID
|
||||||
|
|
||||||
|
**Job Scheduling UI with JWT Authorization (Dec 8, 2025):**
|
||||||
|
- ✅ **New JWT Endpoints** - Created UI-specific endpoints at `/jobs/ui/schedule/*` with JWT authorization (parallel to API key endpoints)
|
||||||
|
- ✅ **Three Scheduling Options** - MudMenu dropdown in Jobs Index with:
|
||||||
|
- Run All Jobs - schedules all import and process jobs
|
||||||
|
- Run All Imports - schedules import jobs only
|
||||||
|
- Run All Processes - schedules process jobs only
|
||||||
|
- ✅ **JobService Methods** - Added three scheduling methods returning (success, jobsCreated, message) tuples
|
||||||
|
- ✅ **Auto-Refresh** - Jobs list automatically reloads after scheduling with success/failure notifications
|
||||||
|
- ✅ **Dual Authorization** - Existing `/jobs/schedule/{apiKey}` endpoints for automation, new `/jobs/ui/schedule` endpoints for UI users
|
||||||
|
- Files modified: [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs), [JobService.cs](DiunaBI.UI.Shared/Services/JobService.cs), [Index.razor](DiunaBI.UI.Shared/Pages/Jobs/Index.razor), [Index.razor.cs](DiunaBI.UI.Shared/Pages/Jobs/Index.razor.cs)
|
||||||
|
- Status: UI users can now schedule jobs directly from Jobs page using JWT authentication
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**API Key Authorization Fix for Cron Jobs (Dec 6, 2025):**
|
||||||
|
- ✅ **Fixed 401 Unauthorized on API Key Endpoints** - Cron jobs calling `/jobs/schedule` endpoints were getting rejected despite valid API keys
|
||||||
|
- ✅ **Added [AllowAnonymous] Attribute** - Bypasses controller-level `[Authorize]` to allow `[ApiKeyAuth]` filter to handle authorization
|
||||||
|
- ✅ **Three Endpoints Fixed** - Applied fix to all job scheduling endpoints:
|
||||||
|
- `POST /jobs/schedule` - Schedule all jobs (imports + processes)
|
||||||
|
- `POST /jobs/schedule/imports` - Schedule import jobs only
|
||||||
|
- `POST /jobs/schedule/processes` - Schedule process jobs only
|
||||||
|
- Root cause: Controller-level `[Authorize]` attribute required JWT Bearer auth for all endpoints, blocking API key authentication
|
||||||
|
- Solution: Add `[AllowAnonymous]` to allow `[ApiKeyAuth]` filter to validate X-API-Key header
|
||||||
|
- Files modified: [JobsController.cs](DiunaBI.API/Controllers/JobsController.cs)
|
||||||
|
- Status: Cron jobs can now authenticate with API key via X-API-Key header
|
||||||
|
|
||||||
|
**SignalR Authentication Token Flow Fix (Dec 6, 2025):**
|
||||||
|
- ✅ **TokenProvider Population** - Fixed `TokenProvider.Token` never being set with JWT, causing 401 Unauthorized on SignalR connections
|
||||||
|
- ✅ **AuthService Token Management** - Injected `TokenProvider` into `AuthService` and set token in 3 key places:
|
||||||
|
- `ValidateWithBackendAsync()` - on fresh Google login
|
||||||
|
- `CheckAuthenticationAsync()` - on session restore from localStorage
|
||||||
|
- `ClearAuthenticationAsync()` - clear token on logout
|
||||||
|
- ✅ **SignalR Initialization Timing** - Moved SignalR initialization from `MainLayout.OnInitializedAsync` to after authentication completes
|
||||||
|
- ✅ **Event-Driven Architecture** - `MainLayout` now subscribes to `AuthenticationStateChanged` event to initialize SignalR when user authenticates
|
||||||
|
- ✅ **Session Restore Support** - `CheckAuthenticationAsync()` now fires `AuthenticationStateChanged` event to initialize SignalR on page refresh
|
||||||
|
- Root cause: SignalR was initialized before authentication, so JWT token was empty during connection setup
|
||||||
|
- Solution: Initialize SignalR only after token is available via event subscription
|
||||||
|
- Files modified: [AuthService.cs](DiunaBI.UI.Shared/Services/AuthService.cs), [MainLayout.razor](DiunaBI.UI.Shared/Components/Layout/MainLayout.razor)
|
||||||
|
- Status: SignalR authentication working for both fresh login and restored sessions
|
||||||
|
|
||||||
|
**SignalR Authentication DI Fix (Dec 6, 2025):**
|
||||||
|
- ✅ **TokenProvider Registration** - Added missing `TokenProvider` service registration in DI container
|
||||||
|
- ✅ **EntityChangeHubService Scope Fix** - Changed from singleton to scoped to support user-specific JWT tokens
|
||||||
|
- ✅ **Bug Fix** - Resolved `InvalidOperationException` preventing app from starting after SignalR authentication was added
|
||||||
|
- Root cause: Singleton service (`EntityChangeHubService`) cannot depend on scoped service (`TokenProvider`) in DI
|
||||||
|
- Solution: Made `EntityChangeHubService` scoped so each user session has its own authenticated SignalR connection
|
||||||
|
- Files modified: [ServiceCollectionExtensions.cs](DiunaBI.UI.Shared/Extensions/ServiceCollectionExtensions.cs)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
**Security Audit & Hardening (Dec 5, 2025):**
|
**Security Audit & Hardening (Dec 5, 2025):**
|
||||||
- ✅ **JWT Token Validation** - Enabled issuer/audience validation in [Program.cs](DiunaBI.API/Program.cs), fixed config key mismatch in [JwtTokenService.cs](DiunaBI.API/Services/JwtTokenService.cs)
|
- ✅ **JWT Token Validation** - Enabled issuer/audience validation in [Program.cs](DiunaBI.API/Program.cs), fixed config key mismatch in [JwtTokenService.cs](DiunaBI.API/Services/JwtTokenService.cs)
|
||||||
- ✅ **API Key Security** - Created [ApiKeyAuthAttribute.cs](DiunaBI.API/Attributes/ApiKeyAuthAttribute.cs) with X-API-Key header auth, constant-time comparison
|
- ✅ **API Key Security** - Created [ApiKeyAuthAttribute.cs](DiunaBI.API/Attributes/ApiKeyAuthAttribute.cs) with X-API-Key header auth, constant-time comparison
|
||||||
|
|||||||
9
.gitignore
vendored
9
.gitignore
vendored
@@ -562,4 +562,11 @@ coverage/
|
|||||||
## Temporary folders
|
## Temporary folders
|
||||||
##
|
##
|
||||||
tmp/
|
tmp/
|
||||||
temp/
|
temp/
|
||||||
|
|
||||||
|
##
|
||||||
|
## LocalDB Development Files
|
||||||
|
##
|
||||||
|
DevTools/LocalDB/backups/*.bak
|
||||||
|
DevTools/LocalDB/backups/*.bacpac
|
||||||
|
DevTools/LocalDB/data/
|
||||||
381
CLAUDE.md
Normal file
381
CLAUDE.md
Normal file
@@ -0,0 +1,381 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
DiunaBI is a full-stack Business Intelligence platform built on .NET 10.0 with a multi-tier Clean Architecture. It provides a plugin-based system for importing, processing, and exporting business data with real-time updates, job queue management, and comprehensive audit trails.
|
||||||
|
|
||||||
|
**Tech Stack:**
|
||||||
|
- Backend: ASP.NET Core 10.0 Web API
|
||||||
|
- Frontend: Blazor Server + MAUI Mobile (iOS, Android, Windows, macOS)
|
||||||
|
- Database: SQL Server + EF Core 10.0
|
||||||
|
- UI Framework: MudBlazor 8.0
|
||||||
|
- Real-time: SignalR (EntityChangeHub)
|
||||||
|
- Authentication: JWT Bearer + Google OAuth
|
||||||
|
- External APIs: Google Sheets API, Google Drive API
|
||||||
|
- Logging: Serilog (Console, File)
|
||||||
|
|
||||||
|
## Common Commands
|
||||||
|
|
||||||
|
### Build and Run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build the entire solution
|
||||||
|
dotnet build DiunaBI.sln
|
||||||
|
|
||||||
|
# Build specific project
|
||||||
|
dotnet build DiunaBI.API/DiunaBI.API.csproj
|
||||||
|
|
||||||
|
# Run API (backend)
|
||||||
|
dotnet run --project DiunaBI.API/DiunaBI.API.csproj
|
||||||
|
|
||||||
|
# Run Web UI (Blazor Server)
|
||||||
|
dotnet run --project DiunaBI.UI.Web/DiunaBI.UI.Web.csproj
|
||||||
|
|
||||||
|
# Clean build artifacts
|
||||||
|
dotnet clean DiunaBI.sln
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Migrations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add new migration
|
||||||
|
dotnet ef migrations add <MigrationName> --project DiunaBI.Infrastructure --startup-project DiunaBI.API
|
||||||
|
|
||||||
|
# Apply migrations to database
|
||||||
|
dotnet ef database update --project DiunaBI.Infrastructure --startup-project DiunaBI.API
|
||||||
|
|
||||||
|
# Remove last migration (if not applied)
|
||||||
|
dotnet ef migrations remove --project DiunaBI.Infrastructure --startup-project DiunaBI.API
|
||||||
|
|
||||||
|
# List all migrations
|
||||||
|
dotnet ef migrations list --project DiunaBI.Infrastructure --startup-project DiunaBI.API
|
||||||
|
```
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run all tests
|
||||||
|
dotnet test DiunaBI.Tests/DiunaBI.Tests.csproj
|
||||||
|
|
||||||
|
# Run tests with detailed output
|
||||||
|
dotnet test DiunaBI.Tests/DiunaBI.Tests.csproj --logger "console;verbosity=detailed"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Plugin Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build plugin project (automatically copies DLL to API bin/Plugins/)
|
||||||
|
dotnet build DiunaBI.Plugins.Morska/DiunaBI.Plugins.Morska.csproj
|
||||||
|
dotnet build DiunaBI.Plugins.PedrolloPL/DiunaBI.Plugins.PedrolloPL.csproj
|
||||||
|
|
||||||
|
# Plugins are auto-loaded from bin/Plugins/ at API startup
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker (Production)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build API container (with Morska plugin)
|
||||||
|
docker build -f DiunaBI.API/Dockerfile -t diunabi-api --build-arg PLUGIN_PROJECT=DiunaBI.Plugins.Morska .
|
||||||
|
|
||||||
|
# Build API container (with PedrolloPL plugin)
|
||||||
|
docker build -f DiunaBI.API/Dockerfile -t diunabi-api --build-arg PLUGIN_PROJECT=DiunaBI.Plugins.PedrolloPL .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Solution Structure (10 Projects)
|
||||||
|
|
||||||
|
**DiunaBI.API** - ASP.NET Core Web API
|
||||||
|
- Controllers: Auth, Layers, Jobs, DataInbox
|
||||||
|
- Hubs: EntityChangeHub (SignalR)
|
||||||
|
- Services: GoogleAuthService, JwtTokenService
|
||||||
|
- API Key authorization via [ApiKeyAuth] attribute
|
||||||
|
|
||||||
|
**DiunaBI.Domain** - Domain entities (9 entities)
|
||||||
|
- User, Layer, Record, RecordHistory, QueueJob, DataInbox, ProcessSource
|
||||||
|
|
||||||
|
**DiunaBI.Application** - DTOs and application models
|
||||||
|
- LayerDto, RecordDto, UserDto, RecordHistoryDto, JobDto, PagedResult<T>
|
||||||
|
|
||||||
|
**DiunaBI.Infrastructure** - Data access and core services
|
||||||
|
- Data: AppDbContext, 47 EF Core migrations
|
||||||
|
- Interceptors: EntityChangeInterceptor (auto-broadcasts DB changes via SignalR)
|
||||||
|
- Services: PluginManager, JobSchedulerService, JobWorkerService
|
||||||
|
- Helpers: GoogleSheetsHelper, GoogleDriveHelper
|
||||||
|
- Plugin Base Classes: BaseDataImporter, BaseDataProcessor, BaseDataExporter
|
||||||
|
|
||||||
|
**DiunaBI.UI.Web** - Blazor Server web application
|
||||||
|
|
||||||
|
**DiunaBI.UI.Mobile** - MAUI mobile application (iOS, Android, Windows, macOS)
|
||||||
|
|
||||||
|
**DiunaBI.UI.Shared** - Shared Blazor component library
|
||||||
|
- Pages/: Feature-based folders (Layers/, Jobs/, DataInbox/)
|
||||||
|
- Components/: Layout/ (MainLayout, EmptyLayout, Routes), Auth/ (AuthGuard, LoginCard)
|
||||||
|
- Services/: LayerService, JobService, DataInboxService, EntityChangeHubService, AuthService
|
||||||
|
|
||||||
|
**DiunaBI.Plugins.Morska** - Production plugin (17 total)
|
||||||
|
- 4 Importers: Standard, D1, D3, FK2
|
||||||
|
- 12 Processors: D6, T1, T3, T4, T5 variants
|
||||||
|
- 1 Exporter: Google Sheets export
|
||||||
|
|
||||||
|
**DiunaBI.Plugins.PedrolloPL** - Production plugin (1 total)
|
||||||
|
- 1 Importer: B3 (DataInbox → Layer with dictionary mapping)
|
||||||
|
|
||||||
|
**DiunaBI.Tests** - Unit and integration tests
|
||||||
|
|
||||||
|
### Data Flow Architecture
|
||||||
|
|
||||||
|
1. **Import Flow**: External data → DataInbox API → Import Plugin → Import Layer
|
||||||
|
2. **Process Flow**: Import Layer → Process Plugin → Processed Layer
|
||||||
|
3. **Export Flow**: Processed Layer → Export Plugin → Google Sheets/Drive
|
||||||
|
4. **Real-time Updates**: DB Change → EntityChangeInterceptor → SignalR Hub → All Clients
|
||||||
|
|
||||||
|
### Plugin System
|
||||||
|
|
||||||
|
**Plugin Discovery:**
|
||||||
|
- Plugins loaded from `bin/Plugins/` directory at startup
|
||||||
|
- PluginManager scans assemblies for IDataImporter, IDataProcessor, IDataExporter implementations
|
||||||
|
- Plugins auto-registered in DI container
|
||||||
|
|
||||||
|
**Plugin Base Classes** (in DiunaBI.Infrastructure/Plugins/):
|
||||||
|
- `BaseDataImporter`: Abstract base for importers, access to AppDbContext, GoogleSheetsHelper, GoogleDriveHelper
|
||||||
|
- `BaseDataProcessor`: Abstract base for processors, access to AppDbContext, PluginManager
|
||||||
|
- `BaseDataExporter`: Abstract base for exporters, access to AppDbContext, GoogleSheetsHelper, GoogleDriveHelper
|
||||||
|
|
||||||
|
**Plugin Execution:**
|
||||||
|
- Importers: Read external data sources, create Layer + Records
|
||||||
|
- Processors: Read source Layers, apply transformations, create target Layers + Records
|
||||||
|
- Exporters: Read Layers, write to Google Sheets/Drive
|
||||||
|
|
||||||
|
**Plugin Configuration:**
|
||||||
|
- Stored in Administration layers (Type = ImportWorker/ProcessWorker)
|
||||||
|
- Records with Code = "Plugin", "Priority", "MaxRetries" define job configs
|
||||||
|
- JobSchedulerService reads configs and creates QueueJobs
|
||||||
|
|
||||||
|
### Job Queue System
|
||||||
|
|
||||||
|
**Components:**
|
||||||
|
- `QueueJob` entity: LayerId, PluginName, JobType (Import/Process), Status (Pending/Running/Completed/Failed/Retrying), Priority (0 = highest)
|
||||||
|
- `JobSchedulerService`: Creates jobs from Administration layer configs
|
||||||
|
- `JobWorkerService`: Background service polling every 5 seconds, executes jobs via PluginManager
|
||||||
|
- Retry logic: 30s → 2m → 5m delays, max 5 retries
|
||||||
|
|
||||||
|
**Job Lifecycle:**
|
||||||
|
1. Creation: JobSchedulerService or manual via `/jobs/create-for-layer/{layerId}`
|
||||||
|
2. Queued: Status = Pending, sorted by CreatedAt DESC then Priority ASC
|
||||||
|
3. Execution: JobWorkerService picks up, Status = Running
|
||||||
|
4. Completion: Status = Completed or Failed
|
||||||
|
5. Retry: On failure, Status = Retrying with exponential backoff
|
||||||
|
6. Real-time: All status changes broadcast via SignalR to UI
|
||||||
|
|
||||||
|
**Job Scheduling Endpoints:**
|
||||||
|
- `POST /jobs/ui/schedule` - Schedule all jobs (JWT auth for UI users)
|
||||||
|
- `POST /jobs/ui/schedule/imports` - Schedule import jobs only (JWT auth)
|
||||||
|
- `POST /jobs/ui/schedule/processes` - Schedule process jobs only (JWT auth)
|
||||||
|
- `POST /jobs/schedule/{apiKey}` - Schedule all jobs (API key auth for cron)
|
||||||
|
- `POST /jobs/schedule/imports/{apiKey}` - Schedule import jobs (API key auth)
|
||||||
|
- `POST /jobs/schedule/processes/{apiKey}` - Schedule process jobs (API key auth)
|
||||||
|
|
||||||
|
### Real-time Updates (SignalR)
|
||||||
|
|
||||||
|
**Architecture:**
|
||||||
|
- Hub: `/hubs/entitychanges` (requires JWT authentication)
|
||||||
|
- Interceptor: `EntityChangeInterceptor` captures EF Core changes (Added, Modified, Deleted)
|
||||||
|
- Broadcast: After SaveChanges, sends `EntityChanged(module, id, operation)` to all clients
|
||||||
|
- Modules: QueueJobs, Layers, Records, RecordHistory
|
||||||
|
|
||||||
|
**UI Integration:**
|
||||||
|
- `EntityChangeHubService`: Singleton service initialized after authentication in MainLayout
|
||||||
|
- Components subscribe: `HubService.EntityChanged += OnEntityChanged`
|
||||||
|
- Auto-reconnect enabled
|
||||||
|
- Pages with real-time updates: Jobs/Index, Jobs/Details, Layers/Index, Layers/Details, DataInbox/Index
|
||||||
|
|
||||||
|
**Authentication Flow:**
|
||||||
|
1. User logs in with Google OAuth → JWT token stored in localStorage
|
||||||
|
2. `TokenProvider.Token` populated in AuthService
|
||||||
|
3. `MainLayout` subscribes to `AuthenticationStateChanged` event
|
||||||
|
4. On auth success, SignalR connection initialized with JWT token
|
||||||
|
5. Token sent via `accessTokenProvider` in HubConnection options
|
||||||
|
|
||||||
|
### Authentication & Security
|
||||||
|
|
||||||
|
**Google OAuth Flow:**
|
||||||
|
1. Client exchanges Google ID token → `POST /auth/apiToken`
|
||||||
|
2. GoogleAuthService validates with Google, maps to internal User entity
|
||||||
|
3. Returns JWT (7-day expiration, HS256 signing)
|
||||||
|
4. JWT required on all protected endpoints except `/auth/apiToken`, `/health`
|
||||||
|
5. UserId extraction middleware sets X-UserId header for audit trails
|
||||||
|
|
||||||
|
**API Key Authentication:**
|
||||||
|
- Custom [ApiKeyAuth] attribute for cron job endpoints
|
||||||
|
- X-API-Key header with constant-time comparison
|
||||||
|
- Used for DataInbox external endpoints and job scheduling
|
||||||
|
|
||||||
|
**Security Features:**
|
||||||
|
- Rate limiting: 100 req/min general, 10 req/min auth
|
||||||
|
- Security headers: XSS, clickjacking, MIME sniffing protection
|
||||||
|
- Input validation: Pagination limits (1-1000), Base64 size limits (10MB)
|
||||||
|
- Stack trace hiding: Generic error messages in production
|
||||||
|
- CORS: Configured for localhost:4200, diuna.bim-it.pl, morska.diunabi.com
|
||||||
|
|
||||||
|
### Database Schema
|
||||||
|
|
||||||
|
**47 EF Core Migrations** - All in DiunaBI.Infrastructure/Migrations/
|
||||||
|
|
||||||
|
**Key Entities:**
|
||||||
|
|
||||||
|
**User**
|
||||||
|
- Id (Guid), Email, UserName
|
||||||
|
- Google OAuth identity
|
||||||
|
- Constant: `User.AutoImportUserId = "f392209e-123e-4651-a5a4-0b1d6cf9ff9d"` (system operations)
|
||||||
|
|
||||||
|
**Layer**
|
||||||
|
- Number, Name, Type (Import/Processed/Administration/Dictionary)
|
||||||
|
- ParentId (hierarchical relationships)
|
||||||
|
- IsDeleted (soft delete), IsCancelled (processing control)
|
||||||
|
- CreatedAt, ModifiedAt, CreatedById, ModifiedById (audit trail)
|
||||||
|
|
||||||
|
**Record**
|
||||||
|
- Code (unique identifier per layer), LayerId
|
||||||
|
- Value1-Value32 (double?), Desc1 (string, max 10000 chars)
|
||||||
|
- IsDeleted (soft delete)
|
||||||
|
- Full audit trail via RecordHistory
|
||||||
|
|
||||||
|
**RecordHistory** (Migration 47)
|
||||||
|
- RecordId, LayerId, ChangedAt, ChangedById
|
||||||
|
- ChangeType (Created/Updated/Deleted)
|
||||||
|
- Code, Desc1 (snapshot)
|
||||||
|
- ChangedFields (comma-separated), ChangesSummary (JSON old/new values)
|
||||||
|
- Indexes: (RecordId, ChangedAt), (LayerId, ChangedAt)
|
||||||
|
|
||||||
|
**QueueJob**
|
||||||
|
- LayerId, PluginName, JobType, Priority, Status
|
||||||
|
- RetryCount, MaxRetries (default 5)
|
||||||
|
- CreatedAt, ModifiedAt, LastAttemptAt, CompletedAt
|
||||||
|
- CreatedById, ModifiedById (uses User.AutoImportUserId for system jobs)
|
||||||
|
|
||||||
|
**DataInbox**
|
||||||
|
- Name, Source (identifiers), Data (base64-encoded JSON array)
|
||||||
|
- Used by importers to stage incoming data
|
||||||
|
|
||||||
|
**ProcessSource**
|
||||||
|
- SourceLayerId, TargetLayerId (defines layer processing relationships)
|
||||||
|
|
||||||
|
**Audit Trail Patterns:**
|
||||||
|
- All entities have CreatedAt, ModifiedAt with GETUTCDATE() defaults
|
||||||
|
- Foreign keys to Users: CreatedById, ModifiedById
|
||||||
|
- Soft deletes via IsDeleted flag
|
||||||
|
- RecordHistory tracks field-level changes with JSON diffs
|
||||||
|
|
||||||
|
### UI Organization
|
||||||
|
|
||||||
|
**Feature-Based Structure** (as of Dec 5, 2025):
|
||||||
|
- `Pages/Layers/` - Index.razor + Details.razor (list, detail, edit, history)
|
||||||
|
- `Pages/Jobs/` - Index.razor + Details.razor (list, detail, retry, cancel, scheduling)
|
||||||
|
- `Pages/DataInbox/` - Index.razor + Details.razor (list, detail, base64 decode)
|
||||||
|
- `Components/Layout/` - MainLayout, EmptyLayout, Routes
|
||||||
|
- `Components/Auth/` - AuthGuard, LoginCard
|
||||||
|
|
||||||
|
**Code-Behind Pattern:**
|
||||||
|
- Complex pages (50+ lines logic): Separate `.razor.cs` files
|
||||||
|
- Simple pages: Inline `@code` blocks
|
||||||
|
- Namespaces: `DiunaBI.UI.Shared.Pages.{Feature}`
|
||||||
|
|
||||||
|
**Filter State Persistence:**
|
||||||
|
- LayerFilterStateService, DataInboxFilterStateService
|
||||||
|
- Singleton services remember search, type, page selections across navigation
|
||||||
|
|
||||||
|
**Timezone Handling:**
|
||||||
|
- DateTimeHelper service detects browser timezone via JS Interop
|
||||||
|
- All dates stored as UTC in DB, converted to user's local timezone for display
|
||||||
|
- Format: "yyyy-MM-dd HH:mm:ss"
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
**Environment Variables** (appsettings.Development.json):
|
||||||
|
- `ConnectionStrings:SQLDatabase` - SQL Server connection (localhost:21433, DB: DiunaBI-PedrolloPL)
|
||||||
|
- `JwtSettings:SecurityKey`, `JwtSettings:ExpiryDays` (7)
|
||||||
|
- `GoogleAuth:ClientId`, `GoogleAuth:RedirectUri`
|
||||||
|
- `apiKey`, `apiUser`, `apiPass` - DataInbox API security
|
||||||
|
- `exportDirectory` - Google Drive folder ID for exports
|
||||||
|
- `InstanceName` - DEV/PROD environment identifier
|
||||||
|
|
||||||
|
**CORS Origins:**
|
||||||
|
- http://localhost:4200 (development)
|
||||||
|
- https://diuna.bim-it.pl (production)
|
||||||
|
- https://morska.diunabi.com (production)
|
||||||
|
|
||||||
|
**Logging:**
|
||||||
|
- Serilog with Console + File sinks
|
||||||
|
- Override levels: Microsoft.AspNetCore = Warning, EF Core = Warning, Google.Apis = Warning
|
||||||
|
- Sensitive data logging only in Development
|
||||||
|
|
||||||
|
## Development Patterns
|
||||||
|
|
||||||
|
### When Adding a New Plugin
|
||||||
|
|
||||||
|
1. Create new project: `DiunaBI.Plugins.{Name}`
|
||||||
|
2. Reference: DiunaBI.Domain, DiunaBI.Infrastructure
|
||||||
|
3. Inherit from: BaseDataImporter/BaseDataProcessor/BaseDataExporter
|
||||||
|
4. Implement abstract methods: ImportAsync/ProcessAsync/ExportAsync, ValidateConfiguration
|
||||||
|
5. Build triggers automatic copy to `bin/Plugins/` via MSBuild target
|
||||||
|
6. No registration needed - PluginManager auto-discovers at startup
|
||||||
|
|
||||||
|
### When Adding a New Entity
|
||||||
|
|
||||||
|
1. Create entity in DiunaBI.Domain/Entities/
|
||||||
|
2. Add DbSet to AppDbContext
|
||||||
|
3. Create migration: `dotnet ef migrations add {Name} --project DiunaBI.Infrastructure --startup-project DiunaBI.API`
|
||||||
|
4. If entity needs real-time updates, add module name to EntityChangeInterceptor
|
||||||
|
5. Create DTO in DiunaBI.Application/DTOs/
|
||||||
|
6. Add controller in DiunaBI.API/Controllers/ with [Authorize] and [RequireRateLimit("api")]
|
||||||
|
|
||||||
|
### When Adding Real-time Updates to a UI Page
|
||||||
|
|
||||||
|
1. Inject EntityChangeHubService
|
||||||
|
2. Subscribe to EntityChanged event in OnInitializedAsync
|
||||||
|
3. Filter by module name
|
||||||
|
4. Call StateHasChanged() in event handler
|
||||||
|
5. Implement IDisposable, unsubscribe in Dispose
|
||||||
|
6. Test both initial load and SignalR updates
|
||||||
|
|
||||||
|
### When Modifying Job System
|
||||||
|
|
||||||
|
- JobSchedulerService: Changes to job creation from layer configs
|
||||||
|
- JobWorkerService: Changes to job execution, retry logic, rate limiting
|
||||||
|
- QueueJob entity: Changes to job schema require migration
|
||||||
|
- Jobs UI: Real-time updates required, test SignalR broadcasts
|
||||||
|
|
||||||
|
### Security Considerations
|
||||||
|
|
||||||
|
- Never log sensitive data (tokens, passwords, API keys) except in Development
|
||||||
|
- Use generic error messages in production (no stack traces)
|
||||||
|
- All new endpoints require [Authorize] unless explicitly [AllowAnonymous]
|
||||||
|
- API key endpoints require [ApiKeyAuth] and [AllowAnonymous]
|
||||||
|
- Validate pagination parameters (1-1000 range)
|
||||||
|
- Use constant-time comparison for API keys
|
||||||
|
- Rate limit all public endpoints
|
||||||
|
|
||||||
|
## Known Limitations
|
||||||
|
|
||||||
|
- JobWorkerService polls every 5 seconds (not event-driven)
|
||||||
|
- Google Sheets API quota: 5-second delay after import jobs
|
||||||
|
- Retry delays fixed: 30s → 2m → 5m (not configurable per job)
|
||||||
|
- Plugin configuration stored in Records (not strongly typed)
|
||||||
|
- No plugin versioning or hot-reload support
|
||||||
|
- SignalR requires JWT authentication (no anonymous connections)
|
||||||
|
|
||||||
|
## Important Notes
|
||||||
|
|
||||||
|
- **DO NOT** commit `.env` file (contains secrets)
|
||||||
|
- **DO NOT** modify migration files after applying to production
|
||||||
|
- **ALWAYS** use User.AutoImportUserId for system-created entities (jobs, automated processes)
|
||||||
|
- **ALWAYS** implement IDisposable for pages subscribing to SignalR events
|
||||||
|
- **ALWAYS** test real-time updates when modifying entities with SignalR broadcasts
|
||||||
|
- **Plugin DLLs** are auto-copied to `bin/Plugins/` on build via MSBuild target
|
||||||
|
- **Database changes** require migration AND applying to production via deployment scripts
|
||||||
|
- **Foreign keys** use RESTRICT (not CASCADE) to prevent accidental data loss (Migration 45)
|
||||||
|
- **Soft deletes** via IsDeleted flag, not physical deletion
|
||||||
|
- **Timezone handling** - store UTC, display local (via DateTimeHelper)
|
||||||
@@ -1,7 +1,9 @@
|
|||||||
DECLARE @JustForDebug TINYINT = 0;
|
DECLARE @JustForDebug TINYINT = 0;
|
||||||
|
|
||||||
|
-- FIX DATAINBOX!
|
||||||
|
|
||||||
-- SETUP VARIABLES
|
-- SETUP VARIABLES
|
||||||
DECLARE @Year INT = 2024;
|
DECLARE @Year INT = 2025;
|
||||||
DECLARE @Type NVARCHAR(5) = 'B3';
|
DECLARE @Type NVARCHAR(5) = 'B3';
|
||||||
DECLARE @StartDate NVARCHAR(10) = '2025.01.02';
|
DECLARE @StartDate NVARCHAR(10) = '2025.01.02';
|
||||||
DECLARE @EndDate NVARCHAR(10) = '2026.12.31'
|
DECLARE @EndDate NVARCHAR(10) = '2026.12.31'
|
||||||
@@ -22,7 +24,7 @@ SET @Plugin =
|
|||||||
DECLARE @DataInboxName NVARCHAR(100);
|
DECLARE @DataInboxName NVARCHAR(100);
|
||||||
SET @DataInboxName =
|
SET @DataInboxName =
|
||||||
CASE @Type
|
CASE @Type
|
||||||
WHEN 'B3' THEN 'B3_2024'
|
WHEN 'B3' THEN 'P2_2025'
|
||||||
ELSE NULL -- If @Type doesn't match, set it to NULL
|
ELSE NULL -- If @Type doesn't match, set it to NULL
|
||||||
END;
|
END;
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
DECLARE @JustForDebug TINYINT = 0;
|
DECLARE @JustForDebug TINYINT = 0;
|
||||||
|
|
||||||
-- SETUP VARIABLES
|
-- SETUP VARIABLES
|
||||||
DECLARE @Year INT = 2024;
|
DECLARE @Year INT = 2025;
|
||||||
|
|
||||||
DECLARE @Number INT = (SELECT COUNT(id) + 1 FROM [DiunaBI-PedrolloPL].[dbo].[Layers]);
|
DECLARE @Number INT = (SELECT COUNT(id) + 1 FROM [DiunaBI-PedrolloPL].[dbo].[Layers]);
|
||||||
DECLARE @CurrentTimestamp NVARCHAR(14) = FORMAT(GETDATE(), 'yyyyMMddHHmm');
|
DECLARE @CurrentTimestamp NVARCHAR(14) = FORMAT(GETDATE(), 'yyyyMMddHHmm');
|
||||||
@@ -56,3 +56,16 @@ VALUES ((SELECT NEWID()), 'Plugin', 'PedrolloPL.Process.P2', GETDATE(), GETDATE(
|
|||||||
INSERT INTO [DiunaBI-PedrolloPL].[dbo].[Records]
|
INSERT INTO [DiunaBI-PedrolloPL].[dbo].[Records]
|
||||||
([Id], [Code], [Desc1], [CreatedAt], [ModifiedAt], [CreatedById], [ModifiedById], [IsDeleted], [LayerId])
|
([Id], [Code], [Desc1], [CreatedAt], [ModifiedAt], [CreatedById], [ModifiedById], [IsDeleted], [LayerId])
|
||||||
VALUES ((SELECT NEWID()), 'Priority', '110', GETDATE(), GETDATE(), '117be4f0-b5d1-41a1-a962-39dc30cce368', '117be4f0-b5d1-41a1-a962-39dc30cce368', 0, @LayerId);
|
VALUES ((SELECT NEWID()), 'Priority', '110', GETDATE(), GETDATE(), '117be4f0-b5d1-41a1-a962-39dc30cce368', '117be4f0-b5d1-41a1-a962-39dc30cce368', 0, @LayerId);
|
||||||
|
--
|
||||||
|
INSERT INTO [DiunaBI-PedrolloPL].[dbo].[Records]
|
||||||
|
([Id], [Code], [Desc1], [CreatedAt], [ModifiedAt], [CreatedById], [ModifiedById], [IsDeleted], [LayerId])
|
||||||
|
VALUES ((SELECT NEWID()), 'GoogleSheetId', '1jI-3QrlBADm5slEl2Balf29cKmHwkYi4pboaHY-gRqc', GETDATE(), GETDATE(), '117be4f0-b5d1-41a1-a962-39dc30cce368', '117be4f0-b5d1-41a1-a962-39dc30cce368', 0, @LayerId);
|
||||||
|
|
||||||
|
INSERT INTO [DiunaBI-PedrolloPL].[dbo].[Records]
|
||||||
|
([Id], [Code], [Desc1], [CreatedAt], [ModifiedAt], [CreatedById], [ModifiedById], [IsDeleted], [LayerId])
|
||||||
|
VALUES ((SELECT NEWID()), 'GoogleSheetTab', 'P2_Export_DiunaBI', GETDATE(), GETDATE(), '117be4f0-b5d1-41a1-a962-39dc30cce368', '117be4f0-b5d1-41a1-a962-39dc30cce368', 0, @LayerId);
|
||||||
|
|
||||||
|
INSERT INTO [DiunaBI-PedrolloPL].[dbo].[Records]
|
||||||
|
([Id], [Code], [Desc1], [CreatedAt], [ModifiedAt], [CreatedById], [ModifiedById], [IsDeleted], [LayerId])
|
||||||
|
VALUES ((SELECT NEWID()), 'GoogleSheetRange', 'C32:O48', GETDATE(), GETDATE(), '117be4f0-b5d1-41a1-a962-39dc30cce368', '117be4f0-b5d1-41a1-a962-39dc30cce368', 0, @LayerId);
|
||||||
|
|
||||||
@@ -36,7 +36,7 @@ public class JobsController : Controller
|
|||||||
public async Task<IActionResult> GetAll(
|
public async Task<IActionResult> GetAll(
|
||||||
[FromQuery] int start = 0,
|
[FromQuery] int start = 0,
|
||||||
[FromQuery] int limit = 50,
|
[FromQuery] int limit = 50,
|
||||||
[FromQuery] JobStatus? status = null,
|
[FromQuery] List<JobStatus>? statuses = null,
|
||||||
[FromQuery] JobType? jobType = null,
|
[FromQuery] JobType? jobType = null,
|
||||||
[FromQuery] Guid? layerId = null)
|
[FromQuery] Guid? layerId = null)
|
||||||
{
|
{
|
||||||
@@ -54,9 +54,9 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
var query = _db.QueueJobs.AsQueryable();
|
var query = _db.QueueJobs.AsQueryable();
|
||||||
|
|
||||||
if (status.HasValue)
|
if (statuses != null && statuses.Count > 0)
|
||||||
{
|
{
|
||||||
query = query.Where(j => j.Status == status.Value);
|
query = query.Where(j => statuses.Contains(j.Status));
|
||||||
}
|
}
|
||||||
|
|
||||||
if (jobType.HasValue)
|
if (jobType.HasValue)
|
||||||
@@ -71,8 +71,10 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
var totalCount = await query.CountAsync();
|
var totalCount = await query.CountAsync();
|
||||||
|
|
||||||
|
// Sort by: CreatedAt DESC (newest first), then Priority ASC (0=highest)
|
||||||
var items = await query
|
var items = await query
|
||||||
.OrderByDescending(j => j.CreatedAt)
|
.OrderByDescending(j => j.CreatedAt)
|
||||||
|
.ThenBy(j => j.Priority)
|
||||||
.Skip(start)
|
.Skip(start)
|
||||||
.Take(limit)
|
.Take(limit)
|
||||||
.AsNoTracking()
|
.AsNoTracking()
|
||||||
@@ -125,6 +127,7 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
[HttpPost]
|
[HttpPost]
|
||||||
[Route("schedule")]
|
[Route("schedule")]
|
||||||
|
[AllowAnonymous] // Bypass controller-level [Authorize] to allow API key auth
|
||||||
[ApiKeyAuth]
|
[ApiKeyAuth]
|
||||||
public async Task<IActionResult> ScheduleJobs([FromQuery] string? nameFilter = null)
|
public async Task<IActionResult> ScheduleJobs([FromQuery] string? nameFilter = null)
|
||||||
{
|
{
|
||||||
@@ -150,6 +153,7 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
[HttpPost]
|
[HttpPost]
|
||||||
[Route("schedule/imports")]
|
[Route("schedule/imports")]
|
||||||
|
[AllowAnonymous] // Bypass controller-level [Authorize] to allow API key auth
|
||||||
[ApiKeyAuth]
|
[ApiKeyAuth]
|
||||||
public async Task<IActionResult> ScheduleImportJobs([FromQuery] string? nameFilter = null)
|
public async Task<IActionResult> ScheduleImportJobs([FromQuery] string? nameFilter = null)
|
||||||
{
|
{
|
||||||
@@ -175,6 +179,7 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
[HttpPost]
|
[HttpPost]
|
||||||
[Route("schedule/processes")]
|
[Route("schedule/processes")]
|
||||||
|
[AllowAnonymous] // Bypass controller-level [Authorize] to allow API key auth
|
||||||
[ApiKeyAuth]
|
[ApiKeyAuth]
|
||||||
public async Task<IActionResult> ScheduleProcessJobs()
|
public async Task<IActionResult> ScheduleProcessJobs()
|
||||||
{
|
{
|
||||||
@@ -198,6 +203,79 @@ public class JobsController : Controller
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UI-friendly endpoints (JWT auth)
|
||||||
|
[HttpPost]
|
||||||
|
[Route("ui/schedule")]
|
||||||
|
public async Task<IActionResult> ScheduleJobsUI([FromQuery] string? nameFilter = null)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var jobsCreated = await _jobScheduler.ScheduleAllJobsAsync(nameFilter);
|
||||||
|
|
||||||
|
_logger.LogInformation("ScheduleJobsUI: Created {Count} jobs by user {UserId}", jobsCreated, User.Identity?.Name);
|
||||||
|
|
||||||
|
return Ok(new
|
||||||
|
{
|
||||||
|
success = true,
|
||||||
|
jobsCreated,
|
||||||
|
message = $"Successfully scheduled {jobsCreated} jobs"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "ScheduleJobsUI: Error scheduling jobs");
|
||||||
|
return BadRequest("An error occurred processing your request");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[HttpPost]
|
||||||
|
[Route("ui/schedule/imports")]
|
||||||
|
public async Task<IActionResult> ScheduleImportJobsUI([FromQuery] string? nameFilter = null)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var jobsCreated = await _jobScheduler.ScheduleImportJobsAsync(nameFilter);
|
||||||
|
|
||||||
|
_logger.LogInformation("ScheduleImportJobsUI: Created {Count} import jobs by user {UserId}", jobsCreated, User.Identity?.Name);
|
||||||
|
|
||||||
|
return Ok(new
|
||||||
|
{
|
||||||
|
success = true,
|
||||||
|
jobsCreated,
|
||||||
|
message = $"Successfully scheduled {jobsCreated} import jobs"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "ScheduleImportJobsUI: Error scheduling import jobs");
|
||||||
|
return BadRequest("An error occurred processing your request");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[HttpPost]
|
||||||
|
[Route("ui/schedule/processes")]
|
||||||
|
public async Task<IActionResult> ScheduleProcessJobsUI()
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var jobsCreated = await _jobScheduler.ScheduleProcessJobsAsync();
|
||||||
|
|
||||||
|
_logger.LogInformation("ScheduleProcessJobsUI: Created {Count} process jobs by user {UserId}", jobsCreated, User.Identity?.Name);
|
||||||
|
|
||||||
|
return Ok(new
|
||||||
|
{
|
||||||
|
success = true,
|
||||||
|
jobsCreated,
|
||||||
|
message = $"Successfully scheduled {jobsCreated} process jobs"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "ScheduleProcessJobsUI: Error scheduling process jobs");
|
||||||
|
return BadRequest("An error occurred processing your request");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
[HttpPost]
|
[HttpPost]
|
||||||
[Route("{id:guid}/retry")]
|
[Route("{id:guid}/retry")]
|
||||||
public async Task<IActionResult> RetryJob(Guid id)
|
public async Task<IActionResult> RetryJob(Guid id)
|
||||||
@@ -221,7 +299,8 @@ public class JobsController : Controller
|
|||||||
job.Status = JobStatus.Pending;
|
job.Status = JobStatus.Pending;
|
||||||
job.RetryCount = 0;
|
job.RetryCount = 0;
|
||||||
job.LastError = null;
|
job.LastError = null;
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
|
|
||||||
await _db.SaveChangesAsync();
|
await _db.SaveChangesAsync();
|
||||||
|
|
||||||
@@ -268,7 +347,8 @@ public class JobsController : Controller
|
|||||||
|
|
||||||
job.Status = JobStatus.Failed;
|
job.Status = JobStatus.Failed;
|
||||||
job.LastError = "Cancelled by user";
|
job.LastError = "Cancelled by user";
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
|
|
||||||
await _db.SaveChangesAsync();
|
await _db.SaveChangesAsync();
|
||||||
|
|
||||||
@@ -399,10 +479,9 @@ public class JobsController : Controller
|
|||||||
MaxRetries = maxRetries,
|
MaxRetries = maxRetries,
|
||||||
Status = JobStatus.Pending,
|
Status = JobStatus.Pending,
|
||||||
CreatedAt = DateTime.UtcNow,
|
CreatedAt = DateTime.UtcNow,
|
||||||
CreatedAtUtc = DateTime.UtcNow,
|
ModifiedAt = DateTime.UtcNow,
|
||||||
ModifiedAtUtc = DateTime.UtcNow,
|
CreatedById = DiunaBI.Domain.Entities.User.AutoImportUserId,
|
||||||
CreatedById = Guid.Empty,
|
ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId
|
||||||
ModifiedById = Guid.Empty
|
|
||||||
};
|
};
|
||||||
|
|
||||||
_db.QueueJobs.Add(job);
|
_db.QueueJobs.Add(job);
|
||||||
|
|||||||
@@ -28,5 +28,17 @@
|
|||||||
],
|
],
|
||||||
"Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"]
|
"Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"]
|
||||||
},
|
},
|
||||||
"AllowedHosts": "*"
|
"AllowedHosts": "*",
|
||||||
|
"AnomalyDetection": {
|
||||||
|
"Provider": "OpenAI",
|
||||||
|
"Model": "gpt-4o-mini",
|
||||||
|
"ApiKey": "",
|
||||||
|
"Endpoint": "",
|
||||||
|
"MaxTokens": 4000,
|
||||||
|
"Temperature": 0.1,
|
||||||
|
"MinHistoricalImports": 5,
|
||||||
|
"RecentImportsWindow": 5,
|
||||||
|
"MonthlyImportsWindow": 5,
|
||||||
|
"ConfidenceThreshold": 0.7
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -25,7 +25,8 @@ public enum LayerType
|
|||||||
Import,
|
Import,
|
||||||
Processed,
|
Processed,
|
||||||
Administration,
|
Administration,
|
||||||
Dictionary
|
Dictionary,
|
||||||
|
Validation
|
||||||
}
|
}
|
||||||
|
|
||||||
public class LayerFilterRequest
|
public class LayerFilterRequest
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ public enum LayerType
|
|||||||
Processed,
|
Processed,
|
||||||
Administration,
|
Administration,
|
||||||
Dictionary,
|
Dictionary,
|
||||||
|
Validation,
|
||||||
}
|
}
|
||||||
public class Layer
|
public class Layer
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ public class QueueJob
|
|||||||
public JobType JobType { get; set; }
|
public JobType JobType { get; set; }
|
||||||
public int Priority { get; set; } = 0; // 0 = highest priority
|
public int Priority { get; set; } = 0; // 0 = highest priority
|
||||||
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
|
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
|
||||||
|
public DateTime ModifiedAt { get; set; } = DateTime.UtcNow;
|
||||||
public int RetryCount { get; set; } = 0;
|
public int RetryCount { get; set; } = 0;
|
||||||
public int MaxRetries { get; set; } = 5;
|
public int MaxRetries { get; set; } = 5;
|
||||||
public JobStatus Status { get; set; } = JobStatus.Pending;
|
public JobStatus Status { get; set; } = JobStatus.Pending;
|
||||||
@@ -19,15 +20,14 @@ public class QueueJob
|
|||||||
public DateTime? LastAttemptAt { get; set; }
|
public DateTime? LastAttemptAt { get; set; }
|
||||||
public DateTime? CompletedAt { get; set; }
|
public DateTime? CompletedAt { get; set; }
|
||||||
public Guid CreatedById { get; set; }
|
public Guid CreatedById { get; set; }
|
||||||
public DateTime CreatedAtUtc { get; set; } = DateTime.UtcNow;
|
|
||||||
public Guid ModifiedById { get; set; }
|
public Guid ModifiedById { get; set; }
|
||||||
public DateTime ModifiedAtUtc { get; set; } = DateTime.UtcNow;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public enum JobType
|
public enum JobType
|
||||||
{
|
{
|
||||||
Import = 0,
|
Import = 0,
|
||||||
Process = 1
|
Process = 1,
|
||||||
|
Validate = 2
|
||||||
}
|
}
|
||||||
|
|
||||||
public enum JobStatus
|
public enum JobStatus
|
||||||
|
|||||||
@@ -5,6 +5,11 @@ namespace DiunaBI.Domain.Entities;
|
|||||||
|
|
||||||
public class User
|
public class User
|
||||||
{
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// System user ID for automated operations (imports, scheduled jobs, etc.)
|
||||||
|
/// </summary>
|
||||||
|
public static readonly Guid AutoImportUserId = Guid.Parse("f392209e-123e-4651-a5a4-0b1d6cf9ff9d");
|
||||||
|
|
||||||
#region Properties
|
#region Properties
|
||||||
public Guid Id { get; init; }
|
public Guid Id { get; init; }
|
||||||
public string? Email { get; init; }
|
public string? Email { get; init; }
|
||||||
|
|||||||
@@ -136,9 +136,8 @@ public class AppDbContext(DbContextOptions<AppDbContext> options) : DbContext(op
|
|||||||
modelBuilder.Entity<QueueJob>().Property(x => x.LastAttemptAt);
|
modelBuilder.Entity<QueueJob>().Property(x => x.LastAttemptAt);
|
||||||
modelBuilder.Entity<QueueJob>().Property(x => x.CompletedAt);
|
modelBuilder.Entity<QueueJob>().Property(x => x.CompletedAt);
|
||||||
modelBuilder.Entity<QueueJob>().Property(x => x.CreatedById).IsRequired();
|
modelBuilder.Entity<QueueJob>().Property(x => x.CreatedById).IsRequired();
|
||||||
modelBuilder.Entity<QueueJob>().Property(x => x.CreatedAtUtc).IsRequired();
|
|
||||||
modelBuilder.Entity<QueueJob>().Property(x => x.ModifiedById).IsRequired();
|
modelBuilder.Entity<QueueJob>().Property(x => x.ModifiedById).IsRequired();
|
||||||
modelBuilder.Entity<QueueJob>().Property(x => x.ModifiedAtUtc).IsRequired();
|
modelBuilder.Entity<QueueJob>().Property(x => x.ModifiedAt).IsRequired();
|
||||||
|
|
||||||
// Configure automatic timestamps for entities with CreatedAt/ModifiedAt
|
// Configure automatic timestamps for entities with CreatedAt/ModifiedAt
|
||||||
ConfigureTimestamps(modelBuilder);
|
ConfigureTimestamps(modelBuilder);
|
||||||
|
|||||||
@@ -22,8 +22,8 @@
|
|||||||
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.0" />
|
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.0" />
|
||||||
<PackageReference Include="Google.Apis.Sheets.v4" Version="1.68.0.3525" />
|
<PackageReference Include="Google.Apis.Sheets.v4" Version="1.68.0.3525" />
|
||||||
<PackageReference Include="Google.Apis.Drive.v3" Version="1.68.0.3490" />
|
<PackageReference Include="Google.Apis.Drive.v3" Version="1.68.0.3490" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="10.0.0" />
|
<PackageReference Include="Microsoft.SemanticKernel" Version="1.68.0" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.0" />
|
<PackageReference Include="Microsoft.SemanticKernel.Connectors.Ollama" Version="1.68.0-alpha" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
|
|||||||
11
DiunaBI.Infrastructure/Interfaces/IDataValidator.cs
Normal file
11
DiunaBI.Infrastructure/Interfaces/IDataValidator.cs
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
using DiunaBI.Domain.Entities;
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Interfaces;
|
||||||
|
|
||||||
|
public interface IDataValidator
|
||||||
|
{
|
||||||
|
string ValidatorType { get; }
|
||||||
|
|
||||||
|
bool CanValidate(string validatorType);
|
||||||
|
void Validate(Layer validationWorker);
|
||||||
|
}
|
||||||
489
DiunaBI.Infrastructure/Migrations/20251208205202_RemoveQueueJobDuplicateUTCFields.Designer.cs
generated
Normal file
489
DiunaBI.Infrastructure/Migrations/20251208205202_RemoveQueueJobDuplicateUTCFields.Designer.cs
generated
Normal file
@@ -0,0 +1,489 @@
|
|||||||
|
// <auto-generated />
|
||||||
|
using System;
|
||||||
|
using DiunaBI.Infrastructure.Data;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||||
|
using Microsoft.EntityFrameworkCore.Metadata;
|
||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Migrations
|
||||||
|
{
|
||||||
|
[DbContext(typeof(AppDbContext))]
|
||||||
|
[Migration("20251208205202_RemoveQueueJobDuplicateUTCFields")]
|
||||||
|
partial class RemoveQueueJobDuplicateUTCFields
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||||
|
{
|
||||||
|
#pragma warning disable 612, 618
|
||||||
|
modelBuilder
|
||||||
|
.HasAnnotation("ProductVersion", "10.0.0")
|
||||||
|
.HasAnnotation("Relational:MaxIdentifierLength", 128);
|
||||||
|
|
||||||
|
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.DataInbox", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<string>("Data")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("Source")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("DataInbox");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<bool>("IsCancelled")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("bit")
|
||||||
|
.HasDefaultValue(false);
|
||||||
|
|
||||||
|
b.Property<bool>("IsDeleted")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("bit")
|
||||||
|
.HasDefaultValue(false);
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<int>("Number")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<Guid?>("ParentId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("Type")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("CreatedById");
|
||||||
|
|
||||||
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
|
b.ToTable("Layers");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<Guid>("SourceId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.HasKey("LayerId", "SourceId");
|
||||||
|
|
||||||
|
b.HasIndex("SourceId");
|
||||||
|
|
||||||
|
b.ToTable("ProcessSources");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.QueueJob", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime?>("CompletedAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("JobType")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime?>("LastAttemptAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<string>("LastError")
|
||||||
|
.HasMaxLength(1000)
|
||||||
|
.HasColumnType("nvarchar(1000)");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("LayerName")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(200)
|
||||||
|
.HasColumnType("nvarchar(200)");
|
||||||
|
|
||||||
|
b.Property<int>("MaxRetries")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("PluginName")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(100)
|
||||||
|
.HasColumnType("nvarchar(100)");
|
||||||
|
|
||||||
|
b.Property<int>("Priority")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<int>("RetryCount")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<int>("Status")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("QueueJobs");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Code")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Desc1")
|
||||||
|
.HasMaxLength(10000)
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<bool>("IsDeleted")
|
||||||
|
.HasColumnType("bit");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<double?>("Value1")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value10")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value11")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value12")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value13")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value14")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value15")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value16")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value17")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value18")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value19")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value2")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value20")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value21")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value22")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value23")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value24")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value25")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value26")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value27")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value28")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value29")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value3")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value30")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value31")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value32")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value4")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value5")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value6")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value7")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value8")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value9")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("CreatedById");
|
||||||
|
|
||||||
|
b.HasIndex("LayerId");
|
||||||
|
|
||||||
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
|
b.ToTable("Records");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("ChangeType")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ChangedAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<Guid>("ChangedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("ChangedFields")
|
||||||
|
.HasMaxLength(200)
|
||||||
|
.HasColumnType("nvarchar(200)");
|
||||||
|
|
||||||
|
b.Property<string>("ChangesSummary")
|
||||||
|
.HasMaxLength(4000)
|
||||||
|
.HasColumnType("nvarchar(4000)");
|
||||||
|
|
||||||
|
b.Property<string>("Code")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("Desc1")
|
||||||
|
.HasMaxLength(10000)
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<Guid>("RecordId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("ChangedById");
|
||||||
|
|
||||||
|
b.HasIndex("LayerId", "ChangedAt");
|
||||||
|
|
||||||
|
b.HasIndex("RecordId", "ChangedAt");
|
||||||
|
|
||||||
|
b.ToTable("RecordHistory");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.User", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<string>("Email")
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("UserName")
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("Users");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "CreatedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("CreatedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ModifiedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ModifiedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("CreatedBy");
|
||||||
|
|
||||||
|
b.Navigation("ModifiedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", null)
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("LayerId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", "Source")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("SourceId")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("Source");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "CreatedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("CreatedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", null)
|
||||||
|
.WithMany("Records")
|
||||||
|
.HasForeignKey("LayerId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ModifiedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ModifiedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("CreatedBy");
|
||||||
|
|
||||||
|
b.Navigation("ModifiedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ChangedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ChangedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("ChangedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.Navigation("Records");
|
||||||
|
});
|
||||||
|
#pragma warning restore 612, 618
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
using System;
|
||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Migrations
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
public partial class RemoveQueueJobDuplicateUTCFields : Migration
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Up(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
migrationBuilder.DropColumn(
|
||||||
|
name: "CreatedAtUtc",
|
||||||
|
table: "QueueJobs");
|
||||||
|
|
||||||
|
migrationBuilder.DropColumn(
|
||||||
|
name: "ModifiedAtUtc",
|
||||||
|
table: "QueueJobs");
|
||||||
|
|
||||||
|
migrationBuilder.AddColumn<DateTime>(
|
||||||
|
name: "ModifiedAt",
|
||||||
|
table: "QueueJobs",
|
||||||
|
type: "datetime2",
|
||||||
|
nullable: false,
|
||||||
|
defaultValueSql: "GETUTCDATE()");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Down(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
migrationBuilder.DropColumn(
|
||||||
|
name: "ModifiedAt",
|
||||||
|
table: "QueueJobs");
|
||||||
|
|
||||||
|
migrationBuilder.AddColumn<DateTime>(
|
||||||
|
name: "CreatedAtUtc",
|
||||||
|
table: "QueueJobs",
|
||||||
|
type: "datetime2",
|
||||||
|
nullable: false,
|
||||||
|
defaultValue: new DateTime(1, 1, 1, 0, 0, 0, 0, DateTimeKind.Unspecified));
|
||||||
|
|
||||||
|
migrationBuilder.AddColumn<DateTime>(
|
||||||
|
name: "ModifiedAtUtc",
|
||||||
|
table: "QueueJobs",
|
||||||
|
type: "datetime2",
|
||||||
|
nullable: false,
|
||||||
|
defaultValue: new DateTime(1, 1, 1, 0, 0, 0, 0, DateTimeKind.Unspecified));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
489
DiunaBI.Infrastructure/Migrations/20251214143012_AddValidationLayerAndJobTypes.Designer.cs
generated
Normal file
489
DiunaBI.Infrastructure/Migrations/20251214143012_AddValidationLayerAndJobTypes.Designer.cs
generated
Normal file
@@ -0,0 +1,489 @@
|
|||||||
|
// <auto-generated />
|
||||||
|
using System;
|
||||||
|
using DiunaBI.Infrastructure.Data;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||||
|
using Microsoft.EntityFrameworkCore.Metadata;
|
||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Migrations
|
||||||
|
{
|
||||||
|
[DbContext(typeof(AppDbContext))]
|
||||||
|
[Migration("20251214143012_AddValidationLayerAndJobTypes")]
|
||||||
|
partial class AddValidationLayerAndJobTypes
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||||
|
{
|
||||||
|
#pragma warning disable 612, 618
|
||||||
|
modelBuilder
|
||||||
|
.HasAnnotation("ProductVersion", "10.0.0")
|
||||||
|
.HasAnnotation("Relational:MaxIdentifierLength", 128);
|
||||||
|
|
||||||
|
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.DataInbox", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<string>("Data")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("Source")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("DataInbox");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<bool>("IsCancelled")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("bit")
|
||||||
|
.HasDefaultValue(false);
|
||||||
|
|
||||||
|
b.Property<bool>("IsDeleted")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("bit")
|
||||||
|
.HasDefaultValue(false);
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<int>("Number")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<Guid?>("ParentId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("Type")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("CreatedById");
|
||||||
|
|
||||||
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
|
b.ToTable("Layers");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<Guid>("SourceId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.HasKey("LayerId", "SourceId");
|
||||||
|
|
||||||
|
b.HasIndex("SourceId");
|
||||||
|
|
||||||
|
b.ToTable("ProcessSources");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.QueueJob", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime?>("CompletedAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("JobType")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime?>("LastAttemptAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<string>("LastError")
|
||||||
|
.HasMaxLength(1000)
|
||||||
|
.HasColumnType("nvarchar(1000)");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("LayerName")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(200)
|
||||||
|
.HasColumnType("nvarchar(200)");
|
||||||
|
|
||||||
|
b.Property<int>("MaxRetries")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("PluginName")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(100)
|
||||||
|
.HasColumnType("nvarchar(100)");
|
||||||
|
|
||||||
|
b.Property<int>("Priority")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<int>("RetryCount")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<int>("Status")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("QueueJobs");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Code")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("CreatedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Desc1")
|
||||||
|
.HasMaxLength(10000)
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<bool>("IsDeleted")
|
||||||
|
.HasColumnType("bit");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ModifiedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<Guid>("ModifiedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<double?>("Value1")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value10")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value11")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value12")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value13")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value14")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value15")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value16")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value17")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value18")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value19")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value2")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value20")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value21")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value22")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value23")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value24")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value25")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value26")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value27")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value28")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value29")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value3")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value30")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value31")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value32")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value4")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value5")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value6")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value7")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value8")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.Property<double?>("Value9")
|
||||||
|
.HasColumnType("float");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("CreatedById");
|
||||||
|
|
||||||
|
b.HasIndex("LayerId");
|
||||||
|
|
||||||
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
|
b.ToTable("Records");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<int>("ChangeType")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.Property<DateTime>("ChangedAt")
|
||||||
|
.HasColumnType("datetime2");
|
||||||
|
|
||||||
|
b.Property<Guid>("ChangedById")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("ChangedFields")
|
||||||
|
.HasMaxLength(200)
|
||||||
|
.HasColumnType("nvarchar(200)");
|
||||||
|
|
||||||
|
b.Property<string>("ChangesSummary")
|
||||||
|
.HasMaxLength(4000)
|
||||||
|
.HasColumnType("nvarchar(4000)");
|
||||||
|
|
||||||
|
b.Property<string>("Code")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("Desc1")
|
||||||
|
.HasMaxLength(10000)
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<Guid>("LayerId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<Guid>("RecordId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("ChangedById");
|
||||||
|
|
||||||
|
b.HasIndex("LayerId", "ChangedAt");
|
||||||
|
|
||||||
|
b.HasIndex("RecordId", "ChangedAt");
|
||||||
|
|
||||||
|
b.ToTable("RecordHistory");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.User", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTime>("CreatedAt")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
|
b.Property<string>("Email")
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.Property<string>("UserName")
|
||||||
|
.HasMaxLength(50)
|
||||||
|
.HasColumnType("nvarchar(50)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("Users");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "CreatedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("CreatedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ModifiedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ModifiedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("CreatedBy");
|
||||||
|
|
||||||
|
b.Navigation("ModifiedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", null)
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("LayerId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", "Source")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("SourceId")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("Source");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "CreatedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("CreatedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.Layer", null)
|
||||||
|
.WithMany("Records")
|
||||||
|
.HasForeignKey("LayerId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ModifiedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ModifiedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("CreatedBy");
|
||||||
|
|
||||||
|
b.Navigation("ModifiedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("DiunaBI.Domain.Entities.User", "ChangedBy")
|
||||||
|
.WithMany()
|
||||||
|
.HasForeignKey("ChangedById")
|
||||||
|
.OnDelete(DeleteBehavior.Restrict)
|
||||||
|
.IsRequired();
|
||||||
|
|
||||||
|
b.Navigation("ChangedBy");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
{
|
||||||
|
b.Navigation("Records");
|
||||||
|
});
|
||||||
|
#pragma warning restore 612, 618
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Migrations
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
public partial class AddValidationLayerAndJobTypes : Migration
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Up(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Down(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -49,7 +49,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasKey("Id");
|
b.HasKey("Id");
|
||||||
|
|
||||||
b.ToTable("DataInbox", (string)null);
|
b.ToTable("DataInbox");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
@@ -104,7 +104,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasIndex("ModifiedById");
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
b.ToTable("Layers", (string)null);
|
b.ToTable("Layers");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.ProcessSource", b =>
|
||||||
@@ -119,7 +119,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasIndex("SourceId");
|
b.HasIndex("SourceId");
|
||||||
|
|
||||||
b.ToTable("ProcessSources", (string)null);
|
b.ToTable("ProcessSources");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.QueueJob", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.QueueJob", b =>
|
||||||
@@ -136,9 +136,6 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
.HasColumnType("datetime2")
|
.HasColumnType("datetime2")
|
||||||
.HasDefaultValueSql("GETUTCDATE()");
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
b.Property<DateTime>("CreatedAtUtc")
|
|
||||||
.HasColumnType("datetime2");
|
|
||||||
|
|
||||||
b.Property<Guid>("CreatedById")
|
b.Property<Guid>("CreatedById")
|
||||||
.HasColumnType("uniqueidentifier");
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
@@ -163,8 +160,10 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
b.Property<int>("MaxRetries")
|
b.Property<int>("MaxRetries")
|
||||||
.HasColumnType("int");
|
.HasColumnType("int");
|
||||||
|
|
||||||
b.Property<DateTime>("ModifiedAtUtc")
|
b.Property<DateTime>("ModifiedAt")
|
||||||
.HasColumnType("datetime2");
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("datetime2")
|
||||||
|
.HasDefaultValueSql("GETUTCDATE()");
|
||||||
|
|
||||||
b.Property<Guid>("ModifiedById")
|
b.Property<Guid>("ModifiedById")
|
||||||
.HasColumnType("uniqueidentifier");
|
.HasColumnType("uniqueidentifier");
|
||||||
@@ -185,7 +184,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasKey("Id");
|
b.HasKey("Id");
|
||||||
|
|
||||||
b.ToTable("QueueJobs", (string)null);
|
b.ToTable("QueueJobs");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Record", b =>
|
||||||
@@ -329,7 +328,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasIndex("ModifiedById");
|
b.HasIndex("ModifiedById");
|
||||||
|
|
||||||
b.ToTable("Records", (string)null);
|
b.ToTable("Records");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.RecordHistory", b =>
|
||||||
@@ -378,7 +377,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasIndex("RecordId", "ChangedAt");
|
b.HasIndex("RecordId", "ChangedAt");
|
||||||
|
|
||||||
b.ToTable("RecordHistory", (string)null);
|
b.ToTable("RecordHistory");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.User", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.User", b =>
|
||||||
@@ -402,7 +401,7 @@ namespace DiunaBI.Infrastructure.Migrations
|
|||||||
|
|
||||||
b.HasKey("Id");
|
b.HasKey("Id");
|
||||||
|
|
||||||
b.ToTable("Users", (string)null);
|
b.ToTable("Users");
|
||||||
});
|
});
|
||||||
|
|
||||||
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
modelBuilder.Entity("DiunaBI.Domain.Entities.Layer", b =>
|
||||||
|
|||||||
21
DiunaBI.Infrastructure/Plugins/BaseDataValidator.cs
Normal file
21
DiunaBI.Infrastructure/Plugins/BaseDataValidator.cs
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
using DiunaBI.Domain.Entities;
|
||||||
|
using DiunaBI.Infrastructure.Interfaces;
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Plugins;
|
||||||
|
|
||||||
|
public abstract class BaseDataValidator : IDataValidator
|
||||||
|
{
|
||||||
|
public abstract string ValidatorType { get; }
|
||||||
|
|
||||||
|
public virtual bool CanValidate(string validatorType) => ValidatorType == validatorType;
|
||||||
|
|
||||||
|
public abstract void Validate(Layer validationWorker);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Helper method to get record value by code from layer records
|
||||||
|
/// </summary>
|
||||||
|
protected string? GetRecordValue(ICollection<Record> records, string code)
|
||||||
|
{
|
||||||
|
return records.FirstOrDefault(x => x.Code == code)?.Desc1;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -40,6 +40,7 @@ public class JobSchedulerService
|
|||||||
_logger.LogInformation("JobScheduler: Found {Count} import workers to schedule", importWorkers.Count);
|
_logger.LogInformation("JobScheduler: Found {Count} import workers to schedule", importWorkers.Count);
|
||||||
|
|
||||||
var jobsCreated = 0;
|
var jobsCreated = 0;
|
||||||
|
var scheduledLayerIds = new HashSet<Guid>(); // Track LayerIds scheduled in this batch
|
||||||
|
|
||||||
foreach (var worker in importWorkers)
|
foreach (var worker in importWorkers)
|
||||||
{
|
{
|
||||||
@@ -61,7 +62,15 @@ public class JobSchedulerService
|
|||||||
var maxRetriesStr = worker.Records?.FirstOrDefault(r => r.Code == "MaxRetries")?.Desc1;
|
var maxRetriesStr = worker.Records?.FirstOrDefault(r => r.Code == "MaxRetries")?.Desc1;
|
||||||
var maxRetries = int.TryParse(maxRetriesStr, out var mr) ? mr : 3;
|
var maxRetries = int.TryParse(maxRetriesStr, out var mr) ? mr : 3;
|
||||||
|
|
||||||
// Check if there's already a pending/running job for this layer
|
// Check in-memory: already scheduled in this batch?
|
||||||
|
if (scheduledLayerIds.Contains(worker.Id))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("JobScheduler: Job already scheduled in this batch for {LayerName} ({LayerId})",
|
||||||
|
worker.Name, worker.Id);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if there's already a pending/running job for this layer in database
|
||||||
var existingJob = await _db.QueueJobs
|
var existingJob = await _db.QueueJobs
|
||||||
.Where(j => j.LayerId == worker.Id &&
|
.Where(j => j.LayerId == worker.Id &&
|
||||||
(j.Status == JobStatus.Pending || j.Status == JobStatus.Running))
|
(j.Status == JobStatus.Pending || j.Status == JobStatus.Running))
|
||||||
@@ -85,13 +94,13 @@ public class JobSchedulerService
|
|||||||
MaxRetries = maxRetries,
|
MaxRetries = maxRetries,
|
||||||
Status = JobStatus.Pending,
|
Status = JobStatus.Pending,
|
||||||
CreatedAt = DateTime.UtcNow,
|
CreatedAt = DateTime.UtcNow,
|
||||||
CreatedAtUtc = DateTime.UtcNow,
|
ModifiedAt = DateTime.UtcNow,
|
||||||
ModifiedAtUtc = DateTime.UtcNow,
|
CreatedById = DiunaBI.Domain.Entities.User.AutoImportUserId,
|
||||||
CreatedById = Guid.Empty, // System user
|
ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId
|
||||||
ModifiedById = Guid.Empty
|
|
||||||
};
|
};
|
||||||
|
|
||||||
_db.QueueJobs.Add(job);
|
_db.QueueJobs.Add(job);
|
||||||
|
scheduledLayerIds.Add(worker.Id); // Track that we've scheduled this layer
|
||||||
jobsCreated++;
|
jobsCreated++;
|
||||||
|
|
||||||
_logger.LogInformation("JobScheduler: Created import job for {LayerName} ({LayerId}) with priority {Priority}",
|
_logger.LogInformation("JobScheduler: Created import job for {LayerName} ({LayerId}) with priority {Priority}",
|
||||||
@@ -130,6 +139,7 @@ public class JobSchedulerService
|
|||||||
_logger.LogInformation("JobScheduler: Found {Count} process workers to schedule", processWorkers.Count);
|
_logger.LogInformation("JobScheduler: Found {Count} process workers to schedule", processWorkers.Count);
|
||||||
|
|
||||||
var jobsCreated = 0;
|
var jobsCreated = 0;
|
||||||
|
var scheduledLayerIds = new HashSet<Guid>(); // Track LayerIds scheduled in this batch
|
||||||
|
|
||||||
foreach (var worker in processWorkers)
|
foreach (var worker in processWorkers)
|
||||||
{
|
{
|
||||||
@@ -151,7 +161,15 @@ public class JobSchedulerService
|
|||||||
var maxRetriesStr = worker.Records?.FirstOrDefault(r => r.Code == "MaxRetries")?.Desc1;
|
var maxRetriesStr = worker.Records?.FirstOrDefault(r => r.Code == "MaxRetries")?.Desc1;
|
||||||
var maxRetries = int.TryParse(maxRetriesStr, out var mr) ? mr : 3;
|
var maxRetries = int.TryParse(maxRetriesStr, out var mr) ? mr : 3;
|
||||||
|
|
||||||
// Check if there's already a pending/running job for this layer
|
// Check in-memory: already scheduled in this batch?
|
||||||
|
if (scheduledLayerIds.Contains(worker.Id))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("JobScheduler: Job already scheduled in this batch for {LayerName} ({LayerId})",
|
||||||
|
worker.Name, worker.Id);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if there's already a pending/running job for this layer in database
|
||||||
var existingJob = await _db.QueueJobs
|
var existingJob = await _db.QueueJobs
|
||||||
.Where(j => j.LayerId == worker.Id &&
|
.Where(j => j.LayerId == worker.Id &&
|
||||||
(j.Status == JobStatus.Pending || j.Status == JobStatus.Running))
|
(j.Status == JobStatus.Pending || j.Status == JobStatus.Running))
|
||||||
@@ -175,13 +193,13 @@ public class JobSchedulerService
|
|||||||
MaxRetries = maxRetries,
|
MaxRetries = maxRetries,
|
||||||
Status = JobStatus.Pending,
|
Status = JobStatus.Pending,
|
||||||
CreatedAt = DateTime.UtcNow,
|
CreatedAt = DateTime.UtcNow,
|
||||||
CreatedAtUtc = DateTime.UtcNow,
|
ModifiedAt = DateTime.UtcNow,
|
||||||
ModifiedAtUtc = DateTime.UtcNow,
|
CreatedById = DiunaBI.Domain.Entities.User.AutoImportUserId,
|
||||||
CreatedById = Guid.Empty,
|
ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId
|
||||||
ModifiedById = Guid.Empty
|
|
||||||
};
|
};
|
||||||
|
|
||||||
_db.QueueJobs.Add(job);
|
_db.QueueJobs.Add(job);
|
||||||
|
scheduledLayerIds.Add(worker.Id); // Track that we've scheduled this layer
|
||||||
jobsCreated++;
|
jobsCreated++;
|
||||||
|
|
||||||
_logger.LogInformation("JobScheduler: Created process job for {LayerName} ({LayerId}) with priority {Priority}",
|
_logger.LogInformation("JobScheduler: Created process job for {LayerName} ({LayerId}) with priority {Priority}",
|
||||||
@@ -203,14 +221,114 @@ public class JobSchedulerService
|
|||||||
return jobsCreated;
|
return jobsCreated;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public async Task<int> ScheduleValidateJobsAsync()
|
||||||
|
{
|
||||||
|
_logger.LogInformation("JobScheduler: Starting validation job scheduling");
|
||||||
|
|
||||||
|
var validationWorkers = await _db.Layers
|
||||||
|
.Include(x => x.Records)
|
||||||
|
.Where(x =>
|
||||||
|
x.Records!.Any(r => r.Code == "Type" && r.Desc1 == "ValidationWorker") &&
|
||||||
|
x.Records!.Any(r => r.Code == "IsEnabled" && r.Desc1 == "True")
|
||||||
|
)
|
||||||
|
.OrderBy(x => x.CreatedAt)
|
||||||
|
.AsNoTracking()
|
||||||
|
.ToListAsync();
|
||||||
|
|
||||||
|
_logger.LogInformation("JobScheduler: Found {Count} validation workers to schedule", validationWorkers.Count);
|
||||||
|
|
||||||
|
var jobsCreated = 0;
|
||||||
|
var scheduledLayerIds = new HashSet<Guid>(); // Track LayerIds scheduled in this batch
|
||||||
|
|
||||||
|
foreach (var worker in validationWorkers)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var plugin = worker.Records?.FirstOrDefault(r => r.Code == "Plugin")?.Desc1;
|
||||||
|
if (string.IsNullOrEmpty(plugin))
|
||||||
|
{
|
||||||
|
_logger.LogWarning("JobScheduler: Validation worker {LayerName} ({LayerId}) has no Plugin configured, skipping",
|
||||||
|
worker.Name, worker.Id);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get priority from config (default: 200 for validation - lower than processes)
|
||||||
|
var priorityStr = worker.Records?.FirstOrDefault(r => r.Code == "Priority")?.Desc1;
|
||||||
|
var priority = int.TryParse(priorityStr, out var p) ? p : 200;
|
||||||
|
|
||||||
|
// Get max retries from config (default: 3)
|
||||||
|
var maxRetriesStr = worker.Records?.FirstOrDefault(r => r.Code == "MaxRetries")?.Desc1;
|
||||||
|
var maxRetries = int.TryParse(maxRetriesStr, out var mr) ? mr : 3;
|
||||||
|
|
||||||
|
// Check in-memory: already scheduled in this batch?
|
||||||
|
if (scheduledLayerIds.Contains(worker.Id))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("JobScheduler: Job already scheduled in this batch for {LayerName} ({LayerId})",
|
||||||
|
worker.Name, worker.Id);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if there's already a pending/running job for this layer in database
|
||||||
|
var existingJob = await _db.QueueJobs
|
||||||
|
.Where(j => j.LayerId == worker.Id &&
|
||||||
|
(j.Status == JobStatus.Pending || j.Status == JobStatus.Running))
|
||||||
|
.FirstOrDefaultAsync();
|
||||||
|
|
||||||
|
if (existingJob != null)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("JobScheduler: Job already exists for {LayerName} ({LayerId}), status: {Status}",
|
||||||
|
worker.Name, worker.Id, existingJob.Status);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var job = new QueueJob
|
||||||
|
{
|
||||||
|
Id = Guid.NewGuid(),
|
||||||
|
LayerId = worker.Id,
|
||||||
|
LayerName = worker.Name ?? "Unknown",
|
||||||
|
PluginName = plugin,
|
||||||
|
JobType = JobType.Validate,
|
||||||
|
Priority = priority,
|
||||||
|
MaxRetries = maxRetries,
|
||||||
|
Status = JobStatus.Pending,
|
||||||
|
CreatedAt = DateTime.UtcNow,
|
||||||
|
ModifiedAt = DateTime.UtcNow,
|
||||||
|
CreatedById = DiunaBI.Domain.Entities.User.AutoImportUserId,
|
||||||
|
ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId
|
||||||
|
};
|
||||||
|
|
||||||
|
_db.QueueJobs.Add(job);
|
||||||
|
scheduledLayerIds.Add(worker.Id); // Track that we've scheduled this layer
|
||||||
|
jobsCreated++;
|
||||||
|
|
||||||
|
_logger.LogInformation("JobScheduler: Created validation job for {LayerName} ({LayerId}) with priority {Priority}",
|
||||||
|
worker.Name, worker.Id, priority);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "JobScheduler: Failed to create job for {LayerName} ({LayerId})",
|
||||||
|
worker.Name, worker.Id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (jobsCreated > 0)
|
||||||
|
{
|
||||||
|
await _db.SaveChangesAsync();
|
||||||
|
_logger.LogInformation("JobScheduler: Successfully created {Count} validation jobs", jobsCreated);
|
||||||
|
}
|
||||||
|
|
||||||
|
return jobsCreated;
|
||||||
|
}
|
||||||
|
|
||||||
public async Task<int> ScheduleAllJobsAsync(string? nameFilter = null)
|
public async Task<int> ScheduleAllJobsAsync(string? nameFilter = null)
|
||||||
{
|
{
|
||||||
var importCount = await ScheduleImportJobsAsync(nameFilter);
|
var importCount = await ScheduleImportJobsAsync(nameFilter);
|
||||||
var processCount = await ScheduleProcessJobsAsync();
|
var processCount = await ScheduleProcessJobsAsync();
|
||||||
|
var validateCount = await ScheduleValidateJobsAsync();
|
||||||
|
|
||||||
_logger.LogInformation("JobScheduler: Scheduled {ImportCount} import jobs and {ProcessCount} process jobs",
|
_logger.LogInformation("JobScheduler: Scheduled {ImportCount} import jobs, {ProcessCount} process jobs, and {ValidateCount} validation jobs",
|
||||||
importCount, processCount);
|
importCount, processCount, validateCount);
|
||||||
|
|
||||||
return importCount + processCount;
|
return importCount + processCount + validateCount;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ public class JobWorkerService : BackgroundService
|
|||||||
{
|
{
|
||||||
private readonly IServiceProvider _serviceProvider;
|
private readonly IServiceProvider _serviceProvider;
|
||||||
private readonly ILogger<JobWorkerService> _logger;
|
private readonly ILogger<JobWorkerService> _logger;
|
||||||
private readonly TimeSpan _pollInterval = TimeSpan.FromSeconds(10);
|
private readonly TimeSpan _pollInterval = TimeSpan.FromSeconds(5);
|
||||||
private readonly TimeSpan _rateLimitDelay = TimeSpan.FromSeconds(5);
|
private readonly TimeSpan _rateLimitDelay = TimeSpan.FromSeconds(5);
|
||||||
|
|
||||||
public JobWorkerService(IServiceProvider serviceProvider, ILogger<JobWorkerService> logger)
|
public JobWorkerService(IServiceProvider serviceProvider, ILogger<JobWorkerService> logger)
|
||||||
@@ -66,7 +66,8 @@ public class JobWorkerService : BackgroundService
|
|||||||
// Mark job as running
|
// Mark job as running
|
||||||
job.Status = JobStatus.Running;
|
job.Status = JobStatus.Running;
|
||||||
job.LastAttemptAt = DateTime.UtcNow;
|
job.LastAttemptAt = DateTime.UtcNow;
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
await db.SaveChangesAsync(stoppingToken);
|
await db.SaveChangesAsync(stoppingToken);
|
||||||
|
|
||||||
try
|
try
|
||||||
@@ -109,12 +110,26 @@ public class JobWorkerService : BackgroundService
|
|||||||
|
|
||||||
processor.Process(layer);
|
processor.Process(layer);
|
||||||
}
|
}
|
||||||
|
else if (job.JobType == JobType.Validate)
|
||||||
|
{
|
||||||
|
var validator = pluginManager.GetValidator(job.PluginName);
|
||||||
|
if (validator == null)
|
||||||
|
{
|
||||||
|
throw new Exception($"Validator '{job.PluginName}' not found");
|
||||||
|
}
|
||||||
|
|
||||||
|
_logger.LogInformation("JobWorker: Executing validation for {LayerName} using {PluginName}",
|
||||||
|
job.LayerName, job.PluginName);
|
||||||
|
|
||||||
|
validator.Validate(layer);
|
||||||
|
}
|
||||||
|
|
||||||
// Job completed successfully
|
// Job completed successfully
|
||||||
job.Status = JobStatus.Completed;
|
job.Status = JobStatus.Completed;
|
||||||
job.CompletedAt = DateTime.UtcNow;
|
job.CompletedAt = DateTime.UtcNow;
|
||||||
job.LastError = null;
|
job.LastError = null;
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
|
|
||||||
_logger.LogInformation("JobWorker: Job {JobId} completed successfully", job.Id);
|
_logger.LogInformation("JobWorker: Job {JobId} completed successfully", job.Id);
|
||||||
|
|
||||||
@@ -131,7 +146,8 @@ public class JobWorkerService : BackgroundService
|
|||||||
|
|
||||||
// Capture full error details including inner exceptions
|
// Capture full error details including inner exceptions
|
||||||
job.LastError = GetFullErrorMessage(ex);
|
job.LastError = GetFullErrorMessage(ex);
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
|
|
||||||
if (job.RetryCount >= job.MaxRetries)
|
if (job.RetryCount >= job.MaxRetries)
|
||||||
{
|
{
|
||||||
@@ -157,7 +173,8 @@ public class JobWorkerService : BackgroundService
|
|||||||
|
|
||||||
// Increment retry count for next attempt
|
// Increment retry count for next attempt
|
||||||
job.RetryCount++;
|
job.RetryCount++;
|
||||||
job.ModifiedAtUtc = DateTime.UtcNow;
|
job.ModifiedAt = DateTime.UtcNow;
|
||||||
|
job.ModifiedById = DiunaBI.Domain.Entities.User.AutoImportUserId;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
finally
|
finally
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ public class PluginManager
|
|||||||
private readonly List<Type> _processorTypes = new();
|
private readonly List<Type> _processorTypes = new();
|
||||||
private readonly List<Type> _importerTypes = new();
|
private readonly List<Type> _importerTypes = new();
|
||||||
private readonly List<Type> _exporterTypes = new();
|
private readonly List<Type> _exporterTypes = new();
|
||||||
|
private readonly List<Type> _validatorTypes = new();
|
||||||
private readonly List<IPlugin> _plugins = new();
|
private readonly List<IPlugin> _plugins = new();
|
||||||
|
|
||||||
public PluginManager(ILogger<PluginManager> logger, IServiceProvider serviceProvider)
|
public PluginManager(ILogger<PluginManager> logger, IServiceProvider serviceProvider)
|
||||||
@@ -42,10 +43,11 @@ public class PluginManager
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
_logger.LogInformation("Loaded {ProcessorCount} processors, {ImporterCount} importers, and {ExporterCount} exporters from {AssemblyCount} assemblies",
|
_logger.LogInformation("Loaded {ProcessorCount} processors, {ImporterCount} importers, {ExporterCount} exporters, and {ValidatorCount} validators from {AssemblyCount} assemblies",
|
||||||
_processorTypes.Count,
|
_processorTypes.Count,
|
||||||
_importerTypes.Count,
|
_importerTypes.Count,
|
||||||
_exporterTypes.Count,
|
_exporterTypes.Count,
|
||||||
|
_validatorTypes.Count,
|
||||||
dllFiles.Length);
|
dllFiles.Length);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -77,6 +79,12 @@ public class PluginManager
|
|||||||
_exporterTypes.Add(type);
|
_exporterTypes.Add(type);
|
||||||
_logger.LogDebug("Registered exporter: {Type}", type.Name);
|
_logger.LogDebug("Registered exporter: {Type}", type.Name);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (typeof(IDataValidator).IsAssignableFrom(type) && !type.IsInterface && !type.IsAbstract)
|
||||||
|
{
|
||||||
|
_validatorTypes.Add(type);
|
||||||
|
_logger.LogDebug("Registered validator: {Type}", type.Name);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
@@ -157,5 +165,29 @@ public class PluginManager
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
public int GetPluginsCount() => _processorTypes.Count + _importerTypes.Count + _exporterTypes.Count;
|
public IDataValidator? GetValidator(string validatorType)
|
||||||
|
{
|
||||||
|
foreach (var type in _validatorTypes)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var scope = _serviceProvider.CreateScope();
|
||||||
|
var instance = (IDataValidator)ActivatorUtilities.CreateInstance(scope.ServiceProvider, type);
|
||||||
|
|
||||||
|
if (instance.CanValidate(validatorType))
|
||||||
|
{
|
||||||
|
return instance;
|
||||||
|
}
|
||||||
|
|
||||||
|
scope.Dispose();
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "Failed to create validator instance of type {Type}", type.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
public int GetPluginsCount() => _processorTypes.Count + _importerTypes.Count + _exporterTypes.Count + _validatorTypes.Count;
|
||||||
}
|
}
|
||||||
496
DiunaBI.Infrastructure/Validators/LlmAnomalyValidator.cs
Normal file
496
DiunaBI.Infrastructure/Validators/LlmAnomalyValidator.cs
Normal file
@@ -0,0 +1,496 @@
|
|||||||
|
using System.Text.Json;
|
||||||
|
using DiunaBI.Domain.Entities;
|
||||||
|
using DiunaBI.Infrastructure.Data;
|
||||||
|
using DiunaBI.Infrastructure.Plugins;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.SemanticKernel;
|
||||||
|
using Microsoft.SemanticKernel.ChatCompletion;
|
||||||
|
using Microsoft.SemanticKernel.Connectors.OpenAI;
|
||||||
|
using Microsoft.SemanticKernel.Connectors.Ollama;
|
||||||
|
|
||||||
|
namespace DiunaBI.Infrastructure.Validators;
|
||||||
|
|
||||||
|
public class LlmAnomalyValidator : BaseDataValidator
|
||||||
|
{
|
||||||
|
public override string ValidatorType => "LlmAnomalyValidator";
|
||||||
|
|
||||||
|
private readonly AppDbContext _db;
|
||||||
|
private readonly IConfiguration _config;
|
||||||
|
private readonly ILogger<LlmAnomalyValidator> _logger;
|
||||||
|
private readonly Kernel _kernel;
|
||||||
|
|
||||||
|
// Configuration loaded from appsettings.json
|
||||||
|
private readonly string _provider;
|
||||||
|
private readonly string _model;
|
||||||
|
private readonly int _minHistoricalImports;
|
||||||
|
private readonly int _recentImportsWindow;
|
||||||
|
private readonly int _monthlyImportsWindow;
|
||||||
|
private readonly double _confidenceThreshold;
|
||||||
|
|
||||||
|
// Configuration loaded from ValidationWorker records
|
||||||
|
private string? SourceLayerName { get; set; }
|
||||||
|
private Layer? SourceImportWorker { get; set; }
|
||||||
|
|
||||||
|
public LlmAnomalyValidator(
|
||||||
|
AppDbContext db,
|
||||||
|
IConfiguration config,
|
||||||
|
ILogger<LlmAnomalyValidator> logger)
|
||||||
|
{
|
||||||
|
_db = db;
|
||||||
|
_config = config;
|
||||||
|
_logger = logger;
|
||||||
|
|
||||||
|
// Load configuration from appsettings.json
|
||||||
|
_provider = config["AnomalyDetection:Provider"] ?? "OpenAI";
|
||||||
|
_model = config["AnomalyDetection:Model"] ?? "gpt-4o-mini";
|
||||||
|
_minHistoricalImports = int.Parse(config["AnomalyDetection:MinHistoricalImports"] ?? "5");
|
||||||
|
_recentImportsWindow = int.Parse(config["AnomalyDetection:RecentImportsWindow"] ?? "5");
|
||||||
|
_monthlyImportsWindow = int.Parse(config["AnomalyDetection:MonthlyImportsWindow"] ?? "5");
|
||||||
|
_confidenceThreshold = double.Parse(config["AnomalyDetection:ConfidenceThreshold"] ?? "0.7");
|
||||||
|
|
||||||
|
// Initialize Semantic Kernel based on provider
|
||||||
|
_kernel = InitializeKernel();
|
||||||
|
|
||||||
|
_logger.LogInformation("LlmAnomalyValidator initialized with provider: {Provider}, model: {Model}",
|
||||||
|
_provider, _model);
|
||||||
|
}
|
||||||
|
|
||||||
|
private Kernel InitializeKernel()
|
||||||
|
{
|
||||||
|
var builder = Kernel.CreateBuilder();
|
||||||
|
|
||||||
|
switch (_provider.ToLower())
|
||||||
|
{
|
||||||
|
case "openai":
|
||||||
|
var openAiKey = _config["AnomalyDetection:ApiKey"];
|
||||||
|
if (string.IsNullOrEmpty(openAiKey))
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("OpenAI API key not configured");
|
||||||
|
}
|
||||||
|
builder.AddOpenAIChatCompletion(_model, openAiKey);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case "azureopenai":
|
||||||
|
var azureEndpoint = _config["AnomalyDetection:Endpoint"];
|
||||||
|
var azureKey = _config["AnomalyDetection:ApiKey"];
|
||||||
|
if (string.IsNullOrEmpty(azureEndpoint) || string.IsNullOrEmpty(azureKey))
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("Azure OpenAI endpoint or API key not configured");
|
||||||
|
}
|
||||||
|
builder.AddAzureOpenAIChatCompletion(_model, azureEndpoint, azureKey);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case "ollama":
|
||||||
|
var ollamaEndpoint = _config["AnomalyDetection:Endpoint"] ?? "http://localhost:11434";
|
||||||
|
builder.AddOllamaChatCompletion(_model, new Uri(ollamaEndpoint));
|
||||||
|
break;
|
||||||
|
|
||||||
|
default:
|
||||||
|
throw new NotSupportedException($"LLM provider '{_provider}' is not supported");
|
||||||
|
}
|
||||||
|
|
||||||
|
return builder.Build();
|
||||||
|
}
|
||||||
|
|
||||||
|
public override void Validate(Layer validationWorker)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
_logger.LogInformation("{ValidatorType}: Starting validation for {ValidationWorkerName} ({ValidationWorkerId})",
|
||||||
|
ValidatorType, validationWorker.Name, validationWorker.Id);
|
||||||
|
|
||||||
|
// Load configuration from layer records
|
||||||
|
LoadConfiguration(validationWorker);
|
||||||
|
|
||||||
|
// Validate configuration
|
||||||
|
ValidateConfiguration();
|
||||||
|
|
||||||
|
// Find latest import layer
|
||||||
|
var latestImport = GetLatestImportLayer();
|
||||||
|
|
||||||
|
// Get historical context
|
||||||
|
var historicalImports = GetHistoricalImports();
|
||||||
|
|
||||||
|
// Check if enough historical data
|
||||||
|
if (historicalImports.Count < _minHistoricalImports)
|
||||||
|
{
|
||||||
|
_logger.LogWarning("{ValidatorType}: Not enough historical imports: {Count} (need {Min}). Skipping validation.",
|
||||||
|
ValidatorType, historicalImports.Count, _minHistoricalImports);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Perform validation
|
||||||
|
PerformValidation(validationWorker, latestImport, historicalImports);
|
||||||
|
|
||||||
|
_logger.LogInformation("{ValidatorType}: Successfully completed validation for {ValidationWorkerName}",
|
||||||
|
ValidatorType, validationWorker.Name);
|
||||||
|
}
|
||||||
|
catch (Exception e)
|
||||||
|
{
|
||||||
|
_logger.LogError(e, "{ValidatorType}: Failed to validate {ValidationWorkerName} ({ValidationWorkerId})",
|
||||||
|
ValidatorType, validationWorker.Name, validationWorker.Id);
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void LoadConfiguration(Layer validationWorker)
|
||||||
|
{
|
||||||
|
if (validationWorker.Records == null)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("ValidationWorker has no records");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load source layer name (ImportWorker Administration Layer)
|
||||||
|
SourceLayerName = GetRecordValue(validationWorker.Records, "SourceLayer");
|
||||||
|
if (string.IsNullOrEmpty(SourceLayerName))
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException("SourceLayer record not found");
|
||||||
|
}
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Configuration loaded - SourceLayer: {SourceLayer}",
|
||||||
|
ValidatorType, SourceLayerName);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void ValidateConfiguration()
|
||||||
|
{
|
||||||
|
var errors = new List<string>();
|
||||||
|
|
||||||
|
if (string.IsNullOrEmpty(SourceLayerName)) errors.Add("SourceLayer is required");
|
||||||
|
|
||||||
|
// Find source import worker (Administration Layer)
|
||||||
|
SourceImportWorker = _db.Layers
|
||||||
|
.SingleOrDefault(x => x.Name == SourceLayerName &&
|
||||||
|
x.Type == LayerType.Administration &&
|
||||||
|
!x.IsDeleted &&
|
||||||
|
!x.IsCancelled);
|
||||||
|
|
||||||
|
if (SourceImportWorker == null)
|
||||||
|
{
|
||||||
|
errors.Add($"SourceImportWorker layer '{SourceLayerName}' not found");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (errors.Any())
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException($"Configuration validation failed: {string.Join(", ", errors)}");
|
||||||
|
}
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Configuration validation passed", ValidatorType);
|
||||||
|
}
|
||||||
|
|
||||||
|
private Layer GetLatestImportLayer()
|
||||||
|
{
|
||||||
|
// Find latest Import layer where ParentId = SourceImportWorker.Id
|
||||||
|
var latestImport = _db.Layers
|
||||||
|
.Include(x => x.Records)
|
||||||
|
.Where(x => x.ParentId == SourceImportWorker!.Id &&
|
||||||
|
x.Type == LayerType.Import &&
|
||||||
|
!x.IsDeleted &&
|
||||||
|
!x.IsCancelled)
|
||||||
|
.OrderByDescending(x => x.CreatedAt)
|
||||||
|
.FirstOrDefault();
|
||||||
|
|
||||||
|
if (latestImport == null)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"No import layers found for import worker '{SourceImportWorker!.Name}'");
|
||||||
|
}
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Found latest import layer: {LayerName} ({LayerId})",
|
||||||
|
ValidatorType, latestImport.Name, latestImport.Id);
|
||||||
|
|
||||||
|
return latestImport;
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<Layer> GetHistoricalImports()
|
||||||
|
{
|
||||||
|
// Get last N import layers (ordered by CreatedAt)
|
||||||
|
var historicalImports = _db.Layers
|
||||||
|
.Include(x => x.Records)
|
||||||
|
.Where(x => x.ParentId == SourceImportWorker!.Id &&
|
||||||
|
x.Type == LayerType.Import &&
|
||||||
|
!x.IsDeleted &&
|
||||||
|
!x.IsCancelled)
|
||||||
|
.OrderByDescending(x => x.CreatedAt)
|
||||||
|
.Take(_recentImportsWindow)
|
||||||
|
.AsNoTracking()
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Found {Count} historical imports for recent window",
|
||||||
|
ValidatorType, historicalImports.Count);
|
||||||
|
|
||||||
|
return historicalImports;
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<Layer> GetMonthlyBaselineImports()
|
||||||
|
{
|
||||||
|
// Get last N "first-of-month" import layers
|
||||||
|
var monthlyImports = _db.Layers
|
||||||
|
.Include(x => x.Records)
|
||||||
|
.Where(x => x.ParentId == SourceImportWorker!.Id &&
|
||||||
|
x.Type == LayerType.Import &&
|
||||||
|
x.CreatedAt.Day == 1 &&
|
||||||
|
!x.IsDeleted &&
|
||||||
|
!x.IsCancelled)
|
||||||
|
.OrderByDescending(x => x.CreatedAt)
|
||||||
|
.Take(_monthlyImportsWindow)
|
||||||
|
.AsNoTracking()
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Found {Count} monthly baseline imports",
|
||||||
|
ValidatorType, monthlyImports.Count);
|
||||||
|
|
||||||
|
return monthlyImports;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void PerformValidation(Layer validationWorker, Layer latestImport, List<Layer> historicalImports)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("{ValidatorType}: Performing validation for import: {ImportName}",
|
||||||
|
ValidatorType, latestImport.Name);
|
||||||
|
|
||||||
|
// Get monthly baseline if available
|
||||||
|
var monthlyBaseline = GetMonthlyBaselineImports();
|
||||||
|
|
||||||
|
// Build prompt with all data
|
||||||
|
var prompt = BuildPrompt(latestImport, historicalImports, monthlyBaseline);
|
||||||
|
|
||||||
|
// Call LLM
|
||||||
|
var startTime = DateTime.UtcNow;
|
||||||
|
var llmResponse = CallLlm(prompt);
|
||||||
|
var processingTime = DateTime.UtcNow - startTime;
|
||||||
|
|
||||||
|
// Create Validation Layer with results
|
||||||
|
var validationLayer = CreateValidationLayer(validationWorker, latestImport, llmResponse, processingTime);
|
||||||
|
|
||||||
|
// Save to database
|
||||||
|
SaveValidationLayer(validationLayer, llmResponse);
|
||||||
|
|
||||||
|
_logger.LogInformation("{ValidatorType}: Created validation layer {LayerName} ({LayerId}) in {ProcessingTime}ms",
|
||||||
|
ValidatorType, validationLayer.Name, validationLayer.Id, processingTime.TotalMilliseconds);
|
||||||
|
}
|
||||||
|
|
||||||
|
private string BuildPrompt(Layer currentImport, List<Layer> recentImports, List<Layer> monthlyBaseline)
|
||||||
|
{
|
||||||
|
var currentRecords = currentImport.Records?.OrderBy(r => r.Code).ToList() ?? new List<Record>();
|
||||||
|
var importType = SourceImportWorker?.Name ?? "Unknown";
|
||||||
|
|
||||||
|
var prompt = $@"You are a data quality analyst specializing in anomaly detection for business intelligence imports.
|
||||||
|
|
||||||
|
**Import Type:** {importType}
|
||||||
|
**Import Date:** {currentImport.CreatedAt:yyyy-MM-dd HH:mm:ss}
|
||||||
|
**Current Import:** {currentImport.Name}
|
||||||
|
|
||||||
|
**Current Import Data ({currentRecords.Count} records):**
|
||||||
|
{JsonSerializer.Serialize(currentRecords.Select(r => new { code = r.Code, value1 = r.Value1 }), new JsonSerializerOptions { WriteIndented = true })}
|
||||||
|
|
||||||
|
**Historical Context - Last {recentImports.Count} Imports:**
|
||||||
|
{string.Join("\n", recentImports.Select((imp, idx) => $"Import {idx + 1} ({imp.CreatedAt:yyyy-MM-dd}): {JsonSerializer.Serialize(imp.Records?.OrderBy(r => r.Code).Select(r => new { code = r.Code, value1 = r.Value1 }) ?? Enumerable.Empty<object>())}"))}
|
||||||
|
";
|
||||||
|
|
||||||
|
if (monthlyBaseline.Any())
|
||||||
|
{
|
||||||
|
prompt += $@"
|
||||||
|
**Monthly Baseline - Last {monthlyBaseline.Count} First-Day Imports:**
|
||||||
|
{string.Join("\n", monthlyBaseline.Select((imp, idx) => $"Monthly Import {idx + 1} ({imp.CreatedAt:yyyy-MM-dd}): {JsonSerializer.Serialize(imp.Records?.OrderBy(r => r.Code).Select(r => new { code = r.Code, value1 = r.Value1 }) ?? Enumerable.Empty<object>())}"))}
|
||||||
|
";
|
||||||
|
}
|
||||||
|
|
||||||
|
prompt += @"
|
||||||
|
**Analysis Tasks:**
|
||||||
|
1. **Record-level anomalies:** Identify unusual values for specific codes compared to historical patterns
|
||||||
|
2. **Structural issues:** Detect missing codes, new codes, or unexpected count changes
|
||||||
|
3. **Pattern breaks:** Find trend reversals, unexpected correlations, or statistical outliers
|
||||||
|
|
||||||
|
**Response Format (JSON):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
""overallStatus"": ""pass|warning|critical"",
|
||||||
|
""recordAnomalies"": [
|
||||||
|
{
|
||||||
|
""code"": ""string"",
|
||||||
|
""value1"": number,
|
||||||
|
""confidence"": 0.0-1.0,
|
||||||
|
""severity"": ""low|medium|high|critical"",
|
||||||
|
""reason"": ""brief explanation"",
|
||||||
|
""recommendation"": ""suggested action""
|
||||||
|
}
|
||||||
|
],
|
||||||
|
""structuralIssues"": [
|
||||||
|
{
|
||||||
|
""issueType"": ""missing_codes|new_codes|count_change"",
|
||||||
|
""description"": ""string"",
|
||||||
|
""codes"": [""code1"", ""code2""],
|
||||||
|
""severity"": ""low|medium|high|critical""
|
||||||
|
}
|
||||||
|
],
|
||||||
|
""summary"": ""Brief overall assessment""
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Analyze the data and respond ONLY with the JSON object. Do not include any markdown formatting or additional text.";
|
||||||
|
|
||||||
|
return prompt;
|
||||||
|
}
|
||||||
|
|
||||||
|
private AnomalyResponse CallLlm(string prompt)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var chatService = _kernel.GetRequiredService<IChatCompletionService>();
|
||||||
|
|
||||||
|
var chatHistory = new ChatHistory();
|
||||||
|
chatHistory.AddUserMessage(prompt);
|
||||||
|
|
||||||
|
var result = chatService.GetChatMessageContentAsync(
|
||||||
|
chatHistory,
|
||||||
|
new OpenAIPromptExecutionSettings
|
||||||
|
{
|
||||||
|
Temperature = _config.GetValue<double?>("AnomalyDetection:Temperature") ?? 0.1,
|
||||||
|
MaxTokens = _config.GetValue<int?>("AnomalyDetection:MaxTokens") ?? 4000
|
||||||
|
}).GetAwaiter().GetResult();
|
||||||
|
|
||||||
|
var jsonResponse = result.Content?.Trim() ?? "{}";
|
||||||
|
|
||||||
|
// Try to parse JSON response
|
||||||
|
try
|
||||||
|
{
|
||||||
|
return JsonSerializer.Deserialize<AnomalyResponse>(jsonResponse)
|
||||||
|
?? throw new InvalidOperationException("LLM returned null response");
|
||||||
|
}
|
||||||
|
catch (JsonException)
|
||||||
|
{
|
||||||
|
_logger.LogWarning("Failed to parse LLM response as JSON. Raw response: {Response}", jsonResponse);
|
||||||
|
throw new InvalidOperationException($"LLM did not return valid JSON. Response: {jsonResponse}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_logger.LogError(ex, "Failed to call LLM for anomaly detection");
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private Layer CreateValidationLayer(Layer validationWorker, Layer importLayer, AnomalyResponse response, TimeSpan processingTime)
|
||||||
|
{
|
||||||
|
var layerNumber = _db.Layers.Count() + 1;
|
||||||
|
var timestamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss");
|
||||||
|
|
||||||
|
var validationLayer = new Layer
|
||||||
|
{
|
||||||
|
Id = Guid.NewGuid(),
|
||||||
|
Type = LayerType.Validation,
|
||||||
|
ParentId = importLayer.Id, // Links to the import that was validated
|
||||||
|
Number = layerNumber,
|
||||||
|
Name = $"L{layerNumber}-V-{timestamp}",
|
||||||
|
CreatedById = User.AutoImportUserId,
|
||||||
|
ModifiedById = User.AutoImportUserId,
|
||||||
|
CreatedAt = DateTime.UtcNow,
|
||||||
|
ModifiedAt = DateTime.UtcNow
|
||||||
|
};
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Created validation layer {LayerName}",
|
||||||
|
ValidatorType, validationLayer.Name);
|
||||||
|
|
||||||
|
return validationLayer;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void SaveValidationLayer(Layer validationLayer, AnomalyResponse response)
|
||||||
|
{
|
||||||
|
// Add the validation layer
|
||||||
|
_db.Layers.Add(validationLayer);
|
||||||
|
|
||||||
|
var records = new List<Record>();
|
||||||
|
|
||||||
|
// Add metadata records
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "ValidatedAt", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss")));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "OverallStatus", response.OverallStatus));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "RecordsChecked", value1: response.RecordAnomalies?.Count ?? 0));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "AnomaliesDetected", value1: response.RecordAnomalies?.Count ?? 0));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "StructuralIssuesDetected", value1: response.StructuralIssues?.Count ?? 0));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "LlmProvider", _provider));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "LlmModel", _model));
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "Summary", response.Summary));
|
||||||
|
|
||||||
|
// Add individual anomaly records
|
||||||
|
if (response.RecordAnomalies != null)
|
||||||
|
{
|
||||||
|
foreach (var anomaly in response.RecordAnomalies)
|
||||||
|
{
|
||||||
|
records.Add(CreateRecord(
|
||||||
|
validationLayer.Id,
|
||||||
|
$"ANOMALY_{anomaly.Code}",
|
||||||
|
$"[{anomaly.Severity}] {anomaly.Reason}. Recommendation: {anomaly.Recommendation}",
|
||||||
|
anomaly.Confidence
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add structural issue records
|
||||||
|
if (response.StructuralIssues != null)
|
||||||
|
{
|
||||||
|
foreach (var issue in response.StructuralIssues)
|
||||||
|
{
|
||||||
|
var codes = issue.Codes != null ? string.Join(", ", issue.Codes) : "";
|
||||||
|
records.Add(CreateRecord(
|
||||||
|
validationLayer.Id,
|
||||||
|
$"STRUCTURAL_{issue.IssueType?.ToUpper()}",
|
||||||
|
$"[{issue.Severity}] {issue.Description}. Codes: {codes}"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store full LLM response as JSON (for debugging)
|
||||||
|
records.Add(CreateRecord(validationLayer.Id, "LLM_RESPONSE_JSON", JsonSerializer.Serialize(response)));
|
||||||
|
|
||||||
|
// Add all records to database
|
||||||
|
_db.Records.AddRange(records);
|
||||||
|
_db.SaveChanges();
|
||||||
|
|
||||||
|
_logger.LogDebug("{ValidatorType}: Saved {RecordCount} records for validation layer {LayerId}",
|
||||||
|
ValidatorType, records.Count, validationLayer.Id);
|
||||||
|
}
|
||||||
|
|
||||||
|
private Record CreateRecord(Guid layerId, string code, string? desc1 = null, double? value1 = null)
|
||||||
|
{
|
||||||
|
return new Record
|
||||||
|
{
|
||||||
|
Id = Guid.NewGuid(),
|
||||||
|
LayerId = layerId,
|
||||||
|
Code = code,
|
||||||
|
Desc1 = desc1,
|
||||||
|
Value1 = value1,
|
||||||
|
CreatedById = User.AutoImportUserId,
|
||||||
|
ModifiedById = User.AutoImportUserId,
|
||||||
|
CreatedAt = DateTime.UtcNow,
|
||||||
|
ModifiedAt = DateTime.UtcNow
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Response models for LLM
|
||||||
|
public class AnomalyResponse
|
||||||
|
{
|
||||||
|
public string OverallStatus { get; set; } = "pass";
|
||||||
|
public List<RecordAnomaly>? RecordAnomalies { get; set; }
|
||||||
|
public List<StructuralIssue>? StructuralIssues { get; set; }
|
||||||
|
public string Summary { get; set; } = "";
|
||||||
|
}
|
||||||
|
|
||||||
|
public class RecordAnomaly
|
||||||
|
{
|
||||||
|
public string Code { get; set; } = "";
|
||||||
|
public double? Value1 { get; set; }
|
||||||
|
public double Confidence { get; set; }
|
||||||
|
public string Severity { get; set; } = "low";
|
||||||
|
public string Reason { get; set; } = "";
|
||||||
|
public string Recommendation { get; set; } = "";
|
||||||
|
}
|
||||||
|
|
||||||
|
public class StructuralIssue
|
||||||
|
{
|
||||||
|
public string? IssueType { get; set; }
|
||||||
|
public string Description { get; set; } = "";
|
||||||
|
public List<string>? Codes { get; set; }
|
||||||
|
public string Severity { get; set; } = "low";
|
||||||
|
}
|
||||||
@@ -2,36 +2,31 @@
|
|||||||
@using DiunaBI.UI.Shared.Services
|
@using DiunaBI.UI.Shared.Services
|
||||||
@inject AppConfig AppConfig
|
@inject AppConfig AppConfig
|
||||||
@inject EntityChangeHubService HubService
|
@inject EntityChangeHubService HubService
|
||||||
|
@inject AuthService AuthService
|
||||||
@inherits LayoutComponentBase
|
@inherits LayoutComponentBase
|
||||||
|
@implements IDisposable
|
||||||
|
|
||||||
<AuthGuard>
|
<AuthGuard>
|
||||||
<MudThemeProvider Theme="_theme"/>
|
<MudThemeProvider Theme="_theme" />
|
||||||
<MudPopoverProvider/>
|
<MudPopoverProvider />
|
||||||
<MudDialogProvider/>
|
<MudDialogProvider />
|
||||||
<MudSnackbarProvider/>
|
<MudSnackbarProvider />
|
||||||
|
|
||||||
<MudLayout>
|
<MudLayout>
|
||||||
<MudBreakpointProvider OnBreakpointChanged="OnBreakpointChanged"></MudBreakpointProvider>
|
<MudBreakpointProvider OnBreakpointChanged="OnBreakpointChanged"></MudBreakpointProvider>
|
||||||
<MudAppBar Elevation="0">
|
<MudAppBar Elevation="0">
|
||||||
<MudIconButton
|
<MudIconButton Icon="@Icons.Material.Filled.Menu" Color="Color.Inherit" Edge="Edge.Start"
|
||||||
Icon="@Icons.Material.Filled.Menu"
|
OnClick="ToggleDrawer" Class="mud-hidden-md-up" />
|
||||||
Color="Color.Inherit"
|
<MudSpacer />
|
||||||
Edge="Edge.Start"
|
|
||||||
OnClick="ToggleDrawer"
|
|
||||||
Class="mud-hidden-md-up"/>
|
|
||||||
<MudSpacer/>
|
|
||||||
<MudText Typo="Typo.h6">@AppConfig.AppName</MudText>
|
<MudText Typo="Typo.h6">@AppConfig.AppName</MudText>
|
||||||
</MudAppBar>
|
</MudAppBar>
|
||||||
|
|
||||||
<MudDrawer @bind-Open="_drawerOpen"
|
<MudDrawer @bind-Open="_drawerOpen" Anchor="Anchor.Start" Variant="@_drawerVariant" Elevation="1"
|
||||||
Anchor="Anchor.Start"
|
ClipMode="DrawerClipMode.Always" Class="mud-width-250">
|
||||||
Variant="@_drawerVariant"
|
|
||||||
Elevation="1"
|
|
||||||
ClipMode="DrawerClipMode.Always"
|
|
||||||
Class="mud-width-250">
|
|
||||||
<div class="nav-logo" style="text-align: center; padding: 20px;">
|
<div class="nav-logo" style="text-align: center; padding: 20px;">
|
||||||
<a href="https://www.diunabi.com" target="_blank">
|
<a href="https://www.diunabi.com" target="_blank">
|
||||||
<img src="_content/DiunaBI.UI.Shared/images/logo.png" alt="DiunaBI" style="max-width: 180px; height: auto;" />
|
<img src="_content/DiunaBI.UI.Shared/images/logo.png" alt="DiunaBI"
|
||||||
|
style="max-width: 180px; height: auto;" />
|
||||||
</a>
|
</a>
|
||||||
</div>
|
</div>
|
||||||
<MudNavMenu>
|
<MudNavMenu>
|
||||||
@@ -40,6 +35,10 @@
|
|||||||
<MudNavLink Href="/datainbox" Icon="@Icons.Material.Filled.Inbox">Data Inbox</MudNavLink>
|
<MudNavLink Href="/datainbox" Icon="@Icons.Material.Filled.Inbox">Data Inbox</MudNavLink>
|
||||||
<MudNavLink Href="/jobs" Icon="@Icons.Material.Filled.WorkHistory">Jobs</MudNavLink>
|
<MudNavLink Href="/jobs" Icon="@Icons.Material.Filled.WorkHistory">Jobs</MudNavLink>
|
||||||
</MudNavMenu>
|
</MudNavMenu>
|
||||||
|
<div class="nav-logo" style="text-align: center; padding: 20px;">
|
||||||
|
<img src="_content/DiunaBI.UI.Shared/images/clients/@AppConfig.ClientLogo" alt="DiunaBI"
|
||||||
|
style="max-width: 180px; height: auto;" />
|
||||||
|
</div>
|
||||||
</MudDrawer>
|
</MudDrawer>
|
||||||
|
|
||||||
<MudMainContent>
|
<MudMainContent>
|
||||||
@@ -55,10 +54,30 @@
|
|||||||
private bool _drawerOpen = true;
|
private bool _drawerOpen = true;
|
||||||
private DrawerVariant _drawerVariant = DrawerVariant.Persistent;
|
private DrawerVariant _drawerVariant = DrawerVariant.Persistent;
|
||||||
|
|
||||||
protected override async Task OnInitializedAsync()
|
protected override void OnInitialized()
|
||||||
{
|
{
|
||||||
// Initialize SignalR connection when layout loads
|
// Subscribe to authentication state changes
|
||||||
await HubService.InitializeAsync();
|
AuthService.AuthenticationStateChanged += OnAuthenticationStateChanged;
|
||||||
|
|
||||||
|
// If already authenticated (e.g., from restored session), initialize SignalR
|
||||||
|
if (AuthService.IsAuthenticated)
|
||||||
|
{
|
||||||
|
_ = HubService.InitializeAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async void OnAuthenticationStateChanged(bool isAuthenticated)
|
||||||
|
{
|
||||||
|
if (isAuthenticated)
|
||||||
|
{
|
||||||
|
Console.WriteLine("🔐 MainLayout: User authenticated, initializing SignalR...");
|
||||||
|
await HubService.InitializeAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
AuthService.AuthenticationStateChanged -= OnAuthenticationStateChanged;
|
||||||
}
|
}
|
||||||
|
|
||||||
private MudTheme _theme = new MudTheme()
|
private MudTheme _theme = new MudTheme()
|
||||||
|
|||||||
@@ -39,20 +39,21 @@ public static class ServiceCollectionExtensions
|
|||||||
});
|
});
|
||||||
|
|
||||||
// Services
|
// Services
|
||||||
|
services.AddScoped<TokenProvider>();
|
||||||
services.AddScoped<AuthService>();
|
services.AddScoped<AuthService>();
|
||||||
services.AddScoped<LayerService>();
|
services.AddScoped<LayerService>();
|
||||||
services.AddScoped<DataInboxService>();
|
services.AddScoped<DataInboxService>();
|
||||||
services.AddScoped<JobService>();
|
services.AddScoped<JobService>();
|
||||||
|
services.AddScoped<DateTimeHelper>();
|
||||||
|
services.AddScoped<NumberFormatHelper>();
|
||||||
|
|
||||||
// Filter state services (scoped to maintain state during user session)
|
// Filter state services (scoped to maintain state during user session)
|
||||||
services.AddScoped<LayerFilterStateService>();
|
services.AddScoped<LayerFilterStateService>();
|
||||||
services.AddScoped<DataInboxFilterStateService>();
|
services.AddScoped<DataInboxFilterStateService>();
|
||||||
|
|
||||||
// SignalR Hub Service (singleton for global connection shared across all users)
|
// SignalR Hub Service (scoped per user session for authenticated connections)
|
||||||
services.AddSingleton<EntityChangeHubService>(sp =>
|
services.AddScoped(sp =>
|
||||||
{
|
{
|
||||||
// For singleton, we can't inject scoped services directly
|
|
||||||
// We'll get them from the service provider when needed
|
|
||||||
var logger = sp.GetRequiredService<ILogger<EntityChangeHubService>>();
|
var logger = sp.GetRequiredService<ILogger<EntityChangeHubService>>();
|
||||||
var tokenProvider = sp.GetRequiredService<TokenProvider>();
|
var tokenProvider = sp.GetRequiredService<TokenProvider>();
|
||||||
return new EntityChangeHubService(apiBaseUrl, sp, logger, tokenProvider);
|
return new EntityChangeHubService(apiBaseUrl, sp, logger, tokenProvider);
|
||||||
|
|||||||
@@ -24,16 +24,43 @@ public class UnauthorizedResponseHandler : DelegatingHandler
|
|||||||
{
|
{
|
||||||
Console.WriteLine("⚠️ 401 Unauthorized response detected - clearing credentials and redirecting to login");
|
Console.WriteLine("⚠️ 401 Unauthorized response detected - clearing credentials and redirecting to login");
|
||||||
|
|
||||||
// Create a scope to get scoped services
|
try
|
||||||
using var scope = _serviceProvider.CreateScope();
|
{
|
||||||
var authService = scope.ServiceProvider.GetRequiredService<AuthService>();
|
// Create a scope to get scoped services
|
||||||
var navigationManager = scope.ServiceProvider.GetRequiredService<NavigationManager>();
|
using var scope = _serviceProvider.CreateScope();
|
||||||
|
var authService = scope.ServiceProvider.GetRequiredService<AuthService>();
|
||||||
|
var navigationManager = scope.ServiceProvider.GetRequiredService<NavigationManager>();
|
||||||
|
|
||||||
// Clear authentication
|
// Clear authentication
|
||||||
await authService.ClearAuthenticationAsync();
|
await authService.ClearAuthenticationAsync();
|
||||||
|
|
||||||
// Navigate to login page with session expired message
|
// Navigate to login page with session expired message
|
||||||
navigationManager.NavigateTo("/login?sessionExpired=true", forceLoad: true);
|
navigationManager.NavigateTo("/login?sessionExpired=true", forceLoad: true);
|
||||||
|
}
|
||||||
|
catch (InvalidOperationException ex)
|
||||||
|
{
|
||||||
|
// NavigationManager may not be initialized in all contexts (e.g., during initial load)
|
||||||
|
// Log the error and allow the 401 response to propagate to the caller
|
||||||
|
Console.WriteLine($"⚠️ Cannot navigate to login - NavigationManager not initialized: {ex.Message}");
|
||||||
|
Console.WriteLine("⚠️ 401 response will be handled by the calling component");
|
||||||
|
|
||||||
|
// Still try to clear authentication if we can get the AuthService
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using var scope = _serviceProvider.CreateScope();
|
||||||
|
var authService = scope.ServiceProvider.GetRequiredService<AuthService>();
|
||||||
|
await authService.ClearAuthenticationAsync();
|
||||||
|
}
|
||||||
|
catch (Exception clearEx)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"⚠️ Could not clear authentication: {clearEx.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
// Log any other unexpected errors
|
||||||
|
Console.WriteLine($"❌ Error handling 401 response: {ex.Message}");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return response;
|
return response;
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
@page "/datainbox"
|
@page "/datainbox"
|
||||||
@using MudBlazor.Internal
|
@using MudBlazor.Internal
|
||||||
@using DiunaBI.Application.DTOModels
|
@using DiunaBI.Application.DTOModels
|
||||||
|
@implements IDisposable
|
||||||
|
|
||||||
<PageTitle>Data Inbox</PageTitle>
|
<PageTitle>Data Inbox</PageTitle>
|
||||||
|
|
||||||
@@ -52,7 +53,7 @@
|
|||||||
<RowTemplate Context="row">
|
<RowTemplate Context="row">
|
||||||
<MudTd DataLabel="Name"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@row.Name</div></MudTd>
|
<MudTd DataLabel="Name"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@row.Name</div></MudTd>
|
||||||
<MudTd DataLabel="Source"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@row.Source</div></MudTd>
|
<MudTd DataLabel="Source"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@row.Source</div></MudTd>
|
||||||
<MudTd DataLabel="Created At"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@row.CreatedAt.ToString("yyyy-MM-dd HH:mm:ss")</div></MudTd>
|
<MudTd DataLabel="Created At"><div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">@DateTimeHelper.FormatDateTime(row.CreatedAt)</div></MudTd>
|
||||||
</RowTemplate>
|
</RowTemplate>
|
||||||
<NoRecordsContent>
|
<NoRecordsContent>
|
||||||
<MudText>No data inbox items to display</MudText>
|
<MudText>No data inbox items to display</MudText>
|
||||||
|
|||||||
@@ -8,13 +8,15 @@ using Microsoft.JSInterop;
|
|||||||
|
|
||||||
namespace DiunaBI.UI.Shared.Pages.DataInbox;
|
namespace DiunaBI.UI.Shared.Pages.DataInbox;
|
||||||
|
|
||||||
public partial class Index : ComponentBase
|
public partial class Index : ComponentBase, IDisposable
|
||||||
{
|
{
|
||||||
[Inject] private DataInboxService DataInboxService { get; set; } = default!;
|
[Inject] private DataInboxService DataInboxService { get; set; } = default!;
|
||||||
|
[Inject] private EntityChangeHubService HubService { get; set; } = default!;
|
||||||
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
||||||
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
||||||
[Inject] private DataInboxFilterStateService FilterStateService { get; set; } = default!;
|
[Inject] private DataInboxFilterStateService FilterStateService { get; set; } = default!;
|
||||||
[Inject] private IJSRuntime JSRuntime { get; set; } = default!;
|
[Inject] private IJSRuntime JSRuntime { get; set; } = default!;
|
||||||
|
[Inject] private DateTimeHelper DateTimeHelper { get; set; } = default!;
|
||||||
|
|
||||||
|
|
||||||
private PagedResult<DataInboxDto> dataInbox = new();
|
private PagedResult<DataInboxDto> dataInbox = new();
|
||||||
@@ -23,8 +25,25 @@ public partial class Index : ComponentBase
|
|||||||
|
|
||||||
protected override async Task OnInitializedAsync()
|
protected override async Task OnInitializedAsync()
|
||||||
{
|
{
|
||||||
|
await DateTimeHelper.InitializeAsync();
|
||||||
filterRequest = FilterStateService.FilterRequest;
|
filterRequest = FilterStateService.FilterRequest;
|
||||||
await LoadDataInbox();
|
await LoadDataInbox();
|
||||||
|
|
||||||
|
// Subscribe to SignalR entity changes
|
||||||
|
HubService.EntityChanged += OnEntityChanged;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async void OnEntityChanged(string module, string id, string operation)
|
||||||
|
{
|
||||||
|
// Only react if it's a DataInbox change
|
||||||
|
if (module.Equals("DataInbox", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
await InvokeAsync(async () =>
|
||||||
|
{
|
||||||
|
await LoadDataInbox();
|
||||||
|
StateHasChanged();
|
||||||
|
});
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task LoadDataInbox()
|
private async Task LoadDataInbox()
|
||||||
@@ -75,4 +94,9 @@ public partial class Index : ComponentBase
|
|||||||
var url = NavigationManager.ToAbsoluteUri($"/datainbox/{dataInboxItem.Id}").ToString();
|
var url = NavigationManager.ToAbsoluteUri($"/datainbox/{dataInboxItem.Id}").ToString();
|
||||||
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
HubService.EntityChanged -= OnEntityChanged;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,7 @@
|
|||||||
@inject EntityChangeHubService HubService
|
@inject EntityChangeHubService HubService
|
||||||
@inject NavigationManager NavigationManager
|
@inject NavigationManager NavigationManager
|
||||||
@inject ISnackbar Snackbar
|
@inject ISnackbar Snackbar
|
||||||
|
@inject DateTimeHelper DateTimeHelper
|
||||||
@implements IDisposable
|
@implements IDisposable
|
||||||
|
|
||||||
<MudCard>
|
<MudCard>
|
||||||
@@ -92,14 +93,14 @@
|
|||||||
</MudItem>
|
</MudItem>
|
||||||
|
|
||||||
<MudItem xs="12" md="6">
|
<MudItem xs="12" md="6">
|
||||||
<MudTextField Value="@job.CreatedAt.ToString("yyyy-MM-dd HH:mm:ss")"
|
<MudTextField Value="@DateTimeHelper.FormatDateTime(job.CreatedAt)"
|
||||||
Label="Created At"
|
Label="Created At"
|
||||||
Variant="Variant.Outlined"
|
Variant="Variant.Outlined"
|
||||||
ReadOnly="true"
|
ReadOnly="true"
|
||||||
FullWidth="true"/>
|
FullWidth="true"/>
|
||||||
</MudItem>
|
</MudItem>
|
||||||
<MudItem xs="12" md="6">
|
<MudItem xs="12" md="6">
|
||||||
<MudTextField Value="@(job.LastAttemptAt?.ToString("yyyy-MM-dd HH:mm:ss") ?? "-")"
|
<MudTextField Value="@DateTimeHelper.FormatDateTime(job.LastAttemptAt)"
|
||||||
Label="Last Attempt At"
|
Label="Last Attempt At"
|
||||||
Variant="Variant.Outlined"
|
Variant="Variant.Outlined"
|
||||||
ReadOnly="true"
|
ReadOnly="true"
|
||||||
@@ -107,7 +108,7 @@
|
|||||||
</MudItem>
|
</MudItem>
|
||||||
|
|
||||||
<MudItem xs="12" md="6">
|
<MudItem xs="12" md="6">
|
||||||
<MudTextField Value="@(job.CompletedAt?.ToString("yyyy-MM-dd HH:mm:ss") ?? "-")"
|
<MudTextField Value="@DateTimeHelper.FormatDateTime(job.CompletedAt)"
|
||||||
Label="Completed At"
|
Label="Completed At"
|
||||||
Variant="Variant.Outlined"
|
Variant="Variant.Outlined"
|
||||||
ReadOnly="true"
|
ReadOnly="true"
|
||||||
@@ -161,6 +162,7 @@
|
|||||||
|
|
||||||
protected override async Task OnInitializedAsync()
|
protected override async Task OnInitializedAsync()
|
||||||
{
|
{
|
||||||
|
await DateTimeHelper.InitializeAsync();
|
||||||
await LoadJob();
|
await LoadJob();
|
||||||
|
|
||||||
// Subscribe to SignalR entity changes
|
// Subscribe to SignalR entity changes
|
||||||
|
|||||||
@@ -12,25 +12,28 @@
|
|||||||
Expanded="true">
|
Expanded="true">
|
||||||
<MudGrid AlignItems="Center">
|
<MudGrid AlignItems="Center">
|
||||||
<MudItem xs="12" sm="6" md="3">
|
<MudItem xs="12" sm="6" md="3">
|
||||||
<MudSelect T="JobStatus?"
|
<MudSelect T="JobStatus"
|
||||||
@bind-Value="selectedStatus"
|
SelectedValues="selectedStatuses"
|
||||||
Label="Status"
|
Label="Status"
|
||||||
Placeholder="All statuses"
|
Placeholder="All statuses"
|
||||||
|
MultiSelection="true"
|
||||||
Clearable="true"
|
Clearable="true"
|
||||||
|
SelectedValuesChanged="OnStatusFilterChanged"
|
||||||
OnClearButtonClick="OnStatusClear">
|
OnClearButtonClick="OnStatusClear">
|
||||||
@foreach (JobStatus status in Enum.GetValues(typeof(JobStatus)))
|
@foreach (JobStatus status in Enum.GetValues(typeof(JobStatus)))
|
||||||
{
|
{
|
||||||
<MudSelectItem T="JobStatus?" Value="@status">@status.ToString()</MudSelectItem>
|
<MudSelectItem T="JobStatus" Value="@status">@status.ToString()</MudSelectItem>
|
||||||
}
|
}
|
||||||
</MudSelect>
|
</MudSelect>
|
||||||
</MudItem>
|
</MudItem>
|
||||||
|
|
||||||
<MudItem xs="12" sm="6" md="3">
|
<MudItem xs="12" sm="6" md="3">
|
||||||
<MudSelect T="JobType?"
|
<MudSelect T="JobType?"
|
||||||
@bind-Value="selectedJobType"
|
Value="selectedJobType"
|
||||||
Label="Job Type"
|
Label="Job Type"
|
||||||
Placeholder="All types"
|
Placeholder="All types"
|
||||||
Clearable="true"
|
Clearable="true"
|
||||||
|
ValueChanged="OnJobTypeFilterChanged"
|
||||||
OnClearButtonClick="OnJobTypeClear">
|
OnClearButtonClick="OnJobTypeClear">
|
||||||
@foreach (JobType type in Enum.GetValues(typeof(JobType)))
|
@foreach (JobType type in Enum.GetValues(typeof(JobType)))
|
||||||
{
|
{
|
||||||
@@ -39,12 +42,33 @@
|
|||||||
</MudSelect>
|
</MudSelect>
|
||||||
</MudItem>
|
</MudItem>
|
||||||
|
|
||||||
<MudItem xs="12" sm="12" md="6" Class="d-flex justify-end align-center">
|
<MudItem xs="12" sm="12" md="6" Class="d-flex justify-end align-center gap-2">
|
||||||
<MudIconButton Icon="@Icons.Material.Filled.Refresh"
|
<MudMenu Icon="@Icons.Material.Filled.PlayArrow"
|
||||||
OnClick="LoadJobs"
|
Label="Schedule Jobs"
|
||||||
Color="Color.Primary"
|
Variant="Variant.Filled"
|
||||||
Size="Size.Medium"
|
Color="Color.Success"
|
||||||
Title="Refresh"/>
|
Size="Size.Medium"
|
||||||
|
EndIcon="@Icons.Material.Filled.KeyboardArrowDown">
|
||||||
|
<MudMenuItem OnClick="@(() => ScheduleJobs("all"))">
|
||||||
|
<div class="d-flex align-center">
|
||||||
|
<MudIcon Icon="@Icons.Material.Filled.PlayCircle" Class="mr-2" />
|
||||||
|
<span>Run All Jobs</span>
|
||||||
|
</div>
|
||||||
|
</MudMenuItem>
|
||||||
|
<MudMenuItem OnClick="@(() => ScheduleJobs("imports"))">
|
||||||
|
<div class="d-flex align-center">
|
||||||
|
<MudIcon Icon="@Icons.Material.Filled.FileDownload" Class="mr-2" />
|
||||||
|
<span>Run All Imports</span>
|
||||||
|
</div>
|
||||||
|
</MudMenuItem>
|
||||||
|
<MudMenuItem OnClick="@(() => ScheduleJobs("processes"))">
|
||||||
|
<div class="d-flex align-center">
|
||||||
|
<MudIcon Icon="@Icons.Material.Filled.Settings" Class="mr-2" />
|
||||||
|
<span>Run All Processes</span>
|
||||||
|
</div>
|
||||||
|
</MudMenuItem>
|
||||||
|
</MudMenu>
|
||||||
|
|
||||||
<MudIconButton Icon="@Icons.Material.Filled.Clear"
|
<MudIconButton Icon="@Icons.Material.Filled.Clear"
|
||||||
OnClick="ClearFilters"
|
OnClick="ClearFilters"
|
||||||
Color="Color.Default"
|
Color="Color.Default"
|
||||||
@@ -108,12 +132,12 @@
|
|||||||
</MudTd>
|
</MudTd>
|
||||||
<MudTd DataLabel="Created">
|
<MudTd DataLabel="Created">
|
||||||
<div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">
|
<div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">
|
||||||
@row.CreatedAt.ToString("yyyy-MM-dd HH:mm")
|
@DateTimeHelper.FormatDateTime(row.CreatedAt, "yyyy-MM-dd HH:mm")
|
||||||
</div>
|
</div>
|
||||||
</MudTd>
|
</MudTd>
|
||||||
<MudTd DataLabel="Last Attempt">
|
<MudTd DataLabel="Last Attempt">
|
||||||
<div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">
|
<div @oncontextmenu="@(async (e) => await OnRowRightClick(e, row))" @oncontextmenu:preventDefault="true">
|
||||||
@(row.LastAttemptAt?.ToString("yyyy-MM-dd HH:mm") ?? "-")
|
@DateTimeHelper.FormatDateTime(row.LastAttemptAt, "yyyy-MM-dd HH:mm")
|
||||||
</div>
|
</div>
|
||||||
</MudTd>
|
</MudTd>
|
||||||
</RowTemplate>
|
</RowTemplate>
|
||||||
|
|||||||
@@ -15,16 +15,18 @@ public partial class Index : ComponentBase, IDisposable
|
|||||||
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
||||||
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
||||||
[Inject] private IJSRuntime JSRuntime { get; set; } = default!;
|
[Inject] private IJSRuntime JSRuntime { get; set; } = default!;
|
||||||
|
[Inject] private DateTimeHelper DateTimeHelper { get; set; } = default!;
|
||||||
|
|
||||||
private PagedResult<QueueJob> jobs = new();
|
private PagedResult<QueueJob> jobs = new();
|
||||||
private bool isLoading = false;
|
private bool isLoading = false;
|
||||||
private int currentPage = 1;
|
private int currentPage = 1;
|
||||||
private int pageSize = 50;
|
private int pageSize = 50;
|
||||||
private JobStatus? selectedStatus = null;
|
private IEnumerable<JobStatus> selectedStatuses = new HashSet<JobStatus>();
|
||||||
private JobType? selectedJobType = null;
|
private JobType? selectedJobType = null;
|
||||||
|
|
||||||
protected override async Task OnInitializedAsync()
|
protected override async Task OnInitializedAsync()
|
||||||
{
|
{
|
||||||
|
await DateTimeHelper.InitializeAsync();
|
||||||
await LoadJobs();
|
await LoadJobs();
|
||||||
|
|
||||||
// Subscribe to SignalR entity changes
|
// Subscribe to SignalR entity changes
|
||||||
@@ -60,7 +62,8 @@ public partial class Index : ComponentBase, IDisposable
|
|||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
jobs = await JobService.GetJobsAsync(currentPage, pageSize, selectedStatus, selectedJobType);
|
var statusList = selectedStatuses?.ToList();
|
||||||
|
jobs = await JobService.GetJobsAsync(currentPage, pageSize, statusList, selectedJobType);
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
@@ -81,15 +84,29 @@ public partial class Index : ComponentBase, IDisposable
|
|||||||
|
|
||||||
private async Task ClearFilters()
|
private async Task ClearFilters()
|
||||||
{
|
{
|
||||||
selectedStatus = null;
|
selectedStatuses = new HashSet<JobStatus>();
|
||||||
selectedJobType = null;
|
selectedJobType = null;
|
||||||
currentPage = 1;
|
currentPage = 1;
|
||||||
await LoadJobs();
|
await LoadJobs();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async Task OnStatusFilterChanged(IEnumerable<JobStatus> values)
|
||||||
|
{
|
||||||
|
selectedStatuses = values;
|
||||||
|
currentPage = 1;
|
||||||
|
await LoadJobs();
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task OnJobTypeFilterChanged(JobType? value)
|
||||||
|
{
|
||||||
|
selectedJobType = value;
|
||||||
|
currentPage = 1;
|
||||||
|
await LoadJobs();
|
||||||
|
}
|
||||||
|
|
||||||
private async Task OnStatusClear()
|
private async Task OnStatusClear()
|
||||||
{
|
{
|
||||||
selectedStatus = null;
|
selectedStatuses = new HashSet<JobStatus>();
|
||||||
currentPage = 1;
|
currentPage = 1;
|
||||||
await LoadJobs();
|
await LoadJobs();
|
||||||
}
|
}
|
||||||
@@ -112,6 +129,41 @@ public partial class Index : ComponentBase, IDisposable
|
|||||||
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async Task ScheduleJobs(string type)
|
||||||
|
{
|
||||||
|
isLoading = true;
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
(bool success, int jobsCreated, string message) result = type switch
|
||||||
|
{
|
||||||
|
"all" => await JobService.ScheduleAllJobsAsync(),
|
||||||
|
"imports" => await JobService.ScheduleImportJobsAsync(),
|
||||||
|
"processes" => await JobService.ScheduleProcessJobsAsync(),
|
||||||
|
_ => (false, 0, "Unknown job type")
|
||||||
|
};
|
||||||
|
|
||||||
|
if (result.success)
|
||||||
|
{
|
||||||
|
Snackbar.Add($"{result.message} ({result.jobsCreated} jobs created)", Severity.Success);
|
||||||
|
await LoadJobs();
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
Snackbar.Add(result.message, Severity.Error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"Scheduling jobs failed: {ex.Message}");
|
||||||
|
Snackbar.Add($"Failed to schedule jobs: {ex.Message}", Severity.Error);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
isLoading = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private Color GetStatusColor(JobStatus status)
|
private Color GetStatusColor(JobStatus status)
|
||||||
{
|
{
|
||||||
return status switch
|
return status switch
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
@using DiunaBI.UI.Shared.Services
|
@using DiunaBI.UI.Shared.Services
|
||||||
@using DiunaBI.Application.DTOModels
|
@using DiunaBI.Application.DTOModels
|
||||||
@using MudBlazor
|
@using MudBlazor
|
||||||
|
@implements IDisposable
|
||||||
|
|
||||||
<MudCard>
|
<MudCard>
|
||||||
<MudCardHeader>
|
<MudCardHeader>
|
||||||
@@ -58,7 +59,7 @@
|
|||||||
}
|
}
|
||||||
</MudItem>
|
</MudItem>
|
||||||
<MudItem xs="12" md="6">
|
<MudItem xs="12" md="6">
|
||||||
<MudTextField Value="@layer.CreatedAt.ToString("g")"
|
<MudTextField Value="@DateTimeHelper.FormatDateTime(layer.CreatedAt, "yyyy-MM-dd HH:mm")"
|
||||||
Label="Created"
|
Label="Created"
|
||||||
Variant="Variant.Outlined"
|
Variant="Variant.Outlined"
|
||||||
ReadOnly="true"
|
ReadOnly="true"
|
||||||
@@ -67,7 +68,7 @@
|
|||||||
AdornmentText="@(layer.CreatedBy?.Username ?? "")"/>
|
AdornmentText="@(layer.CreatedBy?.Username ?? "")"/>
|
||||||
</MudItem>
|
</MudItem>
|
||||||
<MudItem xs="12" md="6">
|
<MudItem xs="12" md="6">
|
||||||
<MudTextField Value="@layer.ModifiedAt.ToString("g")"
|
<MudTextField Value="@DateTimeHelper.FormatDateTime(layer.ModifiedAt, "yyyy-MM-dd HH:mm")"
|
||||||
Label="Modified"
|
Label="Modified"
|
||||||
Variant="Variant.Outlined"
|
Variant="Variant.Outlined"
|
||||||
ReadOnly="true"
|
ReadOnly="true"
|
||||||
@@ -142,7 +143,7 @@
|
|||||||
<MudTd DataLabel="Code">@context.Code</MudTd>
|
<MudTd DataLabel="Code">@context.Code</MudTd>
|
||||||
@foreach (var column in displayedColumns)
|
@foreach (var column in displayedColumns)
|
||||||
{
|
{
|
||||||
<MudTd DataLabel="@column">@GetRecordValue(context, column)</MudTd>
|
<MudTd DataLabel="@column" Style="@GetColumnStyle(column)">@GetRecordValue(context, column)</MudTd>
|
||||||
}
|
}
|
||||||
@if (isEditable)
|
@if (isEditable)
|
||||||
{
|
{
|
||||||
@@ -162,22 +163,25 @@
|
|||||||
}
|
}
|
||||||
</RowTemplate>
|
</RowTemplate>
|
||||||
<FooterContent>
|
<FooterContent>
|
||||||
<MudTd><b>Value1 sum</b></MudTd>
|
@if (showSummary)
|
||||||
@foreach (var column in displayedColumns)
|
|
||||||
{
|
{
|
||||||
@if (column == "Value1")
|
<MudTd Style="text-align: left;"><b>@NumberFormatHelper.FormatNumber(totalSum)</b></MudTd>
|
||||||
|
@foreach (var column in displayedColumns)
|
||||||
{
|
{
|
||||||
<MudTd><b>@valueSum.ToString("N2")</b></MudTd>
|
@if (column.StartsWith("Value") && columnSums.ContainsKey(column))
|
||||||
|
{
|
||||||
|
<MudTd Style="text-align: right;"><b>@NumberFormatHelper.FormatNumber(columnSums[column])</b></MudTd>
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
<MudTd></MudTd>
|
||||||
|
}
|
||||||
}
|
}
|
||||||
else
|
@if (isEditable)
|
||||||
{
|
{
|
||||||
<MudTd></MudTd>
|
<MudTd></MudTd>
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@if (isEditable)
|
|
||||||
{
|
|
||||||
<MudTd></MudTd>
|
|
||||||
}
|
|
||||||
</FooterContent>
|
</FooterContent>
|
||||||
</MudTable>
|
</MudTable>
|
||||||
|
|
||||||
@@ -230,7 +234,92 @@
|
|||||||
}
|
}
|
||||||
</MudTabPanel>
|
</MudTabPanel>
|
||||||
|
|
||||||
<MudTabPanel Text="History" Icon="@Icons.Material.Filled.History">
|
@if (layer?.Type == LayerType.Validation)
|
||||||
|
{
|
||||||
|
<MudTabPanel Text="Validation Results" Icon="@Icons.Material.Filled.CheckCircle">
|
||||||
|
@if (records.Any())
|
||||||
|
{
|
||||||
|
var overallStatus = records.FirstOrDefault(r => r.Code == "OverallStatus")?.Desc1;
|
||||||
|
var confidence = records.FirstOrDefault(r => r.Code == "Confidence")?.Desc1;
|
||||||
|
var summary = records.FirstOrDefault(r => r.Code == "Summary")?.Desc1;
|
||||||
|
|
||||||
|
<MudAlert Severity="@GetValidationSeverity(overallStatus)" Dense="false" Class="mb-4">
|
||||||
|
<strong>Overall Status:</strong> @(overallStatus?.ToUpper() ?? "UNKNOWN")
|
||||||
|
@if (!string.IsNullOrEmpty(confidence))
|
||||||
|
{
|
||||||
|
<span style="margin-left: 16px;"><strong>Confidence:</strong> @confidence</span>
|
||||||
|
}
|
||||||
|
</MudAlert>
|
||||||
|
|
||||||
|
@if (!string.IsNullOrEmpty(summary))
|
||||||
|
{
|
||||||
|
<MudPaper Class="pa-4 mb-4" Outlined="true">
|
||||||
|
<MudText Typo="Typo.h6" Class="mb-2">Summary</MudText>
|
||||||
|
<MudText Typo="Typo.body2">@summary</MudText>
|
||||||
|
</MudPaper>
|
||||||
|
}
|
||||||
|
|
||||||
|
@if (records.Where(r => r.Code.StartsWith("ANOMALY_")).Any())
|
||||||
|
{
|
||||||
|
<MudText Typo="Typo.h6" Class="mb-3">Record Anomalies</MudText>
|
||||||
|
<MudTable Items="@records.Where(r => r.Code.StartsWith("ANOMALY_")).ToList()"
|
||||||
|
Dense="true"
|
||||||
|
Striped="true"
|
||||||
|
FixedHeader="true"
|
||||||
|
Elevation="0"
|
||||||
|
Class="mb-4">
|
||||||
|
<HeaderContent>
|
||||||
|
<MudTh>Code</MudTh>
|
||||||
|
<MudTh>Details</MudTh>
|
||||||
|
</HeaderContent>
|
||||||
|
<RowTemplate>
|
||||||
|
<MudTd DataLabel="Code">
|
||||||
|
<MudChip T="string" Size="Size.Small" Color="Color.Warning">@context.Code.Replace("ANOMALY_", "")</MudChip>
|
||||||
|
</MudTd>
|
||||||
|
<MudTd DataLabel="Details">@context.Desc1</MudTd>
|
||||||
|
</RowTemplate>
|
||||||
|
</MudTable>
|
||||||
|
}
|
||||||
|
|
||||||
|
@if (records.Where(r => r.Code.StartsWith("STRUCTURAL_")).Any())
|
||||||
|
{
|
||||||
|
<MudText Typo="Typo.h6" Class="mb-3">Structural Issues</MudText>
|
||||||
|
<MudTable Items="@records.Where(r => r.Code.StartsWith("STRUCTURAL_")).ToList()"
|
||||||
|
Dense="true"
|
||||||
|
Striped="true"
|
||||||
|
FixedHeader="true"
|
||||||
|
Elevation="0"
|
||||||
|
Class="mb-4">
|
||||||
|
<HeaderContent>
|
||||||
|
<MudTh>Type</MudTh>
|
||||||
|
<MudTh>Details</MudTh>
|
||||||
|
</HeaderContent>
|
||||||
|
<RowTemplate>
|
||||||
|
<MudTd DataLabel="Type">
|
||||||
|
<MudChip T="string" Size="Size.Small" Color="Color.Error">@context.Code.Replace("STRUCTURAL_", "")</MudChip>
|
||||||
|
</MudTd>
|
||||||
|
<MudTd DataLabel="Details">@context.Desc1</MudTd>
|
||||||
|
</RowTemplate>
|
||||||
|
</MudTable>
|
||||||
|
}
|
||||||
|
|
||||||
|
@if (!records.Where(r => r.Code.StartsWith("ANOMALY_")).Any() && !records.Where(r => r.Code.StartsWith("STRUCTURAL_")).Any())
|
||||||
|
{
|
||||||
|
<MudAlert Severity="Severity.Success" Dense="true">
|
||||||
|
No anomalies or structural issues detected. All data appears normal.
|
||||||
|
</MudAlert>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
<MudAlert Severity="Severity.Info">No validation results available.</MudAlert>
|
||||||
|
}
|
||||||
|
</MudTabPanel>
|
||||||
|
}
|
||||||
|
|
||||||
|
@if (showHistoryTab)
|
||||||
|
{
|
||||||
|
<MudTabPanel Text="History" Icon="@Icons.Material.Filled.History">
|
||||||
@if (isLoadingHistory)
|
@if (isLoadingHistory)
|
||||||
{
|
{
|
||||||
<MudProgressLinear Color="Color.Primary" Indeterminate="true" />
|
<MudProgressLinear Color="Color.Primary" Indeterminate="true" />
|
||||||
@@ -311,7 +400,7 @@
|
|||||||
<RowTemplate>
|
<RowTemplate>
|
||||||
<MudTd DataLabel="Code">@context.Code</MudTd>
|
<MudTd DataLabel="Code">@context.Code</MudTd>
|
||||||
<MudTd DataLabel="Description">@context.Desc1</MudTd>
|
<MudTd DataLabel="Description">@context.Desc1</MudTd>
|
||||||
<MudTd DataLabel="Modified">@context.ModifiedAt.ToString("g")</MudTd>
|
<MudTd DataLabel="Modified">@DateTimeHelper.FormatDateTime(context.ModifiedAt, "yyyy-MM-dd HH:mm")</MudTd>
|
||||||
<MudTd DataLabel="Modified By">@GetModifiedByUsername(context.ModifiedById)</MudTd>
|
<MudTd DataLabel="Modified By">@GetModifiedByUsername(context.ModifiedById)</MudTd>
|
||||||
</RowTemplate>
|
</RowTemplate>
|
||||||
</MudTable>
|
</MudTable>
|
||||||
@@ -354,6 +443,7 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
</MudTabPanel>
|
</MudTabPanel>
|
||||||
|
}
|
||||||
</MudTabs>
|
</MudTabs>
|
||||||
}
|
}
|
||||||
</MudCardContent>
|
</MudCardContent>
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ using System.Reflection;
|
|||||||
|
|
||||||
namespace DiunaBI.UI.Shared.Pages.Layers;
|
namespace DiunaBI.UI.Shared.Pages.Layers;
|
||||||
|
|
||||||
public partial class Details : ComponentBase
|
public partial class Details : ComponentBase, IDisposable
|
||||||
{
|
{
|
||||||
[Parameter]
|
[Parameter]
|
||||||
public Guid Id { get; set; }
|
public Guid Id { get; set; }
|
||||||
@@ -20,22 +20,35 @@ public partial class Details : ComponentBase
|
|||||||
[Inject]
|
[Inject]
|
||||||
private JobService JobService { get; set; } = null!;
|
private JobService JobService { get; set; } = null!;
|
||||||
|
|
||||||
|
[Inject]
|
||||||
|
private EntityChangeHubService HubService { get; set; } = null!;
|
||||||
|
|
||||||
[Inject]
|
[Inject]
|
||||||
private NavigationManager NavigationManager { get; set; } = null!;
|
private NavigationManager NavigationManager { get; set; } = null!;
|
||||||
|
|
||||||
[Inject]
|
[Inject]
|
||||||
private ISnackbar Snackbar { get; set; } = null!;
|
private ISnackbar Snackbar { get; set; } = null!;
|
||||||
|
|
||||||
|
[Inject]
|
||||||
|
private DateTimeHelper DateTimeHelper { get; set; } = null!;
|
||||||
|
|
||||||
|
[Inject]
|
||||||
|
private NumberFormatHelper NumberFormatHelper { get; set; } = null!;
|
||||||
|
|
||||||
private LayerDto? layer;
|
private LayerDto? layer;
|
||||||
private List<RecordDto> records = new();
|
private List<RecordDto> records = new();
|
||||||
private List<string> displayedColumns = new();
|
private List<string> displayedColumns = new();
|
||||||
private double valueSum = 0;
|
private double valueSum = 0;
|
||||||
|
private Dictionary<string, double> columnSums = new();
|
||||||
|
private double totalSum = 0;
|
||||||
private bool isLoading = false;
|
private bool isLoading = false;
|
||||||
private Guid? editingRecordId = null;
|
private Guid? editingRecordId = null;
|
||||||
private RecordDto? editingRecord = null;
|
private RecordDto? editingRecord = null;
|
||||||
private bool isAddingNew = false;
|
private bool isAddingNew = false;
|
||||||
private RecordDto newRecord = new();
|
private RecordDto newRecord = new();
|
||||||
private bool isEditable => layer?.Type == LayerType.Dictionary || layer?.Type == LayerType.Administration;
|
private bool isEditable => layer?.Type == LayerType.Dictionary || layer?.Type == LayerType.Administration;
|
||||||
|
private bool showHistoryTab => layer?.Type == LayerType.Administration || layer?.Type == LayerType.Dictionary;
|
||||||
|
private bool showSummary => layer?.Type == LayerType.Import || layer?.Type == LayerType.Processed;
|
||||||
|
|
||||||
// History tab state
|
// History tab state
|
||||||
private bool isLoadingHistory = false;
|
private bool isLoadingHistory = false;
|
||||||
@@ -48,7 +61,41 @@ public partial class Details : ComponentBase
|
|||||||
|
|
||||||
protected override async Task OnInitializedAsync()
|
protected override async Task OnInitializedAsync()
|
||||||
{
|
{
|
||||||
|
await DateTimeHelper.InitializeAsync();
|
||||||
await LoadLayer();
|
await LoadLayer();
|
||||||
|
|
||||||
|
// Subscribe to SignalR entity changes
|
||||||
|
HubService.EntityChanged += OnEntityChanged;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async void OnEntityChanged(string module, string id, string operation)
|
||||||
|
{
|
||||||
|
// React to Layers or Records changes for this layer
|
||||||
|
if (module.Equals("Layers", StringComparison.OrdinalIgnoreCase) ||
|
||||||
|
module.Equals("Records", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
// Check if it's this layer or its records that changed
|
||||||
|
if (Guid.TryParse(id, out var changedId))
|
||||||
|
{
|
||||||
|
if (module.Equals("Layers", StringComparison.OrdinalIgnoreCase) && changedId == Id)
|
||||||
|
{
|
||||||
|
await InvokeAsync(async () =>
|
||||||
|
{
|
||||||
|
await LoadLayer();
|
||||||
|
StateHasChanged();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
else if (module.Equals("Records", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
// For records, we reload to get the latest data
|
||||||
|
await InvokeAsync(async () =>
|
||||||
|
{
|
||||||
|
await LoadLayer();
|
||||||
|
StateHasChanged();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
protected override async Task OnParametersSetAsync()
|
protected override async Task OnParametersSetAsync()
|
||||||
@@ -79,7 +126,7 @@ public partial class Details : ComponentBase
|
|||||||
{
|
{
|
||||||
records = layer.Records.OrderBy(r => r.Code).ToList();
|
records = layer.Records.OrderBy(r => r.Code).ToList();
|
||||||
CalculateDisplayedColumns();
|
CalculateDisplayedColumns();
|
||||||
CalculateValueSum();
|
CalculateColumnSums();
|
||||||
BuildUserCache();
|
BuildUserCache();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -118,11 +165,25 @@ public partial class Details : ComponentBase
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private void CalculateValueSum()
|
private void CalculateColumnSums()
|
||||||
{
|
{
|
||||||
valueSum = records
|
columnSums.Clear();
|
||||||
.Where(r => r.Value1.HasValue)
|
totalSum = 0;
|
||||||
.Sum(r => r.Value1!.Value);
|
|
||||||
|
// Calculate sum for each displayed value column
|
||||||
|
foreach (var columnName in displayedColumns.Where(c => c.StartsWith("Value")))
|
||||||
|
{
|
||||||
|
var sum = records
|
||||||
|
.Select(r => GetRecordValueByName(r, columnName))
|
||||||
|
.Where(v => v.HasValue)
|
||||||
|
.Sum(v => v!.Value);
|
||||||
|
|
||||||
|
columnSums[columnName] = sum;
|
||||||
|
totalSum += sum;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keep valueSum for backward compatibility (Value1 sum)
|
||||||
|
valueSum = columnSums.ContainsKey("Value1") ? columnSums["Value1"] : 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
private string GetRecordValue(RecordDto record, string columnName)
|
private string GetRecordValue(RecordDto record, string columnName)
|
||||||
@@ -133,7 +194,17 @@ public partial class Details : ComponentBase
|
|||||||
}
|
}
|
||||||
|
|
||||||
var value = GetRecordValueByName(record, columnName);
|
var value = GetRecordValueByName(record, columnName);
|
||||||
return value.HasValue ? value.Value.ToString("N2") : string.Empty;
|
return NumberFormatHelper.FormatNumber(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
private string GetColumnStyle(string columnName)
|
||||||
|
{
|
||||||
|
// Align numeric columns (Value1, Value2, etc.) to the left
|
||||||
|
if (columnName.StartsWith("Value"))
|
||||||
|
{
|
||||||
|
return "text-align: right;";
|
||||||
|
}
|
||||||
|
return string.Empty;
|
||||||
}
|
}
|
||||||
|
|
||||||
private double? GetRecordValueByName(RecordDto record, string columnName)
|
private double? GetRecordValueByName(RecordDto record, string columnName)
|
||||||
@@ -246,7 +317,7 @@ public partial class Details : ComponentBase
|
|||||||
{
|
{
|
||||||
records.Remove(record);
|
records.Remove(record);
|
||||||
CalculateDisplayedColumns();
|
CalculateDisplayedColumns();
|
||||||
CalculateValueSum();
|
CalculateColumnSums();
|
||||||
Snackbar.Add("Record deleted successfully", Severity.Success);
|
Snackbar.Add("Record deleted successfully", Severity.Success);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
@@ -297,7 +368,7 @@ public partial class Details : ComponentBase
|
|||||||
{
|
{
|
||||||
records.Add(created);
|
records.Add(created);
|
||||||
CalculateDisplayedColumns();
|
CalculateDisplayedColumns();
|
||||||
CalculateValueSum();
|
CalculateColumnSums();
|
||||||
isAddingNew = false;
|
isAddingNew = false;
|
||||||
newRecord = new();
|
newRecord = new();
|
||||||
Snackbar.Add("Record added successfully", Severity.Success);
|
Snackbar.Add("Record added successfully", Severity.Success);
|
||||||
@@ -432,7 +503,7 @@ public partial class Details : ComponentBase
|
|||||||
if (layer?.Records == null) return false;
|
if (layer?.Records == null) return false;
|
||||||
|
|
||||||
var typeRecord = layer.Records.FirstOrDefault(x => x.Code == "Type");
|
var typeRecord = layer.Records.FirstOrDefault(x => x.Code == "Type");
|
||||||
return typeRecord?.Desc1 == "ImportWorker" || typeRecord?.Desc1 == "ProcessWorker";
|
return typeRecord?.Desc1 == "ImportWorker" || typeRecord?.Desc1 == "ProcessWorker" || typeRecord?.Desc1 == "ValidationWorker";
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task RunNow()
|
private async Task RunNow()
|
||||||
@@ -473,4 +544,21 @@ public partial class Details : ComponentBase
|
|||||||
isRunningJob = false;
|
isRunningJob = false;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Validation tab helper methods
|
||||||
|
private Severity GetValidationSeverity(string? status)
|
||||||
|
{
|
||||||
|
return status?.ToLower() switch
|
||||||
|
{
|
||||||
|
"pass" => Severity.Success,
|
||||||
|
"warning" => Severity.Warning,
|
||||||
|
"critical" => Severity.Error,
|
||||||
|
_ => Severity.Info
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
HubService.EntityChanged -= OnEntityChanged;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
@page "/layers"
|
@page "/layers"
|
||||||
@using MudBlazor.Internal
|
@using MudBlazor.Internal
|
||||||
@using DiunaBI.Application.DTOModels
|
@using DiunaBI.Application.DTOModels
|
||||||
|
@implements IDisposable
|
||||||
|
|
||||||
<PageTitle>Layers</PageTitle>
|
<PageTitle>Layers</PageTitle>
|
||||||
|
|
||||||
|
|||||||
@@ -8,9 +8,10 @@ using Microsoft.JSInterop;
|
|||||||
|
|
||||||
namespace DiunaBI.UI.Shared.Pages.Layers;
|
namespace DiunaBI.UI.Shared.Pages.Layers;
|
||||||
|
|
||||||
public partial class Index : ComponentBase
|
public partial class Index : ComponentBase, IDisposable
|
||||||
{
|
{
|
||||||
[Inject] private LayerService LayerService { get; set; } = default!;
|
[Inject] private LayerService LayerService { get; set; } = default!;
|
||||||
|
[Inject] private EntityChangeHubService HubService { get; set; } = default!;
|
||||||
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
[Inject] private ISnackbar Snackbar { get; set; } = default!;
|
||||||
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
[Inject] private NavigationManager NavigationManager { get; set; } = default!;
|
||||||
[Inject] private LayerFilterStateService FilterStateService { get; set; } = default!;
|
[Inject] private LayerFilterStateService FilterStateService { get; set; } = default!;
|
||||||
@@ -25,6 +26,22 @@ public partial class Index : ComponentBase
|
|||||||
{
|
{
|
||||||
filterRequest = FilterStateService.FilterRequest;
|
filterRequest = FilterStateService.FilterRequest;
|
||||||
await LoadLayers();
|
await LoadLayers();
|
||||||
|
|
||||||
|
// Subscribe to SignalR entity changes
|
||||||
|
HubService.EntityChanged += OnEntityChanged;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async void OnEntityChanged(string module, string id, string operation)
|
||||||
|
{
|
||||||
|
// Only react if it's a Layers change
|
||||||
|
if (module.Equals("Layers", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
await InvokeAsync(async () =>
|
||||||
|
{
|
||||||
|
await LoadLayers();
|
||||||
|
StateHasChanged();
|
||||||
|
});
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task LoadLayers()
|
private async Task LoadLayers()
|
||||||
@@ -89,4 +106,9 @@ public partial class Index : ComponentBase
|
|||||||
var url = NavigationManager.ToAbsoluteUri($"/layers/{layer.Id}").ToString();
|
var url = NavigationManager.ToAbsoluteUri($"/layers/{layer.Id}").ToString();
|
||||||
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
await JSRuntime.InvokeVoidAsync("open", url, "_blank");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
HubService.EntityChanged -= OnEntityChanged;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,4 +3,5 @@ namespace DiunaBI.UI.Shared.Services;
|
|||||||
public class AppConfig
|
public class AppConfig
|
||||||
{
|
{
|
||||||
public string AppName { get; set; } = "DiunaBI";
|
public string AppName { get; set; } = "DiunaBI";
|
||||||
|
public string ClientLogo {get; set;} = "pedrollopl.png";
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -15,16 +15,18 @@ public class AuthService
|
|||||||
{
|
{
|
||||||
private readonly HttpClient _httpClient;
|
private readonly HttpClient _httpClient;
|
||||||
private readonly IJSRuntime _jsRuntime;
|
private readonly IJSRuntime _jsRuntime;
|
||||||
|
private readonly TokenProvider _tokenProvider;
|
||||||
private bool? _isAuthenticated;
|
private bool? _isAuthenticated;
|
||||||
private UserInfo? _userInfo = null;
|
private UserInfo? _userInfo = null;
|
||||||
private string? _apiToken;
|
private string? _apiToken;
|
||||||
|
|
||||||
public event Action<bool>? AuthenticationStateChanged;
|
public event Action<bool>? AuthenticationStateChanged;
|
||||||
|
|
||||||
public AuthService(HttpClient httpClient, IJSRuntime jsRuntime)
|
public AuthService(HttpClient httpClient, IJSRuntime jsRuntime, TokenProvider tokenProvider)
|
||||||
{
|
{
|
||||||
_httpClient = httpClient;
|
_httpClient = httpClient;
|
||||||
_jsRuntime = jsRuntime;
|
_jsRuntime = jsRuntime;
|
||||||
|
_tokenProvider = tokenProvider;
|
||||||
}
|
}
|
||||||
|
|
||||||
public bool IsAuthenticated => _isAuthenticated ?? false;
|
public bool IsAuthenticated => _isAuthenticated ?? false;
|
||||||
@@ -44,6 +46,7 @@ public class AuthService
|
|||||||
if (result != null)
|
if (result != null)
|
||||||
{
|
{
|
||||||
_apiToken = result.Token;
|
_apiToken = result.Token;
|
||||||
|
_tokenProvider.Token = result.Token; // Set token for SignalR
|
||||||
_userInfo = new UserInfo
|
_userInfo = new UserInfo
|
||||||
{
|
{
|
||||||
Id = result.Id,
|
Id = result.Id,
|
||||||
@@ -104,6 +107,7 @@ public class AuthService
|
|||||||
if (_isAuthenticated.Value && !string.IsNullOrEmpty(userInfoJson))
|
if (_isAuthenticated.Value && !string.IsNullOrEmpty(userInfoJson))
|
||||||
{
|
{
|
||||||
_apiToken = token;
|
_apiToken = token;
|
||||||
|
_tokenProvider.Token = token; // Set token for SignalR
|
||||||
_userInfo = JsonSerializer.Deserialize<UserInfo>(userInfoJson);
|
_userInfo = JsonSerializer.Deserialize<UserInfo>(userInfoJson);
|
||||||
|
|
||||||
// Restore header
|
// Restore header
|
||||||
@@ -111,14 +115,17 @@ public class AuthService
|
|||||||
new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _apiToken);
|
new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _apiToken);
|
||||||
|
|
||||||
Console.WriteLine($"✅ Session restored: {_userInfo?.Email}");
|
Console.WriteLine($"✅ Session restored: {_userInfo?.Email}");
|
||||||
|
|
||||||
|
// Notify that authentication state changed (for SignalR initialization)
|
||||||
|
AuthenticationStateChanged?.Invoke(true);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
Console.WriteLine("❌ No valid session");
|
Console.WriteLine("❌ No valid session");
|
||||||
}
|
}
|
||||||
|
|
||||||
Console.WriteLine($"=== AuthService.CheckAuthenticationAsync END (authenticated={_isAuthenticated}) ===");
|
Console.WriteLine($"=== AuthService.CheckAuthenticationAsync END (authenticated={_isAuthenticated}) ===");
|
||||||
|
|
||||||
return _isAuthenticated.Value;
|
return _isAuthenticated.Value;
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
@@ -139,11 +146,12 @@ public class AuthService
|
|||||||
await _jsRuntime.InvokeVoidAsync("localStorage.removeItem", "user_info");
|
await _jsRuntime.InvokeVoidAsync("localStorage.removeItem", "user_info");
|
||||||
|
|
||||||
_apiToken = null;
|
_apiToken = null;
|
||||||
|
_tokenProvider.Token = null; // Clear token for SignalR
|
||||||
_isAuthenticated = false;
|
_isAuthenticated = false;
|
||||||
_userInfo = null;
|
_userInfo = null;
|
||||||
|
|
||||||
_httpClient.DefaultRequestHeaders.Authorization = null;
|
_httpClient.DefaultRequestHeaders.Authorization = null;
|
||||||
|
|
||||||
Console.WriteLine("✅ Authentication cleared");
|
Console.WriteLine("✅ Authentication cleared");
|
||||||
AuthenticationStateChanged?.Invoke(false);
|
AuthenticationStateChanged?.Invoke(false);
|
||||||
}
|
}
|
||||||
|
|||||||
80
DiunaBI.UI.Shared/Services/DateTimeHelper.cs
Normal file
80
DiunaBI.UI.Shared/Services/DateTimeHelper.cs
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
using Microsoft.JSInterop;
|
||||||
|
|
||||||
|
namespace DiunaBI.UI.Shared.Services;
|
||||||
|
|
||||||
|
public class DateTimeHelper
|
||||||
|
{
|
||||||
|
private readonly IJSRuntime _jsRuntime;
|
||||||
|
private TimeZoneInfo? _userTimeZone;
|
||||||
|
private bool _initialized = false;
|
||||||
|
|
||||||
|
public DateTimeHelper(IJSRuntime jsRuntime)
|
||||||
|
{
|
||||||
|
_jsRuntime = jsRuntime;
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task InitializeAsync()
|
||||||
|
{
|
||||||
|
if (_initialized) return;
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// Get the user's timezone from JavaScript
|
||||||
|
var timeZoneId = await _jsRuntime.InvokeAsync<string>("eval", "Intl.DateTimeFormat().resolvedOptions().timeZone");
|
||||||
|
|
||||||
|
// Try to find the TimeZoneInfo
|
||||||
|
try
|
||||||
|
{
|
||||||
|
_userTimeZone = TimeZoneInfo.FindSystemTimeZoneById(timeZoneId);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
// Fallback to local timezone if the IANA timezone ID is not found
|
||||||
|
_userTimeZone = TimeZoneInfo.Local;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
// Fallback to local timezone if JavaScript interop fails
|
||||||
|
_userTimeZone = TimeZoneInfo.Local;
|
||||||
|
}
|
||||||
|
|
||||||
|
_initialized = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatDateTime(DateTime? dateTime, string format = "yyyy-MM-dd HH:mm:ss")
|
||||||
|
{
|
||||||
|
if (!dateTime.HasValue)
|
||||||
|
return "-";
|
||||||
|
|
||||||
|
if (!_initialized)
|
||||||
|
{
|
||||||
|
// If not initialized yet, just format as-is (will be UTC)
|
||||||
|
return dateTime.Value.ToString(format);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert UTC to user's timezone
|
||||||
|
var localDateTime = TimeZoneInfo.ConvertTimeFromUtc(dateTime.Value, _userTimeZone ?? TimeZoneInfo.Local);
|
||||||
|
return localDateTime.ToString(format);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatDate(DateTime? dateTime, string format = "yyyy-MM-dd")
|
||||||
|
{
|
||||||
|
return FormatDateTime(dateTime, format);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatTime(DateTime? dateTime, string format = "HH:mm:ss")
|
||||||
|
{
|
||||||
|
return FormatDateTime(dateTime, format);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string GetTimeZoneAbbreviation()
|
||||||
|
{
|
||||||
|
if (!_initialized || _userTimeZone == null)
|
||||||
|
return "UTC";
|
||||||
|
|
||||||
|
return _userTimeZone.IsDaylightSavingTime(DateTime.Now)
|
||||||
|
? _userTimeZone.DaylightName
|
||||||
|
: _userTimeZone.StandardName;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -19,13 +19,18 @@ public class JobService
|
|||||||
PropertyNameCaseInsensitive = true
|
PropertyNameCaseInsensitive = true
|
||||||
};
|
};
|
||||||
|
|
||||||
public async Task<PagedResult<QueueJob>> GetJobsAsync(int page = 1, int pageSize = 50, JobStatus? status = null, JobType? jobType = null, Guid? layerId = null)
|
public async Task<PagedResult<QueueJob>> GetJobsAsync(int page = 1, int pageSize = 50, List<JobStatus>? statuses = null, JobType? jobType = null, Guid? layerId = null)
|
||||||
{
|
{
|
||||||
var start = (page - 1) * pageSize;
|
var start = (page - 1) * pageSize;
|
||||||
var query = $"Jobs?start={start}&limit={pageSize}";
|
var query = $"Jobs?start={start}&limit={pageSize}";
|
||||||
|
|
||||||
if (status.HasValue)
|
if (statuses != null && statuses.Count > 0)
|
||||||
query += $"&status={(int)status.Value}";
|
{
|
||||||
|
foreach (var status in statuses)
|
||||||
|
{
|
||||||
|
query += $"&statuses={(int)status}";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if (jobType.HasValue)
|
if (jobType.HasValue)
|
||||||
query += $"&jobType={(int)jobType.Value}";
|
query += $"&jobType={(int)jobType.Value}";
|
||||||
@@ -83,6 +88,89 @@ public class JobService
|
|||||||
|
|
||||||
return await response.Content.ReadFromJsonAsync<CreateJobResult>();
|
return await response.Content.ReadFromJsonAsync<CreateJobResult>();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public async Task<(bool success, int jobsCreated, string message)> ScheduleAllJobsAsync(string? nameFilter = null)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var query = string.IsNullOrEmpty(nameFilter) ? "" : $"?nameFilter={Uri.EscapeDataString(nameFilter)}";
|
||||||
|
var response = await _httpClient.PostAsync($"Jobs/ui/schedule{query}", null);
|
||||||
|
|
||||||
|
if (!response.IsSuccessStatusCode)
|
||||||
|
{
|
||||||
|
var error = await response.Content.ReadAsStringAsync();
|
||||||
|
return (false, 0, $"Failed to schedule jobs: {error}");
|
||||||
|
}
|
||||||
|
|
||||||
|
var json = await response.Content.ReadAsStringAsync();
|
||||||
|
var result = JsonSerializer.Deserialize<JsonElement>(json, _jsonOptions);
|
||||||
|
|
||||||
|
var jobsCreated = result.GetProperty("jobsCreated").GetInt32();
|
||||||
|
var message = result.GetProperty("message").GetString() ?? "Jobs scheduled";
|
||||||
|
|
||||||
|
return (true, jobsCreated, message);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"Scheduling jobs failed: {ex.Message}");
|
||||||
|
return (false, 0, $"Error: {ex.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task<(bool success, int jobsCreated, string message)> ScheduleImportJobsAsync(string? nameFilter = null)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var query = string.IsNullOrEmpty(nameFilter) ? "" : $"?nameFilter={Uri.EscapeDataString(nameFilter)}";
|
||||||
|
var response = await _httpClient.PostAsync($"Jobs/ui/schedule/imports{query}", null);
|
||||||
|
|
||||||
|
if (!response.IsSuccessStatusCode)
|
||||||
|
{
|
||||||
|
var error = await response.Content.ReadAsStringAsync();
|
||||||
|
return (false, 0, $"Failed to schedule import jobs: {error}");
|
||||||
|
}
|
||||||
|
|
||||||
|
var json = await response.Content.ReadAsStringAsync();
|
||||||
|
var result = JsonSerializer.Deserialize<JsonElement>(json, _jsonOptions);
|
||||||
|
|
||||||
|
var jobsCreated = result.GetProperty("jobsCreated").GetInt32();
|
||||||
|
var message = result.GetProperty("message").GetString() ?? "Import jobs scheduled";
|
||||||
|
|
||||||
|
return (true, jobsCreated, message);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"Scheduling import jobs failed: {ex.Message}");
|
||||||
|
return (false, 0, $"Error: {ex.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task<(bool success, int jobsCreated, string message)> ScheduleProcessJobsAsync()
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var response = await _httpClient.PostAsync("Jobs/ui/schedule/processes", null);
|
||||||
|
|
||||||
|
if (!response.IsSuccessStatusCode)
|
||||||
|
{
|
||||||
|
var error = await response.Content.ReadAsStringAsync();
|
||||||
|
return (false, 0, $"Failed to schedule process jobs: {error}");
|
||||||
|
}
|
||||||
|
|
||||||
|
var json = await response.Content.ReadAsStringAsync();
|
||||||
|
var result = JsonSerializer.Deserialize<JsonElement>(json, _jsonOptions);
|
||||||
|
|
||||||
|
var jobsCreated = result.GetProperty("jobsCreated").GetInt32();
|
||||||
|
var message = result.GetProperty("message").GetString() ?? "Process jobs scheduled";
|
||||||
|
|
||||||
|
return (true, jobsCreated, message);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
Console.WriteLine($"Scheduling process jobs failed: {ex.Message}");
|
||||||
|
return (false, 0, $"Error: {ex.Message}");
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public class JobStats
|
public class JobStats
|
||||||
|
|||||||
50
DiunaBI.UI.Shared/Services/NumberFormatHelper.cs
Normal file
50
DiunaBI.UI.Shared/Services/NumberFormatHelper.cs
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
using System.Globalization;
|
||||||
|
|
||||||
|
namespace DiunaBI.UI.Shared.Services;
|
||||||
|
|
||||||
|
public class NumberFormatHelper
|
||||||
|
{
|
||||||
|
private readonly CultureInfo _culture;
|
||||||
|
|
||||||
|
public NumberFormatHelper()
|
||||||
|
{
|
||||||
|
// Always use Polish culture for number formatting
|
||||||
|
_culture = CultureInfo.GetCultureInfo("pl-PL");
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatNumber(double? value, int decimals = 2)
|
||||||
|
{
|
||||||
|
if (!value.HasValue)
|
||||||
|
return string.Empty;
|
||||||
|
|
||||||
|
return value.Value.ToString($"N{decimals}", _culture);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatNumber(double value, int decimals = 2)
|
||||||
|
{
|
||||||
|
return value.ToString($"N{decimals}", _culture);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatInteger(double? value)
|
||||||
|
{
|
||||||
|
if (!value.HasValue)
|
||||||
|
return string.Empty;
|
||||||
|
|
||||||
|
return value.Value.ToString("N0", _culture);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string FormatInteger(double value)
|
||||||
|
{
|
||||||
|
return value.ToString("N0", _culture);
|
||||||
|
}
|
||||||
|
|
||||||
|
public string GetCultureName()
|
||||||
|
{
|
||||||
|
return _culture.DisplayName;
|
||||||
|
}
|
||||||
|
|
||||||
|
public CultureInfo GetCulture()
|
||||||
|
{
|
||||||
|
return _culture;
|
||||||
|
}
|
||||||
|
}
|
||||||
BIN
DiunaBI.UI.Shared/wwwroot/images/clients/morska.png
Normal file
BIN
DiunaBI.UI.Shared/wwwroot/images/clients/morska.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
DiunaBI.UI.Shared/wwwroot/images/clients/pedrollopl.png
Normal file
BIN
DiunaBI.UI.Shared/wwwroot/images/clients/pedrollopl.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 40 KiB |
@@ -31,9 +31,24 @@
|
|||||||
<a class="dismiss">🗙</a>
|
<a class="dismiss">🗙</a>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div id="components-reconnect-modal" data-nosnippet>
|
||||||
|
<div class="reconnect-content">
|
||||||
|
<div class="reconnect-spinner"></div>
|
||||||
|
<h5>Connection Lost</h5>
|
||||||
|
<div class="reconnect-message">
|
||||||
|
Attempting to reconnect to the server...
|
||||||
|
</div>
|
||||||
|
<div class="reconnect-timer">
|
||||||
|
<span id="reconnect-elapsed-time">0s</span>
|
||||||
|
</div>
|
||||||
|
<button onclick="location.reload()">Reload Page</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<script src="_framework/blazor.web.js"></script>
|
<script src="_framework/blazor.web.js"></script>
|
||||||
<script src="_content/MudBlazor/MudBlazor.min.js"></script>
|
<script src="_content/MudBlazor/MudBlazor.min.js"></script>
|
||||||
<script src="_content/DiunaBI.UI.Shared/js/auth.js"></script>
|
<script src="_content/DiunaBI.UI.Shared/js/auth.js"></script>
|
||||||
|
<script src="js/reconnect.js"></script>
|
||||||
</body>
|
</body>
|
||||||
|
|
||||||
</html>
|
</html>
|
||||||
@@ -17,9 +17,6 @@ builder.Services.AddSharedServices(apiBaseUrl);
|
|||||||
|
|
||||||
// Configure App settings
|
// Configure App settings
|
||||||
var appConfig = builder.Configuration.GetSection("App").Get<AppConfig>() ?? new AppConfig();
|
var appConfig = builder.Configuration.GetSection("App").Get<AppConfig>() ?? new AppConfig();
|
||||||
Console.WriteLine($"[DEBUG] AppConfig.AppName from config: {appConfig.AppName}");
|
|
||||||
Console.WriteLine($"[DEBUG] App:AppName from Configuration: {builder.Configuration["App:AppName"]}");
|
|
||||||
Console.WriteLine($"[DEBUG] App__AppName env var: {Environment.GetEnvironmentVariable("App__AppName")}");
|
|
||||||
builder.Services.AddSingleton(appConfig);
|
builder.Services.AddSingleton(appConfig);
|
||||||
|
|
||||||
builder.Services.AddScoped<IGoogleAuthService, WebGoogleAuthService>();
|
builder.Services.AddScoped<IGoogleAuthService, WebGoogleAuthService>();
|
||||||
|
|||||||
@@ -58,3 +58,93 @@ h1:focus {
|
|||||||
.mud-pagination li::marker {
|
.mud-pagination li::marker {
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Blazor Server Reconnection UI Customization */
|
||||||
|
#components-reconnect-modal {
|
||||||
|
position: fixed;
|
||||||
|
top: 0;
|
||||||
|
left: 0;
|
||||||
|
right: 0;
|
||||||
|
bottom: 0;
|
||||||
|
background: rgba(0, 0, 0, 0.5);
|
||||||
|
backdrop-filter: blur(4px);
|
||||||
|
z-index: 9999;
|
||||||
|
font-family: 'Roboto', 'Helvetica Neue', Helvetica, Arial, sans-serif;
|
||||||
|
display: none !important;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Show modal when Blazor applies these classes */
|
||||||
|
#components-reconnect-modal.components-reconnect-show,
|
||||||
|
#components-reconnect-modal.components-reconnect-failed,
|
||||||
|
#components-reconnect-modal.components-reconnect-rejected {
|
||||||
|
display: flex !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal .reconnect-content {
|
||||||
|
background: white;
|
||||||
|
border-radius: 8px;
|
||||||
|
padding: 32px;
|
||||||
|
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.2);
|
||||||
|
max-width: 400px;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal h5 {
|
||||||
|
margin: 0 0 16px 0;
|
||||||
|
color: #424242;
|
||||||
|
font-size: 20px;
|
||||||
|
font-weight: 500;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal .reconnect-message {
|
||||||
|
color: #666;
|
||||||
|
margin-bottom: 24px;
|
||||||
|
font-size: 14px;
|
||||||
|
line-height: 1.5;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal .reconnect-spinner {
|
||||||
|
width: 48px;
|
||||||
|
height: 48px;
|
||||||
|
border: 4px solid #f3f3f3;
|
||||||
|
border-top: 4px solid #e7163d;
|
||||||
|
border-radius: 50%;
|
||||||
|
animation: spin 1s linear infinite;
|
||||||
|
margin: 0 auto 16px;
|
||||||
|
}
|
||||||
|
|
||||||
|
@keyframes spin {
|
||||||
|
0% { transform: rotate(0deg); }
|
||||||
|
100% { transform: rotate(360deg); }
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal .reconnect-timer {
|
||||||
|
color: #e7163d;
|
||||||
|
font-size: 16px;
|
||||||
|
font-weight: 500;
|
||||||
|
margin-bottom: 16px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal button {
|
||||||
|
background-color: #e7163d;
|
||||||
|
color: white;
|
||||||
|
border: none;
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 10px 24px;
|
||||||
|
font-size: 14px;
|
||||||
|
font-weight: 500;
|
||||||
|
cursor: pointer;
|
||||||
|
transition: background-color 0.2s;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.5px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal button:hover {
|
||||||
|
background-color: #c01234;
|
||||||
|
}
|
||||||
|
|
||||||
|
#components-reconnect-modal button:active {
|
||||||
|
background-color: #a01028;
|
||||||
|
}
|
||||||
|
|||||||
82
DiunaBI.UI.Web/wwwroot/js/reconnect.js
Normal file
82
DiunaBI.UI.Web/wwwroot/js/reconnect.js
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
// Blazor Server Reconnection Timer
|
||||||
|
(function() {
|
||||||
|
let reconnectTimer = null;
|
||||||
|
let startTime = null;
|
||||||
|
|
||||||
|
function startTimer() {
|
||||||
|
if (reconnectTimer) return; // Already running
|
||||||
|
|
||||||
|
console.log('Blazor reconnection started, timer running...');
|
||||||
|
startTime = Date.now();
|
||||||
|
|
||||||
|
reconnectTimer = setInterval(() => {
|
||||||
|
const elapsedSeconds = Math.floor((Date.now() - startTime) / 1000);
|
||||||
|
const timerElement = document.getElementById('reconnect-elapsed-time');
|
||||||
|
|
||||||
|
if (timerElement) {
|
||||||
|
timerElement.textContent = `${elapsedSeconds}s`;
|
||||||
|
}
|
||||||
|
}, 1000);
|
||||||
|
}
|
||||||
|
|
||||||
|
function stopTimer() {
|
||||||
|
if (reconnectTimer) {
|
||||||
|
console.log('Blazor reconnection ended, stopping timer');
|
||||||
|
clearInterval(reconnectTimer);
|
||||||
|
reconnectTimer = null;
|
||||||
|
|
||||||
|
// Reset timer display
|
||||||
|
const timerElement = document.getElementById('reconnect-elapsed-time');
|
||||||
|
if (timerElement) {
|
||||||
|
timerElement.textContent = '0s';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function checkReconnectionState() {
|
||||||
|
const modal = document.getElementById('components-reconnect-modal');
|
||||||
|
|
||||||
|
if (!modal) return;
|
||||||
|
|
||||||
|
// Check if modal has the "show" class (Blazor applies this when reconnecting)
|
||||||
|
if (modal.classList.contains('components-reconnect-show')) {
|
||||||
|
startTimer();
|
||||||
|
} else {
|
||||||
|
stopTimer();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// MutationObserver to watch for class changes on the modal
|
||||||
|
const observer = new MutationObserver((mutations) => {
|
||||||
|
mutations.forEach((mutation) => {
|
||||||
|
if (mutation.type === 'attributes' && mutation.attributeName === 'class') {
|
||||||
|
checkReconnectionState();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Start observing when DOM is ready
|
||||||
|
function init() {
|
||||||
|
const modal = document.getElementById('components-reconnect-modal');
|
||||||
|
|
||||||
|
if (modal) {
|
||||||
|
observer.observe(modal, {
|
||||||
|
attributes: true,
|
||||||
|
attributeFilter: ['class']
|
||||||
|
});
|
||||||
|
|
||||||
|
// Check initial state
|
||||||
|
checkReconnectionState();
|
||||||
|
console.log('Blazor reconnection timer initialized');
|
||||||
|
} else {
|
||||||
|
console.warn('components-reconnect-modal not found, retrying...');
|
||||||
|
setTimeout(init, 100);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (document.readyState === 'loading') {
|
||||||
|
document.addEventListener('DOMContentLoaded', init);
|
||||||
|
} else {
|
||||||
|
init();
|
||||||
|
}
|
||||||
|
})();
|
||||||
Reference in New Issue
Block a user