Søren Eggert Lundsteen Olsen
fdaa7e76bf
After the first read, BATCHJOBHISTORY was read from the watermark minus an overlap of 2000 RecIds with TOP (1000), oldest first. From AX 4.0 RecIds are handed out per table and are dense, so once 1000 rows existed in the overlap every read returned the same old rows: the watermark never moved and new runs were never read again (every AX 2009/2012 installation with more than ~1000 history rows). The read after the watermark now takes the overlap plus 1000 rows (TOP (3000)), so at least 1000 new runs are read each time. Test with 20,000 dense RecIds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
159 lines
14 KiB
Transact-SQL
159 lines
14 KiB
Transact-SQL
-- ===================================================================================================================
|
|
-- SQLI-AX-001: Dynamics AX-batchjob er endt med fejl
|
|
-- ===================================================================================================================
|
|
-- Regel-id: SQLI-AX-001
|
|
-- Titel: Dynamics AX-batchjob er endt med fejl
|
|
-- Alvor: High
|
|
-- Versioner: SQL Server 2005 og nyere (ikke 2000). Axapta 3.0: verificeret mod en kundes skema (docs/dynamics-ax/
|
|
-- schema-ax30.tsv). AX 4.0, AX 2009, AX 2012 og F&O: baseret på Microsofts dokumenterede skema - ikke
|
|
-- verificeret mod kundedata.
|
|
--
|
|
-- Hvad tjekkes, og hvorfor:
|
|
-- Et batchjob i Dynamics AX er stoppet med fejl. Det, jobbet skulle gøre (bogføring, lagerafslutning, integration
|
|
-- ...), er ikke gjort eller kun delvist, og gentagne job fortsætter ikke nødvendigvis af sig selv.
|
|
--
|
|
-- Vurdering (sker i C# ud fra de indsamlede data, ikke i SQL'en, medmindre andet er nævnt):
|
|
-- Et fund pr. job, når jobbet nu har status Fejl og sluttede (eller var planlagt) inden for grænsen, eller når jobbet
|
|
-- venter på næste gentagelse, men dets seneste kørsel i SQLInsights egen historik (insight.AxBatchRuns) endte med
|
|
-- fejl inden for grænsen.
|
|
-- STATUS oversættes af versionens adapter: AX 2009+ 0 Tilbageholdt, 1 Venter, 2 Udfører, 3 Fejl, 4 Afsluttet,
|
|
-- 5 Klar, 6 Ikke kørt, 7 Annullerer, 8 Annulleret; Axapta 3.0/4.0 (antaget, ikke verificeret mod kundedata)
|
|
-- 0 Venter, 1 Udfører, 2 Fejl, 3 Afsluttet, 4 Tilbageholdt. Højst 25 fund pr. database.
|
|
--
|
|
-- Grænseværdier (konfigurationsnøgle i agentens appsettings.json og standardværdi):
|
|
-- SQLInsight:DynamicsAx:BatchFailedLookbackHours = 24: fejl inden for så mange timer rapporteres.
|
|
--
|
|
-- Rettigheder til agentens login (se db/permissions/grant-least-privilege.sql og grant-dynamics-ax-read.sql):
|
|
-- Valgfri AX-læseadgang, kun SELECT (db/permissions/grant-dynamics-ax-read.sql; [bruger] = agentens databasebruger):
|
|
-- Axapta 3.0: GRANT SELECT ON [bmssa].[BATCH] TO [bruger];
|
|
-- AX 4.0: GRANT SELECT ON [dbo].[BATCH] TO [bruger];
|
|
-- AX 2009/2012/F&O: GRANT SELECT ON [dbo].[BATCHJOB] TO [bruger];
|
|
-- GRANT SELECT ON [dbo].[BATCHJOBHISTORY] TO [bruger];
|
|
-- Mangler den, er reglen "ikke målbar" med den nøjagtige GRANT for databasen (også under Dynamics AX → AX-databaser
|
|
-- og GET /api/v1/dynamics-ax/grant-script). Uden GRANT ændres intet.
|
|
-- Genkendelsen: kun grant-least-privilege.sql (VIEW ANY DEFINITION; CONNECT ANY DATABASE fra 2014, ellers en
|
|
-- databasebruger: CREATE USER [agent] FOR LOGIN [agent];).
|
|
--
|
|
-- Data kommer fra (collector, fil : medlemmer, interval):
|
|
-- AxDatabaseDetector (den fælles AX-genkendelse)
|
|
-- Fil: src/SQLInsight.Collectors/DynamicsAx/AxDatabaseDetector.cs
|
|
-- Medlemmer: AxDatabaseDetector.DatabasesSql, AxDatabaseDetector.BuildDetectionSql(database)
|
|
-- Interval: hver time pr. database (hvert 5. minut, mens AX-læseadgangen mangler), delt af alle AX-collectors
|
|
-- Delt script: SQL Scripts/_collectors/AxDatabaseDetector.sql
|
|
-- AxBatchCollector
|
|
-- Fil: src/SQLInsight.Collectors/DynamicsAx/AxBatchCollector.cs; SQL: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs
|
|
-- Medlemmer: AxSchemaAdapter.BatchJobs(database), AxSchemaAdapter.BatchHistory(database, afterRecId), AxSql.InDatabase
|
|
-- Interval: 5 min som standard (SQLInsight:CollectorIntervals:dynamics-ax-batch)
|
|
-- Delt script: SQL Scripts/_collectors/AxBatchCollector.sql
|
|
-- Regelkode: src/SQLInsight.Rules/DynamicsAx/DynamicsAxRules.cs : AxBatchFailedRule
|
|
--
|
|
-- Read-only: Ja: kun SELECT med TOP og SET-sessionsindstillinger. AX-tabellerne læses i AX-databasen via
|
|
-- [db].sys.sp_executesql (sessionen skifter aldrig database) under READ COMMITTED med SET LOCK_TIMEOUT 2000
|
|
-- (SNAPSHOT, når databasen tillader det), aldrig NOLOCK; bagefter sættes agentens sessionsindstillinger
|
|
-- igen. Intet ændrer data, indstillinger eller objekter. Kun i databaser genkendt som AX og valgt til
|
|
-- overvågning.
|
|
-- Kørsel: Kør i SSMS som agentens login (eller et login med samme rettigheder). Hver variant er sin egen batch
|
|
-- (GO); varianter for andre AX-versioner end din fejler eller giver ingen rækker - det er forventet.
|
|
-- AX-forespørgslerne er vist for databasen AXDB: ret [AXDB] til din AX-database.
|
|
-- ===================================================================================================================
|
|
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
-- Forbindelsen: Sessionsindstillinger, agenten sætter på hver forbindelse
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
|
|
-- Variant: Alle versioner: sættes på hver forbindelse, agenten åbner
|
|
-- Kilde: SqlConnectionFactory.SessionSettings
|
|
-- Kode: src/SQLInsight.Connectivity/SqlConnectionFactory.cs : SqlConnectionFactory.SessionSettings
|
|
SET LOCK_TIMEOUT 5000; SET DEADLOCK_PRIORITY LOW; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
-- AxDatabaseDetector: Hvilke overvågede databaser er Dynamics AX (kun metadata)
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
|
|
-- Variant: Alle versioner fra SQL Server 2005: online brugerdatabaser og deres snapshot-indstillinger
|
|
-- Kilde: AxDatabaseDetector.DatabasesSql
|
|
-- Kode: src/SQLInsight.Collectors/DynamicsAx/AxDatabaseDetector.cs : AxDatabaseDetector.DatabasesSql
|
|
SELECT name, snapshot_isolation_state, is_read_committed_snapshot_on
|
|
FROM sys.databases
|
|
WHERE database_id > 4 AND state = 0 AND source_database_id IS NULL AND HAS_DBACCESS(name) = 1
|
|
ORDER BY name
|
|
GO
|
|
|
|
-- Variant: SQL Server 2005 og nyere: AX-genkendelsen i hver overvåget database (kun metadata; vist for AXDB)
|
|
-- Kilde: AxDatabaseDetector.BuildDetectionSql
|
|
-- Kode: src/SQLInsight.Collectors/DynamicsAx/AxDatabaseDetector.cs : AxDatabaseDetector.BuildDetectionSql(database) = AxSql.InDatabase(database, AxDetection.MetadataSql)
|
|
-- Agenten kører dette i hver overvåget database fra listen ovenfor (ikke *_model). Ret [AXDB] til den database, du vil tjekke.
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT s.name, UPPER(t.name), UPPER(c.name), TYPE_NAME(c.system_type_id), c.precision, c.max_length,
|
|
ISNULL(HAS_PERMS_BY_NAME(QUOTENAME(s.name) + N''.'' + QUOTENAME(t.name), N''OBJECT'', N''SELECT''), 0),
|
|
(SELECT TOP 1 p.name FROM sys.database_principals p WHERE p.sid = SUSER_SID())
|
|
FROM sys.tables t
|
|
JOIN sys.schemas s ON s.schema_id = t.schema_id
|
|
JOIN sys.columns c ON c.object_id = t.object_id
|
|
WHERE UPPER(t.name) IN (N''SQLDICTIONARY'', N''DATAAREA'', N''SYSCONFIG'', N''BATCH'', N''BATCHJOB'', N''BATCHJOBHISTORY'', N''BATCHGROUP'', N''NUMBERSEQUENCETABLE'', N''NUMBERSEQUENCESCOPE'', N''NUMBERSEQUENCEREFERENCE'', N''SYSTEMSEQUENCES'', N''MODELELEMENT'')
|
|
ORDER BY s.name, t.name, c.column_id';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
-- AxBatchCollector: Batchjob og kørselshistorik i AX-databasen (valgfri AX-læseadgang)
|
|
-- -------------------------------------------------------------------------------------------------------------------
|
|
|
|
-- Variant: Axapta 3.0 (skema bmssa, verificeret mod kundedata): batchjob
|
|
-- Kilde: AxSchemaAdapter.BatchJobs(3.0)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchJobs(database) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(b.[RECID] AS bigint), b.[CLASSNUM], CAST(NULL AS nvarchar(60)), b.[COMPANY], b.[GROUPID], b.[STATUS], b.[STARTDATE], b.[STARTTIME], CAST(NULL AS datetime), CAST(NULL AS int), b.[ENDDATE], b.[ENDTIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [bmssa].[BATCH] b ORDER BY b.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 4.0 (skema dbo) - ikke verificeret mod kundedata: batchjob
|
|
-- Kilde: AxSchemaAdapter.BatchJobs(4.0)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchJobs(database) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(b.[RECID] AS bigint), b.[CLASSNUM], CAST(NULL AS nvarchar(60)), b.[COMPANY], b.[GROUPID], b.[STATUS], b.[STARTDATE], b.[STARTTIME], CAST(NULL AS datetime), CAST(NULL AS int), b.[ENDDATE], b.[ENDTIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCH] b ORDER BY b.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2009 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: batchjob
|
|
-- Kilde: AxSchemaAdapter.BatchJobs(2009)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchJobs(database) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(j.[RECID] AS bigint), CAST(NULL AS int), j.[CAPTION], j.[COMPANY], CAST(NULL AS nvarchar(20)), j.[STATUS], j.[ORIGSTARTDATETIME], CAST(NULL AS int), j.[STARTDATETIME], CAST(NULL AS int), j.[ENDDATETIME], CAST(NULL AS int), DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOB] j ORDER BY j.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2009 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: kørselshistorik, første læsning (de nyeste)
|
|
-- Kilde: AxSchemaAdapter.BatchHistory(2009, første)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchHistory(database, afterRecId: null) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(h.[RECID] AS bigint), CAST(h.[BATCHJOBID] AS bigint), h.[CAPTION], h.[COMPANY], h.[STATUS], h.[STARTDATETIME], h.[ENDDATETIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOBHISTORY] h ORDER BY h.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2009 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: kørselshistorik, de næste læsninger (fra den højeste RECID, SQLInsight har læst, minus overlappet)
|
|
-- Kilde: AxSchemaAdapter.BatchHistory(2009, næste)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchHistory(database, afterRecId) via AxSql.InDatabase
|
|
-- Agenten indsætter den højeste RECID, den har læst, minus overlappet, hvor der her står 0.
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (3000) CAST(h.[RECID] AS bigint), CAST(h.[BATCHJOBID] AS bigint), h.[CAPTION], h.[COMPANY], h.[STATUS], h.[STARTDATETIME], h.[ENDDATETIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOBHISTORY] h WHERE h.[RECID] > 0 ORDER BY h.[RECID]';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2012 RTM/R2/R3 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: batchjob
|
|
-- Kilde: AxSchemaAdapter.BatchJobs(2012)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchJobs(database) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(j.[RECID] AS bigint), CAST(NULL AS int), j.[CAPTION], j.[COMPANY], CAST(NULL AS nvarchar(20)), j.[STATUS], j.[ORIGSTARTDATETIME], CAST(NULL AS int), j.[STARTDATETIME], CAST(NULL AS int), j.[ENDDATETIME], CAST(NULL AS int), DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOB] j ORDER BY j.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2012 RTM/R2/R3 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: kørselshistorik, første læsning (de nyeste)
|
|
-- Kilde: AxSchemaAdapter.BatchHistory(2012, første)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchHistory(database, afterRecId: null) via AxSql.InDatabase
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (1000) CAST(h.[RECID] AS bigint), CAST(h.[BATCHJOBID] AS bigint), h.[CAPTION], h.[COMPANY], h.[STATUS], h.[STARTDATETIME], h.[ENDDATETIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOBHISTORY] h ORDER BY h.[RECID] DESC';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|
|
-- Variant: AX 2012 RTM/R2/R3 - baseret på Microsofts dokumenterede skema, ikke verificeret mod kundedata: kørselshistorik, de næste læsninger (fra den højeste RECID, SQLInsight har læst, minus overlappet)
|
|
-- Kilde: AxSchemaAdapter.BatchHistory(2012, næste)
|
|
-- Kode: src/SQLInsight.Core/DynamicsAx/AxSchemaAdapter.cs : AxSchemaAdapter.BatchHistory(database, afterRecId) via AxSql.InDatabase
|
|
-- Agenten indsætter den højeste RECID, den har læst, minus overlappet, hvor der her står 0.
|
|
EXEC [AXDB].sys.sp_executesql N'SET NOCOUNT ON; SET LOCK_TIMEOUT 2000; SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SELECT TOP (3000) CAST(h.[RECID] AS bigint), CAST(h.[BATCHJOBID] AS bigint), h.[CAPTION], h.[COMPANY], h.[STATUS], h.[STARTDATETIME], h.[ENDDATETIME], DATEDIFF(minute, GETDATE(), GETUTCDATE()) FROM [dbo].[BATCHJOBHISTORY] h WHERE h.[RECID] > 0 ORDER BY h.[RECID]';
|
|
SET LOCK_TIMEOUT 5000; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
|
|
GO
|
|
|