Commit graph

15 commits

Author SHA1 Message Date
36ad4c0656 Repository and agent: Dynamics AX history (migration 0017), seeding, hub opt-in
0017_dynamics_ax adds insight.AxBatchRuns (completed batch runs, EventDays
retention) and insight.AxSequenceSamples (one NEXTREC/NEXTVAL sample per
sequence per day, HourlyDays retention). The agent reads them back after a
restart so job comparisons and forecasts continue, takes its thresholds from
SQLInsight:DynamicsAx, and only sends findings of the built-in SQLI-AX rules
to the hub for databases with the AX opt-in ticked; AX rule reasons (which
can name tables, logins and GRANT text) are replaced by a fixed text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 13:53:34 +02:00
6cc65e3c75 Index usage over time and never-updated statistics (8g, F)
The index collector now sums index reads and writes between collections
across restarts of SQL Server (and of the agent, by seeding from the
stored snapshot) and keeps them per day in insight.IndexUsageDaily
(migration 0015, HourlyDays). SQLI-IDX-002 judges an index once it has
been followed for 7 days instead of needing 7 days of uptime.
GET /api/v1/instances/{name}/index-usage lists usage per index.

SQLI-STAT-001 also reports statistics that were never updated on tables
with rows; statistic rows now come from the table's heap or clustered
index, as a never-updated statistic has no row count of its own.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 10:55:22 +02:00
4d18d20ab0 tempdb and version store (8c)
A five-minute collector reads tempdb space (free, version store, user and
internal objects, whether files can grow), the 'Version Store Size (KB)'
counter, the sessions using most tempdb, the oldest snapshot transaction
and, only when the version store is large, the databases generating
versions. Trend and heavy sessions survive restarts; used space and the
version store become metrics with explanation and assessment.

Rules SQLI-TEMP-001 (version store > 1 GB or steadily growing),
SQLI-TEMP-002 (nearly full and cannot grow) and SQLI-TEMP-003 (session
above 1 GB for 15 minutes), configurable under SQLInsight:Tempdb.
"tempdb lige nu" on the instance page; GET /api/v1/instances/{name}/tempdb.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 10:42:38 +02:00
6f7839d011 Long and large transactions and long-running statements (8b, 8d)
A one-minute collector reads open user transactions (age, log bytes used
and reserved, database, session, login, program, statement) and user
requests running longer than the threshold, leaving out the agent's own
and system sessions, BACKUP/RESTORE/DBCC (long operations) and requests
waiting on purpose. Transactions open a minute or holding 100 MB are kept
in insight.LongTransactions (migration 0014, RawDays).

Rules SQLI-TRAN-001 (open > 10 min, High at 6x), SQLI-TRAN-002 (> 1 GB
log) and SQLI-REQ-001 (statement > 15 min), configurable under
SQLInsight:Activity. Shown "right now" on the instance page and in the
time window analysis and PDF; GET /api/v1/instances/{name}/activity and
the analysis JSON leave logins and statements out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 10:36:41 +02:00
e6840e40d7 SQL Server error log without elevated rights (8a)
A five-minute collector reads important error log entries from the best
source the agent's rights allow: the error log itself (xp_readerrorlog,
only with securityadmin), the default trace's ErrorLog and Audit Login
Failed events (optional ALTER TRACE), or the exception and connectivity
ring buffers (VIEW SERVER STATE: numbers and severity, failed logins).
Repetitions are summed per error; the watermark is the newest entry
read, because the default trace is buffered. Stored in
insight.ErrorLogEvents (migration 0013, EventDays), resumed after restart.

Rules SQLI-LOG-001..006: corruption 823/824/825, I/O over 15 s (833),
severe errors (17+), login failure bursts (configurable), stack dumps and
AG errors; rules needing the log text are "not measurable" with the
right to grant. Page /instanser/{navn}/fejllog (menu Overvågning) and
GET /api/v1/instances/{name}/error-log with messages redacted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 10:30:41 +02:00
384e1f154f Long operations, full-text rules and statement activity
- Long operations: a one-minute collector reads operations with progress
  from sys.dm_exec_requests (BACKUP, RESTORE, DBCC, REORGANIZE, SHRINK,
  ROLLBACK): command, database, start, percent, expected end, login.
  Stored in insight.LongOperations (migration 0011, RawDays), shown on
  the instance page, in the time window analysis and its PDF. The last
  10 durations per command and database survive restarts; SQLI-OPS-001
  flags a backup/restore running over twice the median.
- Full-text community rules COM-FT-001..005 (metadata only, 2005 syntax):
  disk-full or paused/throttled catalogs, stale manual/off change
  tracking indexes and disabled indexes.
- Statement activity (CPU time / time in plan cache): column in top SQL
  and SQLI-CPU-002 when above SQLInsight:TopSql:ActivityThresholdPercent
  (default 50) for at least an hour. Never clears the plan cache.
- BuiltInCollectors.Create takes a CollectorSettings record.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 09:07:29 +02:00
20671bfb06 Wait type drill-down with active session sampling
A new light collector samples sys.dm_exec_requests every 15 seconds:
waiting user sessions (not the agent's own) by wait type, database and
statement (same key as top SQL), with blocking. Sampling-only cycles
reuse the last probe and skip rule evaluation. Stored in
insight.ActiveWaitSamples (migration 0010) for RawDays.

Wait types on the instance page and in the analysis link to
/instanser/{navn}/ventetyper/{type}: Danish explanation from a new
knowledge base (shared with the SQLI-WAIT rules, with Dynamics AX notes),
trend over time, the statements that waited with share and estimated
wait, a link to the full text (/instanser/{navn}/sql) and, for lock
waits, to blocking in the same window. SQL Server 2005 compatible.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:50:31 +02:00
4ba7e5195e Privilege audit: sysadmin and server roles with AD unfolding
Collects members of the fixed server roles and CONTROL SERVER every 15
minutes with login type, disabled state and SQL login password policy.
Windows groups are unfolded recursively through Active Directory on a
Windows agent (cached, 20 s timeout, failures shown with the reason).
Changes since the previous collection are logged, and who/when is read
from the default trace (event 108) when the agent has the optional
ALTER TRACE; authors that only reach the buffered trace later are filled
in at the next collection. The first collection is the baseline.

New rules SQLI-SEC-001 (new privilege, High for sysadmin/securityadmin/
CONTROL SERVER) and SQLI-SEC-002 (more sysadmins than the limit, default
3). Names are sensitive evidence and never in titles or subjects.

Web: /instanser/{navn}/rettigheder and /rettigheder with change log by
period, alert banner on the instance page and overview, and an audit
PDF (/rapporter/rettigheder.pdf) linked from Rapporter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:39:56 +02:00
ab8cfcad97 Configuration page with change log
- new collector instance-config: collation, edition, version, services and
  accounts (sys.dm_server_services, 2008 R2 SP1+), Instant File
  Initialization and Lock Pages in Memory (where the build has the
  columns), tempdb files, default paths; plus a fingerprint of the whole
  setup (sp_configure, trace flags, services, version, tempdb files)
- changes are computed against the previous collection, which the agent
  restores from the repository after a restart; migration 0008 keeps them
  in insight.ConfigurationChanges (as long as resolved findings)
- SQLI-CFG-013 "Konfiguration ændret": information for a day when
  sp_configure, a trace flag or a service account changes (account names
  are sensitive evidence)
- page /instanser/{navn}/konfiguration: instance, CPU and memory, services,
  tempdb, trace flags, non-default sp_configure (default vs. in use vs.
  configured) and the change log; linked from the instance page
- the management report lists configuration changes in the period
- trace flag reading shared by trace-flags and instance-config

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:12:26 +02:00
ac13e3d826 Disk speed per drive: IOPS, MB/s and latency
- file-io records each file's drive: sys.dm_os_volume_stats from 2008 R2 SP1
  (new capability VolumeStats, mount points included), otherwise derived
  from the path (D:\..., \server\share\, / on Linux), so 2005 works too
- migration 0007: Volume and IntervalSeconds on FileIoSamples, and
  DiskHourly (rolled up by maintenance, kept HourlyDays)
- DiskActivity sums files per drive: read/write IOPS, MB/s and latency
  weighted by I/O count
- instance page: "Diskhastighed pr. drev" (table and MB/s + latency charts);
  time window analysis and its PDF: per drive against the same hours of the
  four previous weeks
- SQLI-IO-001/002 now report per drive (one finding when the storage is
  slow, the slowest files in the evidence) instead of per file

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:03:11 +02:00
25a67cb98b Stop SQLInsight deadlocking itself, and stop reporting its own deadlocks
At a customer whose repository sits on a monitored instance, the agent
reported 12 deadlocks that were its own: two instances' monitors save
findings in parallel, one transaction each, and while insight.Findings is
small SQL Server scans it, so the two lock each other's rows.

- SaveRuleResultsAsync takes an exclusive application lock for its
  transaction (the agent is the only writer; a save takes milliseconds)
  and retries if chosen as deadlock victim anyway
- repository connections are named "SQLInsight Pro (repository)"
- deadlocks where every participant is SQLInsight Pro are not reported
  (the read position still moves past them); any other session keeps it

The new integration test (two instances saving 25 rules x 40 rounds in
parallel) failed 2 of 2 without the fix and passed 5 of 5 with it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 07:34:59 +02:00
bb3b38e77f Top SQL: collect what the heaviest statements use every 5 minutes
New collector query-stats reads sys.dm_exec_query_stats for the statements
that ran since the previous collection, grouped by query_hash (SQL Server
2008+) or sql_handle + offsets (2005, via legacy ODBC; literals only). Only
the 50 heaviest by CPU, duration, reads and executions, text cut to 4000
characters, no plan XML. The database comes from the plan's dbid attribute,
because ad hoc batches (as Dynamics AX sends) have none in sql_text.

QueryStatsDelta diffs against the previous collection: a statement seen
before with the same plan gets the difference, a plan compiled since then
counts in full, a recompiled or evicted plan or counters that went
backwards start over (never negative), a restart gives no delta. Up to 2000
statements are remembered between collections.

Repository migration 0006: insight.QueryStatSamples (deltas, top 100 per
collection) and insight.QueryTexts (text once per statement), kept for
Retention:RawDays. The text is customer data: never stored as a JSON
snapshot and never part of hub submissions. Window queries for metrics,
waits (raw and hourly), file I/O, blocking, deadlocks, findings active in a
window and top SQL by CPU/duration/reads/executions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 15:46:45 +02:00
Claude
eac799a1f1
Fase 2: notifications by e-mail, Teams and webhook
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
- Repository reports which findings opened and resolved per evaluation
- Agent notifies on new and resolved findings and on instance
  unreachable/recovered transitions, with links to the web UI
- Channels: e-mail (MailKit, HTML-encoded), Microsoft Teams Workflows
  webhook (Adaptive Card), generic JSON webhook signed with HMAC-SHA256
- Per-channel minimum severity (outages always sent), one message per
  batch, hourly cap per channel; a failing channel never stops monitoring
- Tested with a real SMTP server (Mailpit) and the real agent pipeline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 08:08:06 +00:00
Claude
5bb1c37fef
Fase 1: on-prem web UI (IIS or Windows service), light design system
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
- ASP.NET Core Blazor static SSR: overview with KPI tiles, instance page
  (findings, CPU/batch/PLE trends, top waits, file latency, blocking,
  deadlocks with .xdl download, not-measurable rules, resolved), finding
  detail with explanation, evidence, fix script and references
- Light-first design (sidebar, cards, self-hosted Inter, teal accent),
  optional dark theme via toggle; strict CSP, no inline styles
- Windows authentication (Negotiate/IIS) with optional group restriction;
  refuses to run without authentication outside Development
- Hosting: IIS (in-process ASP.NET Core Module), Windows service with
  Kestrel, or systemd; web reads the repository with db_datareader only
- docs/installation.md for repository, permissions, agent and web
- Fix: CPU used by other processes is not derived on Linux (SystemIdle is
  unreliable there); deadlocks are not re-reported or stored twice after
  an agent restart; Dapper sends datetime2 so time comparisons are exact

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 06:58:51 +00:00
Claude
9372c2e927
Fase 1: local repository with finding tracking, time series and retention
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
- Versioned, embedded migrations serialized with sp_getapplock so the agent
  and web UI can both migrate at startup
- Findings tracked per rule and subject: first seen, last seen, resolved;
  a rule that could not run never resolves its findings
- Time series (waits, file I/O, CPU, counters), blocking chains and
  deadlock graphs; idempotent inserts (timestamps truncated to match
  datetime2(0)), hourly aggregation and batched retention deletes
- Read API for the web UI; integration tests against SQL Server

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 06:23:38 +00:00