Commit graph

14 commits

Author SHA1 Message Date
ab8cfcad97 Configuration page with change log
- new collector instance-config: collation, edition, version, services and
  accounts (sys.dm_server_services, 2008 R2 SP1+), Instant File
  Initialization and Lock Pages in Memory (where the build has the
  columns), tempdb files, default paths; plus a fingerprint of the whole
  setup (sp_configure, trace flags, services, version, tempdb files)
- changes are computed against the previous collection, which the agent
  restores from the repository after a restart; migration 0008 keeps them
  in insight.ConfigurationChanges (as long as resolved findings)
- SQLI-CFG-013 "Konfiguration ændret": information for a day when
  sp_configure, a trace flag or a service account changes (account names
  are sensitive evidence)
- page /instanser/{navn}/konfiguration: instance, CPU and memory, services,
  tempdb, trace flags, non-default sp_configure (default vs. in use vs.
  configured) and the change log; linked from the instance page
- the management report lists configuration changes in the period
- trace flag reading shared by trace-flags and instance-config

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:12:26 +02:00
ac13e3d826 Disk speed per drive: IOPS, MB/s and latency
- file-io records each file's drive: sys.dm_os_volume_stats from 2008 R2 SP1
  (new capability VolumeStats, mount points included), otherwise derived
  from the path (D:\..., \server\share\, / on Linux), so 2005 works too
- migration 0007: Volume and IntervalSeconds on FileIoSamples, and
  DiskHourly (rolled up by maintenance, kept HourlyDays)
- DiskActivity sums files per drive: read/write IOPS, MB/s and latency
  weighted by I/O count
- instance page: "Diskhastighed pr. drev" (table and MB/s + latency charts);
  time window analysis and its PDF: per drive against the same hours of the
  four previous weeks
- SQLI-IO-001/002 now report per drive (one finding when the storage is
  slow, the slowest files in the evidence) instead of per file

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:03:11 +02:00
25a67cb98b Stop SQLInsight deadlocking itself, and stop reporting its own deadlocks
At a customer whose repository sits on a monitored instance, the agent
reported 12 deadlocks that were its own: two instances' monitors save
findings in parallel, one transaction each, and while insight.Findings is
small SQL Server scans it, so the two lock each other's rows.

- SaveRuleResultsAsync takes an exclusive application lock for its
  transaction (the agent is the only writer; a save takes milliseconds)
  and retries if chosen as deadlock victim anyway
- repository connections are named "SQLInsight Pro (repository)"
- deadlocks where every participant is SQLInsight Pro are not reported
  (the read position still moves past them); any other session keeps it

The new integration test (two instances saving 25 rules x 40 rounds in
parallel) failed 2 of 2 without the fix and passed 5 of 5 with it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 07:34:59 +02:00
25b8e95611 SQL Server 2005: fix agent jobs and plan analysis, guard against 2008+ syntax
Seen on a customer's SQL Server 2005 SP4 (legacy ODBC):
- agent-jobs declared @since with an initial value (2008+): "Incorrect syntax
  near CONVERT" and SQLI-JOB-001/002 never ran. Now DECLARE; SET.
- plan-analysis used query_hash (2008+) and failed every 30 minutes for
  SQLI-PLAN-001/002/003. On 2005 a statement is now identified by
  sql_handle + offsets, as top SQL does. It needs sys.dm_exec_text_query_plan,
  so the new capability TextQueryPlan (2005 SP2, 9.0.3042) is required.
- COM-DB-001 used OBJECT_SCHEMA_NAME (2005 SP2+); now joins sys.objects and
  sys.schemas.

Audit of all collector SQL and community rules without minimumVersion 10.0:
the rest is 2005 syntax or behind a version/capability check.

Sql2005CompatibilityTests (2005 cannot run in CI): every collector that can
run on 2005 is scanned for 2008+ syntax in its SQL (inline DECLARE, +=,
MERGE, CONCAT/IIF/FORMAT/TRY_*, OFFSET/FETCH, date/time types, multi-row
VALUES), version-branched files listed explicitly; the same for community
rules; and the 2005 variants of plan analysis and top SQL must not use
query_hash. Verified to fail on the old agent-jobs SQL.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 15:51:39 +02:00
bb3b38e77f Top SQL: collect what the heaviest statements use every 5 minutes
New collector query-stats reads sys.dm_exec_query_stats for the statements
that ran since the previous collection, grouped by query_hash (SQL Server
2008+) or sql_handle + offsets (2005, via legacy ODBC; literals only). Only
the 50 heaviest by CPU, duration, reads and executions, text cut to 4000
characters, no plan XML. The database comes from the plan's dbid attribute,
because ad hoc batches (as Dynamics AX sends) have none in sql_text.

QueryStatsDelta diffs against the previous collection: a statement seen
before with the same plan gets the difference, a plan compiled since then
counts in full, a recompiled or evicted plan or counters that went
backwards start over (never negative), a restart gives no delta. Up to 2000
statements are remembered between collections.

Repository migration 0006: insight.QueryStatSamples (deltas, top 100 per
collection) and insight.QueryTexts (text once per statement), kept for
Retention:RawDays. The text is customer data: never stored as a JSON
snapshot and never part of hub submissions. Window queries for metrics,
waits (raw and hourly), file I/O, blocking, deadlocks, findings active in a
window and top SQL by CPU/duration/reads/executions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 15:46:45 +02:00
Claude
7f640e74e5
Fase 4: community and local rules written as SQL
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
Rules can now be data: a JSON file with one read-only SELECT where every
row is a finding, with {column} templates, server or per-database scope,
minimum version, sensitive columns and references. Community rules live
in rules/community (reviewed via pull request), ship with every build
and with the signed rule pack; local rules are read from the agent's
data folder every five minutes. Queries are checked for anything that
could write, run with the least-privilege login, and a missing
permission makes the rule not measurable rather than failing. The
rule engine picks up catalog changes without a restart. Authors get
'rules validate' and 'rules test --instance'. Four example community
rules (sa active, orphaned database owners, default trace off,
untrusted foreign keys).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 10:23:01 +00:00
Claude
edb7410f76
Fase 2: Azure SQL Managed Instance, docs and rule catalog test
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
- Managed Instance reports 12.0.2000.8 but is evergreen: treated as the
  newest engine with all capabilities; new SelfManagedInfrastructure
  capability so backup, CHECKDB, files, tempdb, memory, patch level and
  lifecycle rules report 'handled by Azure' instead of false findings
- Microsoft Entra ID login (managed identity, service principal, default
  credential chain) via SqlConnection.AccessTokenCallback
- Least-privilege script recognizes Managed Instance; deadlock collector
  falls back to the ring buffer when event files cannot be read
- Web UI shows the right release for Managed Instance
- README and installation docs: notifications and Managed Instance
- Catalog test: 61 rules with unique, well-formed ids

Integration suite green on SQL Server 2017 and 2022.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 08:23:40 +00:00
Claude
15907401a9
Fase 2: Always On availability groups and SQL Agent jobs
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
- Availability group collector (replica and database replica states,
  queues, commit times) and rules: SQLI-AG-001 unhealthy/disconnected
  replica or suspended data movement (scripts HADR RESUME), SQLI-AG-002
  secondary behind (estimated data loss and redo time)
- SQL Agent collector using only granted msdb tables (no EXECUTE on msdb
  functions): SQLI-JOB-001 failed job with the failing step's message
  (backup/CHECKDB jobs are High), SQLI-JOB-002 SQL Agent not running
- Integration tests create a real availability group (CLUSTER_TYPE NONE)
  and a failing job in the test container (HADR and Agent enabled)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 07:53:49 +00:00
Claude
8cde94038c
Fase 2: index and statistics analysis
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
Collector (every 6 h, metadata and DMVs only, no scans): index inventory
with usage and forwarded fetches, missing-index suggestions, statistics
modifications via sysindexes.rowmodctr (needs only metadata visibility).
Rules:
- SQLI-IDX-001 missing index, evaluated against existing indexes:
  extends an index with the same keys instead of adding one; notes on
  write-heavy tables, many indexes, wide includes, short uptime
- SQLI-IDX-002 unused index (needs 7 days of uptime; suggests DISABLE)
- SQLI-IDX-003 duplicate or covered index (filters, uniqueness respected)
- SQLI-IDX-004 heap with forwarded records
- SQLI-STAT-001 stale statistics, SQLI-STAT-002 auto statistics off
Column lists use U+E000 as separator: FOR XML cannot serialize NCHAR(1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 07:47:53 +00:00
Claude
f1d3f5188e
Fase 2: plan cache analysis
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
Collector takes the 50 heaviest statements (CPU, last 24 h), fetches
statement plans as text (dm_exec_query_plan returns NULL for deeply
nested plans) and analyzes them in C#. Rules:
- SQLI-PLAN-001 implicit conversion preventing an index seek
- SQLI-PLAN-002 join without predicate (deliberate CROSS JOIN excluded)
- SQLI-PLAN-003 non-inlined scalar UDF in a heavy statement
- SQLI-PLAN-004 spills to tempdb, SQLI-PLAN-005 oversized memory grants
  (2016 SP2+ / 2017 CU3+ columns, new capability)
Verified against real plans in SQL Server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 07:40:09 +00:00
Claude
e4910aeba5
Fase 2: Query Store analysis (plan regressions, failed forced plans, parameter sensitivity)
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
Collector reads the 100 heaviest queries per Query Store database, split
into the last 24 hours and the week before. Rules:
- SQLI-QS-001 plan regression: new plan >= 2x slower than the best plan
  from the week before and >= 60 s extra per day; scripts the force
- SQLI-QS-002 forced plan that fails (e.g. NO_INDEX); scripts the unforce
- SQLI-QS-003 parameter sensitivity: very different plans or volatile
  durations; suggests PSP optimization / Query Store hints on 2022+
Also: backups taken in the same second a database was created count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 07:33:14 +00:00
Claude
5bb1c37fef
Fase 1: on-prem web UI (IIS or Windows service), light design system
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
- ASP.NET Core Blazor static SSR: overview with KPI tiles, instance page
  (findings, CPU/batch/PLE trends, top waits, file latency, blocking,
  deadlocks with .xdl download, not-measurable rules, resolved), finding
  detail with explanation, evidence, fix script and references
- Light-first design (sidebar, cards, self-hosted Inter, teal accent),
  optional dark theme via toggle; strict CSP, no inline styles
- Windows authentication (Negotiate/IIS) with optional group restriction;
  refuses to run without authentication outside Development
- Hosting: IIS (in-process ASP.NET Core Module), Windows service with
  Kestrel, or systemd; web reads the repository with db_datareader only
- docs/installation.md for repository, permissions, agent and web
- Fix: CPU used by other processes is not derived on Linux (SystemIdle is
  unreliable there); deadlocks are not re-reported or stored twice after
  an agent restart; Dapper sends datetime2 so time comparisons are exact

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 06:58:51 +00:00
Claude
9372c2e927
Fase 1: local repository with finding tracking, time series and retention
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
- Versioned, embedded migrations serialized with sp_getapplock so the agent
  and web UI can both migrate at startup
- Findings tracked per rule and subject: first seen, last seen, resolved;
  a rule that could not run never resolves its findings
- Time series (waits, file I/O, CPU, counters), blocking chains and
  deadlock graphs; idempotent inserts (timestamps truncated to match
  datetime2(0)), hourly aggregation and batched retention deletes
- Read API for the web UI; integration tests against SQL Server

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 06:23:38 +00:00
Claude
63f23c4bbd
Fase 1: collectors and rule pack (41 rules)
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
Collectors (SQL Server 2000+, version-specific SQL, least privilege):
server, configuration, trace flags, databases (settings, backups in UTC,
CHECKDB, VLFs, Query Store, files, tempdb), suspect pages, plan cache,
wait stats and file I/O as deltas with restart/clear detection,
performance counters, CPU from the scheduler monitor ring buffer,
blocking (sleeping head blockers with open transactions), and deadlocks
from system_health continuing from the last file offset.

Rules: configuration (memory, MAXDOP per Microsoft NUMA guidance, cost
threshold, ad hoc bloat, risky options, pending changes), databases
(state, settings, backups, log backups, CHECKDB, suspect pages, VLFs,
autogrowth, max size, Query Store, tempdb), waits (THREADPOOL, signal
waits, locks, I/O, WRITELOG, parallelism, PAGELATCH), I/O latency, CPU,
memory grants, blocking and deadlocks. Missing data makes a rule not
measurable with the collector's reason.

Integration tests run every collector as the least-privilege login and
reproduce real blocking and a real deadlock; green on 2017 and 2022.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 06:17:08 +00:00