Commit graph

14 commits

Author SHA1 Message Date
6d8a5d9ad7 Review 1.5: a Windows server that hangs no longer holds up the agent; reads are bounded and backed off
Deadlines
- Event log over RPC (MS-EVEN6): the whole read has a deadline (OperationTimeout per log);
  the token cancels the EventLogSession's operations and is checked before each
  FormatDescription, and the agent stops waiting even if the native call ignores it.
  Reported as ServerAccessProblem.Timeout ("svarede ikke i tide over hændelsesloggens
  protokol (RPC) inden for N sekunder").
- Every CIM call runs on the thread pool and is awaited with the caller's token; a call
  that ignores cancellation is abandoned and the CIM session is closed once it returns.
- Each server cycle has a deadline (new SQLInsight:Servers:CycleTimeout, default 5 min,
  1-30 min, at least 2 x OperationTimeout): the areas not read by then are "ikke målbar"
  ("Serveren svarede ikke inden for N minutter, så indsamlingen blev afbrudt; …"),
  logged once, and the slot is freed.
- Scheduled tasks: run information is read within a time budget (CycleTimeout / 3); the
  rest keep their state without the last run, and the coverage note says so.
- "Test server" has an overall deadline (WindowsServerTester, default 2 min); areas not
  done in time are reported as not answered, so the result is always stored.

Backoff and memory
- A server that does not answer (Unreachable/Timeout, or a timed-out cycle) is backed
  off 1, 2, 5, 10, then 15 minutes, reset when it answers.
- Protocol Auto: the protocol that worked (e.g. DCOM after WS-Man was refused, also for
  access denied) is tried first for an hour, then WS-Man is tried again; a server whose
  event log RPC failed is read over CIM for an hour.
- Over WS-Man with read-only WinRM rights Task Scheduler's GetInfoByName is denied: with
  Auto the tasks are read over DCOM instead (remembered for the server); with WSMan forced
  the reason says to choose Auto or DCOM. The advice "winrm configSDDL default" (broader
  WinRM rights) is gone; texts follow the grant script (-WinRM read right,
  -EnableFirewall -AgentAddress), not Remote Management Users.

Events
- Each log continues from its own mark (ServerEventSnapshot.LogReadUntil, a small additive
  field in Core): a truncated System log no longer loses the events between its last
  read event and the newest Application event.
- CIM fallback: Win32_NTLogEvent is streamed and only the oldest max per log are kept
  (memory bounded, deterministic order), consistent with the RPC read and the coverage
  text, which now names the logs that had more.

Other
- CimInstances (and method results) are mapped to SQLInsight's types and disposed at once.
- Options are clamped when bound (ServerOptions/SyntheticCheckOptions.Normalize):
  OperationTimeout 5 s-5 min, MaxEventsPerCollection 10-10000, MaxWatched 1-200,
  MaxParallelServers/Checks 1-32, retention days >= 1, ForecastHistoryDays 1-365,
  FailuresBeforeFinding/SlowRunsBeforeFinding 1-99.
- Shutdown: the timer's cancellation is caught, running cycles are awaited before the
  throttle is disposed; a cycle's time is taken when it gets its slot; a TaskCanceledException
  that is not shutdown no longer ends the service.
- Retention maintenance also runs (hourly) when SQLInsight:Servers:Enabled is false but
  synthetic checks are on.
- LocalCimTests run alone (collection without parallelization), share one warmed-up
  session, use the same 2 min operation timeout and a 5 min deadline per test, a smaller
  workload, and report timings in assertion messages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:02:17 +02:00
36ad4c0656 Repository and agent: Dynamics AX history (migration 0017), seeding, hub opt-in
0017_dynamics_ax adds insight.AxBatchRuns (completed batch runs, EventDays
retention) and insight.AxSequenceSamples (one NEXTREC/NEXTVAL sample per
sequence per day, HourlyDays retention). The agent reads them back after a
restart so job comparisons and forecasts continue, takes its thresholds from
SQLInsight:DynamicsAx, and only sends findings of the built-in SQLI-AX rules
to the hub for databases with the AX opt-in ticked; AX rule reasons (which
can name tables, logins and GRANT text) are replaced by a fixed text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 13:53:34 +02:00
f5ff42ca83 Review 1.3: an unreadable agent key file is kept, not overwritten
LoadOrCreate replaced a key file it could not decrypt, which made the old key
unrecoverable even when only the machine (DPAPI) had changed and the file could
have been put back. The file is now moved aside as credential-key.bin.unreadable-*
and the RSA object of the failed import is disposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 13:22:42 +02:00
29e1909e98 Review 1.3: one implementation of a service account's SQL login
RepositorySetup.LoginFor/IsLocal duplicated Core's ServiceAccounts, which the web
uses for the GRANT script. Setup now uses ServiceAccounts too, so the login the
installer grants and the one the web shows cannot drift apart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 13:22:41 +02:00
81b8376026 Agent: instances from the repository, request queue, sealed passwords, AX hub opt-in
The agent mirrors SQLInsight:Instances into the repository at start (read-only
"fra konfigurationsfil" rows, every database monitored as before) and reads the
configuration on every tick: instances added, paused, removed or changed in
SQLInsight Web, and database choices, take effect without a restart. If the
repository cannot be read before the first load, the configuration file alone
is used.

AgentRequestService runs connection tests (with the permission check), database
discovery and instance discovery (SQL Server Browser, registry) as the agent's
own identity, and refreshes every instance's database list hourly. The agent
keeps an RSA key (DPAPI LocalMachine, restricted folder) and publishes its public
key and identity; SQL login passwords sealed by the web UI are opened only here.

Dynamics AX findings and AX rule outcomes are sent to SQLInsight Hub only for
databases where it is turned on (default off).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 13:22:41 +02:00
0dc7866808 Separate "check by hand" from "not relevant" for version-gated rules
A rule whose topic applies but whose way of measuring needs a newer
version or patch now gives the new outcome ManualCheck with concrete
Danish guidance, instead of NotApplicable. It is marked explicitly per
rule (IRule.ManualCheck) and only used when every missing capability is
a measurement capability (InstanceCapabilities.IsMeasurement): CHECKDB
status (SQLI-DB-009), VLFs (SQLI-DB-011), spills and memory grants
(SQLI-PLAN-004/005) and deadlocks on 2005 (SQLI-DLK-001). Query Store,
AG and Azure-managed topics stay "not relevant".

Shown in a visible "Skal tjekkes manuelt" section on the instance page,
a KPI on the overview, a "Tjek manuelt" column in the management
report's controls, manualChecks in /api/v1/instances and manualCheck in
rule-states; not counted as not measurable.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 09:59:50 +02:00
7a908cc28b Explain and assess measurements for the customer
MetricAssessment rates CPU (SQL/other), batch requests, compilations
(against batches), page life expectancy (300 s per 4 GB buffer pool,
average and dips over 50 %, max server memory unset), user connections,
memory grants pending, processes blocked and disk latency as OK, Bemærk
or Reagér with a concrete next step and link, plus a plain Danish
"Hvad betyder det?" for each. Shown under every graph on the instance
page (now all eight metrics) and in the analysis, per drive in the disk
table, and in the analysis PDF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:57:10 +02:00
4ba7e5195e Privilege audit: sysadmin and server roles with AD unfolding
Collects members of the fixed server roles and CONTROL SERVER every 15
minutes with login type, disabled state and SQL login password policy.
Windows groups are unfolded recursively through Active Directory on a
Windows agent (cached, 20 s timeout, failures shown with the reason).
Changes since the previous collection are logged, and who/when is read
from the default trace (event 108) when the agent has the optional
ALTER TRACE; authors that only reach the buffered trace later are filled
in at the next collection. The first collection is the baseline.

New rules SQLI-SEC-001 (new privilege, High for sysadmin/securityadmin/
CONTROL SERVER) and SQLI-SEC-002 (more sysadmins than the limit, default
3). Names are sensitive evidence and never in titles or subjects.

Web: /instanser/{navn}/rettigheder and /rettigheder with change log by
period, alert banner on the instance page and overview, and an audit
PDF (/rapporter/rettigheder.pdf) linked from Rapporter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 08:39:56 +02:00
244153e9e4 Rules the version does not support are "not applicable", not "not measurable"
On SQL Server 2005 twenty rules stood as "not measurable" (Query Store,
system_health, memory columns …) between the real gaps, and counted in the
overview's "regler ikke målbare". They are not gaps: the version simply
lacks what they look at.

- New RuleOutcome.NotApplicable (last in the enum: outcomes are stored as
  numbers, so existing rows keep their meaning).
- A rule or collector missing only capabilities inherent to the instance
  (version-gated, plan cache memory columns, or Azure-managed) is not
  applicable, with the version each needs: "Ikke relevant for SQL Server
  2005: kræver QueryStore (fra SQL Server 2016)". Missing settings or rights
  (HADR not enabled …), collector failures and permission errors stay
  "not measurable". CollectedData.MarkNotApplicable carries this from
  collectors to the rules that use their data; community rules below their
  minimumVersion are not applicable too.
- Counts are unchanged in shape: the overview, KPI, JSON API and Prometheus
  count outcomes 2 and 3 only. The instance page lists the rest in a
  collapsed "Ikke relevant for SQL Server 2005 (n)" (details/summary, no
  JS) after the real gaps. Report controls whose core rule does not apply
  count neither as met nor as not measured.

Tests: engine (version vs. setting vs. collected data), inherent
capabilities, the existing version tests now expect not applicable, report
control status, and the web page and overview count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 16:05:39 +02:00
554a6a71f5 Time window analysis: what happened between 03 and 04, as a page and PDF
SQLInsight Web: /instanser/{navn}/analyse?fra=&til= (datetime-local in the
display time zone; presets last hour, tonight 00-06, yesterday; at most 31
days). "Analysér et tidsrum" on the instance page, and blocking/deadlock
times there open the window half an hour around the event.

For the window: load (CPU, batch requests, compilations, PLE, connections,
memory grants pending, blocked processes) from 5-minute samples within
Retention:RawDays, otherwise hourly history (shown on the page); waits per
minute against the same hours and weekday of the previous 4 weeks, with
clear deviations marked; top SQL sortable by CPU, duration, reads and
executions; file I/O latency; blocking chains; deadlocks; findings active
in the window. An empty window explains the retention. "Hent som PDF"
renders the same via MigraDoc (/analyse.pdf).

Docs: docs/analyse.md (also on the community site); databehandling.md: the
statement text stays in the customer's repository and is never sent to
the hub.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 15:46:56 +02:00
6d755aa3d0 Installer creates the repository database, logins and rights itself
Some checks failed
deploy community / deploy (push) Successful in 1s
deploy hub / deploy (push) Successful in 19s
CI / Build og unit tests (push) Successful in 8s
CI / MSI-installer (push) Successful in 15s
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Has been cancelled
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Has been cancelled
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Has been cancelled
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Has been cancelled
Entering a connection string in the installer is now enough: the new
command `SQLInsight.Agent.exe setup-repository` runs as a deferred custom
action after the services exist and sqlinsight.ini is written, and before
they start. It runs as the installing user on purpose (SQL Server must see
a person with rights, not SYSTEM) and:

- creates the database from Database=... if it is missing
- creates logins/users for the service accounts: agent db_owner, web
  db_datareader. NT SERVICE\... accounts arrive at another computer as
  DOMAIN\COMPUTER$, which then gets the login instead
- runs the schema migrations
- with MONITOR_SERVER: creates the agent's login there and applies
  grant-least-privilege.sql (embedded). A failure there is a warning only

A repository failure stops the installation; the reason goes to the
Application event log with a Danish hint (login, rights, certificate,
unreachable). SKIP_REPOSITORY_SETUP=1 leaves it to a DBA. Safe to rerun.

verify-msi.py allows this one action to impersonate and checks its
placement, Return=check and condition. Tested against a local SQL
Express (twice, idempotent) and with an unreachable server. Version 1.0.1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 08:18:26 +02:00
Claude
d764552c2c
MSI installer for Agent and Web
Some checks are pending
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Build og unit tests (push) Waiting to run
CI / MSI-installer (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
installer/SQLInsight.wxs (WiX 3 syntax) builds on Linux with wixl
(installer/build-msi.sh) and in Forgejo CI on every v* tag. Agent and Web
are installed self-contained as Windows services under virtual service
accounts (or a gMSA), with a settings page in the wizard or MSI
properties for silent installs; the values go to sqlinsight.ini, read
after appsettings.json. The data folder is locked to SYSTEM,
Administrators and the agent account, the event log source is
registered, the web port is opened in the firewall, and the automatic
update task is an optional feature. Upgrades happen in place, keep
edited settings and the chosen features; uninstall keeps the settings.

msi-tables.py writes what wixl lacks (component and feature conditions,
Permanent/NeverOverwrite, vital services, REG_EXPAND_SZ) and
verify-msi.py checks the result (sequencing, deferred actions, ASCII
only). test-msi-wine.sh runs install, upgrade and uninstall under Wine.
Installations made with the MSI get new versions as MSI packages run by
the update task, so Windows Installer stays in charge of the files.
The example instance is removed from the shipped appsettings.json, and
the CLI writes UTF-8 to the console.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 11:31:28 +00:00
Claude
d238580614
Fase 4: management and compliance reports as PDF
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
A PDF report for management and auditors (PDFsharp/MigraDoc, MIT, with
the Inter font embedded): key figures, a plain-language summary,
compliance controls backed by rules (met, not met, not measured),
instances and the critical and high findings. Downloadable from the
web UI (/rapporter) and mailed monthly by the agent with the PDF
attached, once per month, across daylight saving changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 10:09:10 +00:00
Claude
bfb27778d2
Fase 3: findings sync from agent to hub (queue, retry, data minimization)
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions
- Evidence can be marked sensitive (query text, logins, hosts, programs,
  job error messages); rules mark it. Sensitive values and the server
  name are sent as [udeladt] unless the customer allows them.
- Full-state submission contract (all active findings and rule outcomes
  per instance), idempotent by submission id
- Local outbound queue in the repository (migration 0002): newest state
  per instance only, exponential backoff up to an hour, permanent
  rejections (bad license, bad request) are given up, sent items purged
- Hub status (license state, last sent, last error) for the web UI
- Sync runs only with a valid license (or in grace); monitoring never
  depends on it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
2026-09-29 08:35:22 +00:00