Disk speed per drive: IOPS, MB/s and latency

- file-io records each file's drive: sys.dm_os_volume_stats from 2008 R2 SP1
  (new capability VolumeStats, mount points included), otherwise derived
  from the path (D:\..., \server\share\, / on Linux), so 2005 works too
- migration 0007: Volume and IntervalSeconds on FileIoSamples, and
  DiskHourly (rolled up by maintenance, kept HourlyDays)
- DiskActivity sums files per drive: read/write IOPS, MB/s and latency
  weighted by I/O count
- instance page: "Diskhastighed pr. drev" (table and MB/s + latency charts);
  time window analysis and its PDF: per drive against the same hours of the
  four previous weeks
- SQLI-IO-001/002 now report per drive (one finding when the storage is
  slow, the slowest files in the evidence) instead of per file

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Søren Eggert Lundsteen Olsen 2026-10-02 08:03:11 +02:00
parent c2cdb58a27
commit ac13e3d826
20 changed files with 559 additions and 44 deletions

View file

@ -13,6 +13,7 @@ Angiv fra og til (i webbrugerfladens tidszone, `SQLInsight:Web:TimeZone`), eller
| Belastning | Forløb af CPU (SQL Server og andre processer), batch requests og compilations pr. sekund, page life expectancy, brugerforbindelser, memory grants pending og blokerede processer | | Belastning | Forløb af CPU (SQL Server og andre processer), batch requests og compilations pr. sekund, page life expectancy, brugerforbindelser, memory grants pending og blokerede processer |
| Største ventetider | Ventetid pr. minut i tidsrummet, sammenlignet med normalen. Harmløse baggrundsventetider er udeladt | | Største ventetider | Ventetid pr. minut i tidsrummet, sammenlignet med normalen. Harmløse baggrundsventetider er udeladt |
| Tungeste forespørgsler | De forespørgsler, der brugte mest CPU, tid, læsninger eller blev kørt flest gange i tidsrummet, med tekst og database | | Tungeste forespørgsler | De forespørgsler, der brugte mest CPU, tid, læsninger eller blev kørt flest gange i tidsrummet, med tekst og database |
| Diskhastighed pr. drev | Læse- og skrive-IOPS, MB/s og gennemsnitlig latens pr. drev, sammenlignet med normalen (se nedenfor) |
| I/O-latens pr. fil | Læsninger, skrivninger og gennemsnitlig latens pr. databasefil | | I/O-latens pr. fil | Læsninger, skrivninger og gennemsnitlig latens pr. databasefil |
| Blocking og deadlocks | Blocking-kæder over 5 sekunder (head blocker, antal blokerede, længste ventetid) og deadlocks med graf (.xdl) | | Blocking og deadlocks | Blocking-kæder over 5 sekunder (head blocker, antal blokerede, længste ventetid) og deadlocks med graf (.xdl) |
| Fund | Fund, der var aktive på et tidspunkt i tidsrummet | | Fund | Fund, der var aktive på et tidspunkt i tidsrummet |
@ -26,11 +27,22 @@ Kun tidsrum, hvor agenten kørte og data stadig er gemt:
| Data | Opbevaring (standard, `SQLInsight:Retention`) | | Data | Opbevaring (standard, `SQLInsight:Retention`) |
|---|---| |---|---|
| Målinger hvert 5. minut: belastning, ventetider, I/O, forespørgsler | 14 dage (`RawDays`) | | Målinger hvert 5. minut: belastning, ventetider, I/O, forespørgsler | 14 dage (`RawDays`) |
| Timeværdier: belastning og ventetider | 400 dage (`HourlyDays`) | | Timeværdier: belastning, ventetider og diskhastighed pr. drev | 400 dage (`HourlyDays`) |
| Blocking og deadlocks | 90 dage (`EventDays`) | | Blocking og deadlocks | 90 dage (`EventDays`) |
Ligger tidsrummet længere tilbage end `RawDays`, bruges timeværdier. Det står på siden. Så er der ingen I/O pr. fil og ingen forespørgsler, men belastning og ventetider kan stadig sammenlignes. Ligger tidsrummet længere tilbage end `RawDays`, bruges timeværdier. Det står på siden. Så er der ingen I/O pr. fil og ingen forespørgsler, men belastning og ventetider kan stadig sammenlignes.
## Diskhastighed pr. drev
Agenten læser hvert 5. minut, hvor meget hver databasefil har læst og skrevet (`sys.dm_io_virtual_file_stats`: antal I/O'er, bytes og ventetid), og lægger filerne sammen pr. drev:
- **IOPS** = læsninger eller skrivninger pr. sekund, **MB/s** = data pr. sekund, **latens** = gennemsnitlig ventetid pr. læsning eller skrivning, vægtet efter antal I/O'er.
- Drevet kommer fra `sys.dm_os_volume_stats` (SQL Server 2008 R2 SP1 og nyere), så mount points vises rigtigt. På ældre versioner udledes det af filens sti (`D:\…` → `D:\`), og et mount point under et drevbogstav tælles da med drevet.
- Det er tal set fra SQL Server: kun SQL Servers egne filer, ikke andre programmer på samme drev.
- Instanssiden viser de seneste 24 timer med graf over MB/s og latens pr. drev. Fundene SQLI-IO-001 (læselatens på datafiler) og SQLI-IO-002 (skrivelatens på logfiler) gælder pr. drev, med de langsomste filer som målinger.
Et drev markeres med rødt, når datafiler har over 20 ms pr. læsning, logfiler over 5 ms pr. skrivning, eller latensen er mindst dobbelt så høj som normalt.
## Tungeste forespørgsler ## Tungeste forespørgsler
Agenten læser `sys.dm_exec_query_stats` hvert 5. minut (collector `query-stats`, kan ændres med `SQLInsight:CollectorIntervals`) og gemmer, hvad hver forespørgsel brugte siden sidst: Agenten læser `sys.dm_exec_query_stats` hvert 5. minut (collector `query-stats`, kan ændres med `SQLInsight:CollectorIntervals`) og gemmer, hvad hver forespørgsel brugte siden sidst:

View file

@ -8,12 +8,25 @@ public sealed class FileIoCollector : Collector<CumulativeSample<FileIoSnapshot,
internal const string Sql = """ internal const string Sql = """
SELECT DB_NAME(vfs.database_id), vfs.file_id, mf.type_desc, mf.physical_name, SELECT DB_NAME(vfs.database_id), vfs.file_id, mf.type_desc, mf.physical_name,
vfs.num_of_reads, vfs.num_of_bytes_read, vfs.io_stall_read_ms, vfs.num_of_reads, vfs.num_of_bytes_read, vfs.io_stall_read_ms,
vfs.num_of_writes, vfs.num_of_bytes_written, vfs.io_stall_write_ms vfs.num_of_writes, vfs.num_of_bytes_written, vfs.io_stall_write_ms,
CAST(NULL AS nvarchar(260))
FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs
JOIN sys.master_files mf ON mf.database_id = vfs.database_id AND mf.file_id = vfs.file_id JOIN sys.master_files mf ON mf.database_id = vfs.database_id AND mf.file_id = vfs.file_id
WHERE mf.type_desc IN (N'ROWS', N'LOG') WHERE mf.type_desc IN (N'ROWS', N'LOG')
"""; """;
/// <summary>With the real volume (mount points included), from 2008 R2 SP1.</summary>
internal const string SqlWithVolumes = """
SELECT DB_NAME(vfs.database_id), vfs.file_id, mf.type_desc, mf.physical_name,
vfs.num_of_reads, vfs.num_of_bytes_read, vfs.io_stall_read_ms,
vfs.num_of_writes, vfs.num_of_bytes_written, vfs.io_stall_write_ms,
vs.volume_mount_point
FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs
JOIN sys.master_files mf ON mf.database_id = vfs.database_id AND mf.file_id = vfs.file_id
OUTER APPLY sys.dm_os_volume_stats(vfs.database_id, vfs.file_id) vs
WHERE mf.type_desc IN (N'ROWS', N'LOG')
""";
public override string Name => "file-io"; public override string Name => "file-io";
public override TimeSpan DefaultInterval => TimeSpan.FromMinutes(5); public override TimeSpan DefaultInterval => TimeSpan.FromMinutes(5);
public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews]; public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews];
@ -23,18 +36,24 @@ public sealed class FileIoCollector : Collector<CumulativeSample<FileIoSnapshot,
{ {
var ct = context.CancellationToken; var ct = context.CancellationToken;
var startTime = await Db.StartTimeAsync(context.Connection, context.Instance.Version, ct).ConfigureAwait(false); var startTime = await Db.StartTimeAsync(context.Connection, context.Instance.Version, ct).ConfigureAwait(false);
var files = await Db.QueryAsync(context.Connection, Sql, r => new FileIoStat var sql = context.Capabilities.Has(Capability.VolumeStats) ? SqlWithVolumes : Sql;
var files = await Db.QueryAsync(context.Connection, sql, r =>
{ {
DatabaseName = r.NullableString(0) ?? "?", var physicalName = r.String(3);
FileId = r.Int(1), return new FileIoStat
FileType = r.String(2), {
PhysicalName = r.String(3), DatabaseName = r.NullableString(0) ?? "?",
Reads = r.Long(4), FileId = r.Int(1),
BytesRead = r.Long(5), FileType = r.String(2),
ReadStallMs = r.Long(6), PhysicalName = physicalName,
Writes = r.Long(7), Reads = r.Long(4),
BytesWritten = r.Long(8), BytesRead = r.Long(5),
WriteStallMs = r.Long(9), ReadStallMs = r.Long(6),
Writes = r.Long(7),
BytesWritten = r.Long(8),
WriteStallMs = r.Long(9),
Volume = r.NullableString(10) is { Length: > 0 } volume ? volume : DiskVolumes.FromPhysicalName(physicalName),
};
}, ct).ConfigureAwait(false); }, ct).ConfigureAwait(false);
var snapshot = new FileIoSnapshot { CapturedAt = context.Now, SqlServerStartTime = startTime, Files = files }; var snapshot = new FileIoSnapshot { CapturedAt = context.Now, SqlServerStartTime = startTime, Files = files };

View file

@ -19,6 +19,13 @@ public sealed record FileIoStat
public required string FileType { get; init; } public required string FileType { get; init; }
public required string PhysicalName { get; init; } public required string PhysicalName { get; init; }
/// <summary>
/// The volume (drive or mount point) the file lives on, e.g. <c>D:\</c>. From <c>sys.dm_os_volume_stats</c> where
/// the instance has it, otherwise derived from the path (<see cref="DiskVolumes.FromPhysicalName"/>).
/// </summary>
public string? Volume { get; init; }
public required long Reads { get; init; } public required long Reads { get; init; }
public required long BytesRead { get; init; } public required long BytesRead { get; init; }
public required long ReadStallMs { get; init; } public required long ReadStallMs { get; init; }
@ -30,6 +37,69 @@ public sealed record FileIoStat
public double? AverageWriteLatencyMs => Writes == 0 ? null : (double)WriteStallMs / Writes; public double? AverageWriteLatencyMs => Writes == 0 ? null : (double)WriteStallMs / Writes;
} }
/// <summary>
/// Which drive a database file is on, when SQL Server cannot say (before 2008 R2 SP1 there is no
/// <c>sys.dm_os_volume_stats</c>).
/// </summary>
public static class DiskVolumes
{
/// <summary>
/// <c>D:\Data\x.mdf</c> → <c>D:\</c>; <c>\\nas\sql\x.mdf</c> → <c>\\nas\sql\</c>; a Linux path → <c>/</c>. A mount point
/// under a drive letter cannot be told apart from the drive itself this way.
/// </summary>
public static string FromPhysicalName(string physicalName)
{
ArgumentNullException.ThrowIfNull(physicalName);
var path = physicalName.Trim();
if (path.Length >= 2 && char.IsAsciiLetter(path[0]) && path[1] == ':')
{
return char.ToUpperInvariant(path[0]) + @":\";
}
if (path.StartsWith(@"\\", StringComparison.Ordinal))
{
var parts = path[2..].Split('\\', StringSplitOptions.RemoveEmptyEntries);
return parts.Length >= 2 ? $@"\\{parts[0]}\{parts[1]}\" : path;
}
return path.StartsWith('/') ? "/" : path;
}
/// <summary>The file's volume as reported, or derived from its path.</summary>
public static string Of(FileIoStat file)
{
ArgumentNullException.ThrowIfNull(file);
return string.IsNullOrWhiteSpace(file.Volume) ? FromPhysicalName(file.PhysicalName) : file.Volume;
}
}
/// <summary>
/// I/O on one drive over an interval: the sum of its database files. Latency is weighted by the number of I/Os.
/// </summary>
public sealed record DiskActivity(string Volume, int DataFiles, int LogFiles, long Reads, long Writes, long BytesRead, long BytesWritten,
long ReadStallMs, long WriteStallMs, double Seconds)
{
public double? ReadIops => Seconds > 0 ? Reads / Seconds : null;
public double? WriteIops => Seconds > 0 ? Writes / Seconds : null;
public double? ReadMbPerSecond => Seconds > 0 ? BytesRead / 1048576.0 / Seconds : null;
public double? WriteMbPerSecond => Seconds > 0 ? BytesWritten / 1048576.0 / Seconds : null;
public double? AverageReadMs => Reads == 0 ? null : (double)ReadStallMs / Reads;
public double? AverageWriteMs => Writes == 0 ? null : (double)WriteStallMs / Writes;
/// <summary>Groups the files of one interval by drive.</summary>
public static IReadOnlyList<DiskActivity> ByVolume(IEnumerable<FileIoStat> files, double seconds)
{
ArgumentNullException.ThrowIfNull(files);
return files
.GroupBy(DiskVolumes.Of, StringComparer.OrdinalIgnoreCase)
.Select(g => new DiskActivity(g.Key, g.Count(f => f.FileType != "LOG"), g.Count(f => f.FileType == "LOG"),
g.Sum(f => f.Reads), g.Sum(f => f.Writes), g.Sum(f => f.BytesRead), g.Sum(f => f.BytesWritten),
g.Sum(f => f.ReadStallMs), g.Sum(f => f.WriteStallMs), seconds))
.OrderBy(d => d.Volume, StringComparer.OrdinalIgnoreCase)
.ToList();
}
}
/// <summary> /// <summary>
/// I/O per file between two snapshots. /// I/O per file between two snapshots.
/// </summary> /// </summary>

View file

@ -64,4 +64,7 @@ public enum Capability
/// <summary><c>sys.dm_exec_text_query_plan</c> (cached plans as text, per statement). SQL Server 2005 SP2+.</summary> /// <summary><c>sys.dm_exec_text_query_plan</c> (cached plans as text, per statement). SQL Server 2005 SP2+.</summary>
TextQueryPlan, TextQueryPlan,
/// <summary><c>sys.dm_os_volume_stats</c> and <c>sys.dm_server_services</c>. SQL Server 2008 R2 SP1+.</summary>
VolumeStats,
} }

View file

@ -13,6 +13,7 @@ public sealed class InstanceCapabilities
[Capability.DefaultTrace] = new(9, 0, 0, 0), [Capability.DefaultTrace] = new(9, 0, 0, 0),
[Capability.ViewServerStatePermission] = new(9, 0, 0, 0), [Capability.ViewServerStatePermission] = new(9, 0, 0, 0),
[Capability.TextQueryPlan] = new(9, 0, 3042, 0), // 2005 SP2 [Capability.TextQueryPlan] = new(9, 0, 3042, 0), // 2005 SP2
[Capability.VolumeStats] = new(10, 50, 2500, 0), // 2008 R2 SP1
[Capability.ExtendedEvents] = new(10, 0, 0, 0), [Capability.ExtendedEvents] = new(10, 0, 0, 0),
[Capability.SystemHealthSession] = new(10, 0, 0, 0), [Capability.SystemHealthSession] = new(10, 0, 0, 0),
[Capability.SystemHealthFileTarget] = new(11, 0, 0, 0), [Capability.SystemHealthFileTarget] = new(11, 0, 0, 0),

View file

@ -162,4 +162,38 @@ public static class WindowBaseline
var values = earlierWeeks.SelectMany(w => w).Select(p => p.Value).ToList(); var values = earlierWeeks.SelectMany(w => w).Select(p => p.Value).ToList();
return values.Count == 0 ? null : values.Average(); return values.Count == 0 ? null : values.Average();
} }
/// <summary>
/// Each drive in the window against the same hours of earlier weeks: all earlier weeks' I/O pooled, so latency is
/// weighted by the number of I/Os and rates by the time measured.
/// </summary>
public static IReadOnlyList<DiskComparison> CompareDisks(IReadOnlyList<Core.Data.DiskActivity> window, IReadOnlyList<IReadOnlyList<Core.Data.DiskActivity>> earlierWeeks)
{
ArgumentNullException.ThrowIfNull(window);
ArgumentNullException.ThrowIfNull(earlierWeeks);
return window
.Select(d =>
{
var earlier = earlierWeeks.SelectMany(w => w).Where(e => string.Equals(e.Volume, d.Volume, StringComparison.OrdinalIgnoreCase)).ToList();
Core.Data.DiskActivity? normal = earlier.Count == 0 || earlier.Sum(e => e.Seconds) <= 0
? null
: new Core.Data.DiskActivity(d.Volume, 0, 0, earlier.Sum(e => e.Reads), earlier.Sum(e => e.Writes), earlier.Sum(e => e.BytesRead),
earlier.Sum(e => e.BytesWritten), earlier.Sum(e => e.ReadStallMs), earlier.Sum(e => e.WriteStallMs), earlier.Sum(e => e.Seconds));
return new DiskComparison(d, normal);
})
.ToList();
}
}
/// <summary>A drive's I/O in the window compared with the same hours of the week in the previous weeks.</summary>
public sealed record DiskComparison(Core.Data.DiskActivity Window, Core.Data.DiskActivity? Normal)
{
/// <summary>Data files over 20 ms per read or log files over 5 ms per write: slow whatever the history says.</summary>
public bool IsSlow => Window.AverageReadMs >= 20 && Window.Reads >= 100 || Window.AverageWriteMs >= (Window.DataFiles == 0 ? 5 : 20) && Window.Writes >= 100;
/// <summary>Latency at least twice the normal (and noticeably so), or a slow drive.</summary>
public bool IsUnusual => IsSlow
|| Worse(Window.AverageReadMs, Normal?.AverageReadMs) || Worse(Window.AverageWriteMs, Normal?.AverageWriteMs);
private static bool Worse(double? now, double? normal) => now is { } n && normal is > 0 and var m && n >= m * 2 && n - m >= 2;
} }

View file

@ -16,6 +16,10 @@ public sealed record TimeWindowAnalysis
public required IReadOnlyList<MetricComparison> Metrics { get; init; } public required IReadOnlyList<MetricComparison> Metrics { get; init; }
public required IReadOnlyList<WaitComparison> Waits { get; init; } public required IReadOnlyList<WaitComparison> Waits { get; init; }
public required IReadOnlyList<FileLatency> Files { get; init; } public required IReadOnlyList<FileLatency> Files { get; init; }
/// <summary>Disk speed per drive, against the normal for the same hours.</summary>
public IReadOnlyList<DiskComparison> Disks { get; init; } = [];
public required IReadOnlyList<StoredBlocking> Blocking { get; init; } public required IReadOnlyList<StoredBlocking> Blocking { get; init; }
public required IReadOnlyList<StoredDeadlock> Deadlocks { get; init; } public required IReadOnlyList<StoredDeadlock> Deadlocks { get; init; }
public required IReadOnlyList<StoredFinding> Findings { get; init; } public required IReadOnlyList<StoredFinding> Findings { get; init; }
@ -24,7 +28,7 @@ public sealed record TimeWindowAnalysis
/// <summary>The heaviest statements. Their text is customer data: only for the local UI and PDF.</summary> /// <summary>The heaviest statements. Their text is customer data: only for the local UI and PDF.</summary>
public required IReadOnlyList<TopQuery> TopQueries { get; init; } public required IReadOnlyList<TopQuery> TopQueries { get; init; }
public bool HasData => Metrics.Any(m => m.Points.Count > 0) || Waits.Count > 0 || Files.Count > 0 || Blocking.Count > 0 public bool HasData => Metrics.Any(m => m.Points.Count > 0) || Waits.Count > 0 || Files.Count > 0 || Disks.Count > 0 || Blocking.Count > 0
|| Deadlocks.Count > 0 || TopQueries.Count > 0; || Deadlocks.Count > 0 || TopQueries.Count > 0;
/// <summary>The metrics shown, in order: key, label, unit, decimals, lower-is-worse, fixed maximum.</summary> /// <summary>The metrics shown, in order: key, label, unit, decimals, lower-is-worse, fixed maximum.</summary>
@ -80,11 +84,17 @@ public sealed record TimeWindowAnalysis
} }
var earlierWaits = new List<IReadOnlyList<HourlyWaitTotal>>(); var earlierWaits = new List<IReadOnlyList<HourlyWaitTotal>>();
var earlierDisks = new List<IReadOnlyList<Core.Data.DiskActivity>>();
foreach (var back in weeks) foreach (var back in weeks)
{ {
earlierWaits.Add(await repository.GetWaitsHourlyAsync(id, from - back, to - back, cancellationToken).ConfigureAwait(false)); earlierWaits.Add(await repository.GetWaitsHourlyAsync(id, from - back, to - back, cancellationToken).ConfigureAwait(false));
earlierDisks.Add(await repository.GetDiskActivityHourlyAsync(id, from - back, to - back, cancellationToken).ConfigureAwait(false));
} }
var windowDisks = resolution == WindowResolution.Raw
? await repository.GetDiskActivityAsync(id, from, to, cancellationToken).ConfigureAwait(false)
: await repository.GetDiskActivityHourlyAsync(id, from, to, cancellationToken).ConfigureAwait(false);
return new TimeWindowAnalysis return new TimeWindowAnalysis
{ {
Instance = instance, Instance = instance,
@ -95,6 +105,7 @@ public sealed record TimeWindowAnalysis
Metrics = metrics, Metrics = metrics,
Waits = WindowBaseline.CompareWaits(windowWaits, minutes, earlierWaits), Waits = WindowBaseline.CompareWaits(windowWaits, minutes, earlierWaits),
Files = await repository.GetFileLatencyAsync(id, from, to, cancellationToken).ConfigureAwait(false), Files = await repository.GetFileLatencyAsync(id, from, to, cancellationToken).ConfigureAwait(false),
Disks = WindowBaseline.CompareDisks(windowDisks, earlierDisks),
Blocking = await repository.GetBlockingAsync(id, from, to, cancellationToken).ConfigureAwait(false), Blocking = await repository.GetBlockingAsync(id, from, to, cancellationToken).ConfigureAwait(false),
Deadlocks = await repository.GetDeadlocksAsync(id, from, to, cancellationToken).ConfigureAwait(false), Deadlocks = await repository.GetDeadlocksAsync(id, from, to, cancellationToken).ConfigureAwait(false),
Findings = await repository.GetFindingsActiveDuringAsync(id, from, to, cancellationToken).ConfigureAwait(false), Findings = await repository.GetFindingsActiveDuringAsync(id, from, to, cancellationToken).ConfigureAwait(false),

View file

@ -65,6 +65,7 @@ public static class TimeWindowReport
Metrics(section, analysis); Metrics(section, analysis);
Waits(section, analysis); Waits(section, analysis);
TopQueries(section, analysis); TopQueries(section, analysis);
Disks(section, analysis);
Files(section, analysis); Files(section, analysis);
Blocking(section, analysis, zone); Blocking(section, analysis, zone);
Deadlocks(section, analysis, zone); Deadlocks(section, analysis, zone);
@ -144,6 +145,29 @@ public static class TimeWindowReport
} }
} }
private static void Disks(Section section, TimeWindowAnalysis analysis)
{
section.AddParagraph("Diskhastighed pr. drev", StyleNames.Heading1);
if (analysis.Disks.Count == 0)
{
section.AddParagraph("Ingen målinger pr. drev i tidsrummet (indsamles fra version 1.0.5).", "Muted");
return;
}
var table = Table(section, [3.2, 2.2, 2.2, 2.2, 2.2, 2.6, 2.8]);
Header(table, "Drev", "Læs IOPS", "Skriv IOPS", "Læs MB/s", "Skriv MB/s", "Gns. læs", "Gns. skriv");
foreach (var d in analysis.Disks)
{
var w = d.Window;
Row(table, d.IsUnusual, w.Volume, Number(w.ReadIops, 1), Number(w.WriteIops, 1), Number(w.ReadMbPerSecond, 1), Number(w.WriteMbPerSecond, 1),
Ms(w.AverageReadMs) + Normal(d.Normal?.AverageReadMs), Ms(w.AverageWriteMs) + Normal(d.Normal?.AverageWriteMs));
}
section.AddParagraph("Latens i parentes: det normale for samme klokkeslæt de foregående uger.", "Muted");
static string Normal(double? value) => value is { } v ? $" ({v.ToString("N1", Danish)})" : "";
}
private static void Files(Section section, TimeWindowAnalysis analysis) private static void Files(Section section, TimeWindowAnalysis analysis)
{ {
section.AddParagraph("I/O-latens pr. fil", StyleNames.Heading1); section.AddParagraph("I/O-latens pr. fil", StyleNames.Heading1);

View file

@ -0,0 +1,63 @@
using Dapper;
using SQLInsight.Core.Data;
namespace SQLInsight.Repository;
/// <summary>Throughput and latency of one drive over time, for the charts.</summary>
public sealed record DiskSeries(string Volume, IReadOnlyList<MetricPoint> MbPerSecond, IReadOnlyList<MetricPoint> LatencyMs);
public sealed partial class InsightRepository
{
/// <summary>
/// I/O per drive in [from, to], from the 5-minute samples. Each sample's interval counts once per drive, so IOPS and
/// MB/s are averages over the time actually measured. Samples from before disk speed was collected have no drive.
/// </summary>
public async Task<IReadOnlyList<DiskActivity>> GetDiskActivityAsync(int instanceId, DateTimeOffset from, DateTimeOffset to, CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);
var rows = await connection.QueryAsync<(string Volume, int DataFiles, int LogFiles, long Reads, long Writes, long BytesRead, long BytesWritten, long ReadStallMs, long WriteStallMs, int Seconds)>("""
SELECT Volume, MAX(DataFiles), MAX(LogFiles), SUM(Reads), SUM(Writes), SUM(BytesRead), SUM(BytesWritten), SUM(ReadStallMs), SUM(WriteStallMs), SUM(Seconds)
FROM (
SELECT Volume, SampledAt,
COUNT(CASE WHEN FileType <> 'LOG' THEN 1 END) AS DataFiles, COUNT(CASE WHEN FileType = 'LOG' THEN 1 END) AS LogFiles,
SUM(Reads) AS Reads, SUM(Writes) AS Writes, SUM(BytesRead) AS BytesRead, SUM(BytesWritten) AS BytesWritten,
SUM(ReadStallMs) AS ReadStallMs, SUM(WriteStallMs) AS WriteStallMs, MAX(IntervalSeconds) AS Seconds
FROM insight.FileIoSamples
WHERE InstanceId = @instanceId AND SampledAt > @from AND SampledAt <= @to AND Volume IS NOT NULL AND IntervalSeconds IS NOT NULL
GROUP BY Volume, SampledAt
) AS samples
GROUP BY Volume ORDER BY Volume;
""", new { instanceId, from = from.UtcDateTime, to = to.UtcDateTime }).ConfigureAwait(false);
return rows.Select(r => new DiskActivity(r.Volume, r.DataFiles, r.LogFiles, r.Reads, r.Writes, r.BytesRead, r.BytesWritten, r.ReadStallMs, r.WriteStallMs, r.Seconds)).ToList();
}
/// <summary>I/O per drive from the hourly history for the hours overlapping [from, to).</summary>
public async Task<IReadOnlyList<DiskActivity>> GetDiskActivityHourlyAsync(int instanceId, DateTimeOffset from, DateTimeOffset to, CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);
var rows = await connection.QueryAsync<(string Volume, long Reads, long Writes, long BytesRead, long BytesWritten, long ReadStallMs, long WriteStallMs, int Seconds)>("""
SELECT Volume, SUM(Reads), SUM(Writes), SUM(BytesRead), SUM(BytesWritten), SUM(ReadStallMs), SUM(WriteStallMs), SUM(Seconds)
FROM insight.DiskHourly WHERE InstanceId = @instanceId AND [Hour] >= @from AND [Hour] < @to
GROUP BY Volume ORDER BY Volume;
""", new { instanceId, from = FloorHour(from), to = to.UtcDateTime }).ConfigureAwait(false);
return rows.Select(r => new DiskActivity(r.Volume, 0, 0, r.Reads, r.Writes, r.BytesRead, r.BytesWritten, r.ReadStallMs, r.WriteStallMs, r.Seconds)).ToList();
}
/// <summary>MB/s (read + write) and average latency per drive per sample since <paramref name="from"/>.</summary>
public async Task<IReadOnlyList<DiskSeries>> GetDiskSeriesAsync(int instanceId, DateTimeOffset from, CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);
var rows = await connection.QueryAsync<(string Volume, DateTime SampledAt, long Ios, long Bytes, long StallMs, int Seconds)>("""
SELECT Volume, SampledAt, SUM(Reads + Writes), SUM(BytesRead + BytesWritten), SUM(ReadStallMs + WriteStallMs), MAX(IntervalSeconds)
FROM insight.FileIoSamples
WHERE InstanceId = @instanceId AND SampledAt >= @from AND Volume IS NOT NULL AND IntervalSeconds IS NOT NULL
GROUP BY Volume, SampledAt ORDER BY Volume, SampledAt;
""", new { instanceId, from = from.UtcDateTime }).ConfigureAwait(false);
return rows.GroupBy(r => r.Volume)
.Select(g => new DiskSeries(
g.Key,
g.Where(r => r.Seconds > 0).Select(r => new MetricPoint(r.SampledAt, r.Bytes / 1048576.0 / r.Seconds)).ToList(),
g.Where(r => r.Ios > 0).Select(r => new MetricPoint(r.SampledAt, (double)r.StallMs / r.Ios)).ToList()))
.ToList();
}
}

View file

@ -53,6 +53,24 @@ public sealed partial class InsightRepository
) AS s ON t.InstanceId = s.InstanceId AND t.[Hour] = s.[Hour] AND t.WaitType = s.WaitType ) AS s ON t.InstanceId = s.InstanceId AND t.[Hour] = s.[Hour] AND t.WaitType = s.WaitType
WHEN MATCHED THEN UPDATE SET WaitingTasks = s.WaitingTasks, WaitTimeMs = s.WaitTimeMs, SignalWaitTimeMs = s.SignalWaitTimeMs WHEN MATCHED THEN UPDATE SET WaitingTasks = s.WaitingTasks, WaitTimeMs = s.WaitTimeMs, SignalWaitTimeMs = s.SignalWaitTimeMs
WHEN NOT MATCHED THEN INSERT (InstanceId, [Hour], WaitType, WaitingTasks, WaitTimeMs, SignalWaitTimeMs) VALUES (s.InstanceId, s.[Hour], s.WaitType, s.WaitingTasks, s.WaitTimeMs, s.SignalWaitTimeMs); WHEN NOT MATCHED THEN INSERT (InstanceId, [Hour], WaitType, WaitingTasks, WaitTimeMs, SignalWaitTimeMs) VALUES (s.InstanceId, s.[Hour], s.WaitType, s.WaitingTasks, s.WaitTimeMs, s.SignalWaitTimeMs);
MERGE insight.DiskHourly AS t
USING (
SELECT InstanceId, [Hour], Volume, SUM(Reads) AS Reads, SUM(Writes) AS Writes, SUM(BytesRead) AS BytesRead, SUM(BytesWritten) AS BytesWritten,
SUM(ReadStallMs) AS ReadStallMs, SUM(WriteStallMs) AS WriteStallMs, SUM(Seconds) AS Seconds
FROM (
SELECT InstanceId, DATEADD(hour, DATEDIFF(hour, 0, SampledAt), 0) AS [Hour], Volume, SampledAt,
SUM(Reads) AS Reads, SUM(Writes) AS Writes, SUM(BytesRead) AS BytesRead, SUM(BytesWritten) AS BytesWritten,
SUM(ReadStallMs) AS ReadStallMs, SUM(WriteStallMs) AS WriteStallMs, MAX(IntervalSeconds) AS Seconds
FROM insight.FileIoSamples WHERE SampledAt >= @from AND Volume IS NOT NULL AND IntervalSeconds IS NOT NULL
GROUP BY InstanceId, Volume, SampledAt
) AS samples
GROUP BY InstanceId, [Hour], Volume
) AS s ON t.InstanceId = s.InstanceId AND t.[Hour] = s.[Hour] AND t.Volume = s.Volume
WHEN MATCHED THEN UPDATE SET Reads = s.Reads, Writes = s.Writes, BytesRead = s.BytesRead, BytesWritten = s.BytesWritten,
ReadStallMs = s.ReadStallMs, WriteStallMs = s.WriteStallMs, Seconds = s.Seconds
WHEN NOT MATCHED THEN INSERT (InstanceId, [Hour], Volume, Reads, Writes, BytesRead, BytesWritten, ReadStallMs, WriteStallMs, Seconds)
VALUES (s.InstanceId, s.[Hour], s.Volume, s.Reads, s.Writes, s.BytesRead, s.BytesWritten, s.ReadStallMs, s.WriteStallMs, s.Seconds);
""", new { from }, commandTimeout: 600).ConfigureAwait(false); """, new { from }, commandTimeout: 600).ConfigureAwait(false);
var raw = utc.AddDays(-options.RawDays); var raw = utc.AddDays(-options.RawDays);
@ -67,6 +85,7 @@ public sealed partial class InsightRepository
await DeleteInBatchesAsync(connection, "insight.QueryTexts", "LastSeenAt < @cutoff", raw).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.QueryTexts", "LastSeenAt < @cutoff", raw).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.MetricHourly", "[Hour] < @cutoff", hourly).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.MetricHourly", "[Hour] < @cutoff", hourly).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.WaitHourly", "[Hour] < @cutoff", hourly).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.WaitHourly", "[Hour] < @cutoff", hourly).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.DiskHourly", "[Hour] < @cutoff", hourly).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.BlockingEvents", "SampledAt < @cutoff", events).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.BlockingEvents", "SampledAt < @cutoff", events).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.Deadlocks", "OccurredAt < @cutoff", events).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.Deadlocks", "OccurredAt < @cutoff", events).ConfigureAwait(false);
await DeleteInBatchesAsync(connection, "insight.Findings", "ResolvedAt < @cutoff", resolved).ConfigureAwait(false); await DeleteInBatchesAsync(connection, "insight.Findings", "ResolvedAt < @cutoff", resolved).ConfigureAwait(false);

View file

@ -279,12 +279,18 @@ public sealed partial class InsightRepository(string connectionString)
private static Task<int> SaveFileIoAsync(SqlConnection connection, int instanceId, FileIoDelta delta) => private static Task<int> SaveFileIoAsync(SqlConnection connection, int instanceId, FileIoDelta delta) =>
connection.ExecuteAsync(""" connection.ExecuteAsync("""
INSERT insight.FileIoSamples (InstanceId, SampledAt, DatabaseName, FileId, FileType, Reads, ReadStallMs, Writes, WriteStallMs, BytesRead, BytesWritten) INSERT insight.FileIoSamples (InstanceId, SampledAt, DatabaseName, FileId, FileType, Reads, ReadStallMs, Writes, WriteStallMs, BytesRead, BytesWritten, Volume, IntervalSeconds)
SELECT @instanceId, @at, DatabaseName, FileId, FileType, Reads, ReadStallMs, Writes, WriteStallMs, BytesRead, BytesWritten SELECT @instanceId, @at, DatabaseName, FileId, FileType, Reads, ReadStallMs, Writes, WriteStallMs, BytesRead, BytesWritten, Volume, @interval
FROM OPENJSON(@json) WITH (DatabaseName nvarchar(128), FileId int, FileType varchar(10), Reads bigint, ReadStallMs bigint, FROM OPENJSON(@json) WITH (DatabaseName nvarchar(128), FileId int, FileType varchar(10), Reads bigint, ReadStallMs bigint,
Writes bigint, WriteStallMs bigint, BytesRead bigint, BytesWritten bigint) j Writes bigint, WriteStallMs bigint, BytesRead bigint, BytesWritten bigint, Volume nvarchar(260)) j
WHERE NOT EXISTS (SELECT 1 FROM insight.FileIoSamples f WHERE f.InstanceId = @instanceId AND f.SampledAt = @at AND f.DatabaseName = j.DatabaseName AND f.FileId = j.FileId); WHERE NOT EXISTS (SELECT 1 FROM insight.FileIoSamples f WHERE f.InstanceId = @instanceId AND f.SampledAt = @at AND f.DatabaseName = j.DatabaseName AND f.FileId = j.FileId);
""", new { instanceId, at = Utc(delta.To), json = JsonSerializer.Serialize(delta.Files, Json) }); """, new
{
instanceId,
at = Utc(delta.To),
interval = (int)Math.Round((delta.To - delta.From).TotalSeconds),
json = JsonSerializer.Serialize(delta.Files.Select(f => f with { Volume = DiskVolumes.Of(f) }), Json),
});
private static Task<int> InsertMetricsAsync(SqlConnection connection, int instanceId, IEnumerable<(string Metric, DateTimeOffset At, double Value)> metrics) => private static Task<int> InsertMetricsAsync(SqlConnection connection, int instanceId, IEnumerable<(string Metric, DateTimeOffset At, double Value)> metrics) =>
connection.ExecuteAsync(""" connection.ExecuteAsync("""

View file

@ -0,0 +1,21 @@
/* Disk speed per drive: which volume each file sample belongs to, and how long the sample interval was (for IOPS and
MB/s). NULL on samples taken before this version. */
ALTER TABLE insight.FileIoSamples ADD Volume nvarchar(260) NULL, IntervalSeconds int NULL;
GO
/* Hourly I/O per drive, kept as long as the other hourly aggregates (Retention:HourlyDays). Seconds = the measured
time within the hour, so rates stay right when samples are missing. */
CREATE TABLE insight.DiskHourly
(
InstanceId int NOT NULL,
[Hour] datetime2(0) NOT NULL,
Volume nvarchar(260) NOT NULL,
Reads bigint NOT NULL,
Writes bigint NOT NULL,
BytesRead bigint NOT NULL,
BytesWritten bigint NOT NULL,
ReadStallMs bigint NOT NULL,
WriteStallMs bigint NOT NULL,
Seconds int NOT NULL,
CONSTRAINT PK_DiskHourly PRIMARY KEY (InstanceId, [Hour], Volume)
);

View file

@ -14,65 +14,97 @@ internal static class FileIo
} }
/// <summary> /// <summary>
/// Slow reads from data files. /// Slow reads from the data files on a drive. Per drive rather than per file: when storage is slow, every file on it
/// is, and one finding per drive says so once. The slowest files are in the evidence.
/// </summary> /// </summary>
public sealed class DataFileReadLatencyRule : Rule public sealed class DataFileReadLatencyRule : Rule
{ {
public const int MinReads = 1000; public const int MinReads = 1000;
public override string Id => "SQLI-IO-001"; public override string Id => "SQLI-IO-001";
public override string Title => "Høj læselatens på datafiler"; public override string Title => "Høj læselatens på drev med datafiler";
public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews]; public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews];
public override RuleEvaluation Evaluate(RuleContext context) public override RuleEvaluation Evaluate(RuleContext context)
{ {
ArgumentNullException.ThrowIfNull(context); ArgumentNullException.ThrowIfNull(context);
var delta = FileIo.Delta(context); var delta = FileIo.Delta(context);
var seconds = (delta.To - delta.From).TotalSeconds;
return Result(delta.Files return Result(delta.Files
.Where(f => f.FileType == "ROWS" && f.Reads >= MinReads && f.AverageReadLatencyMs >= 20) .Where(f => f.FileType == "ROWS")
.Select(f => CreateFinding( .GroupBy(DiskVolumes.Of, StringComparer.OrdinalIgnoreCase)
f.AverageReadLatencyMs >= 50 ? Severity.High : Severity.Medium, .Select(g => (Files: g.ToList(), Disk: DiskActivity.ByVolume(g, seconds)[0]))
$"{f.DatabaseName}: {Format.Number(f.AverageReadLatencyMs!.Value, 1)} ms i gennemsnit pr. læsning", .Where(d => d.Disk.Reads >= MinReads && d.Disk.AverageReadMs >= 20)
"Læsninger fra datafilen tager i gennemsnit over 20 ms. På moderne storage (SSD/SAN) bør de ligge under 5-10 ms. " .Select(d => CreateFinding(
+ "Langsomme læsninger rammer direkte alle forespørgsler, der ikke finder data i buffer pool.", d.Disk.AverageReadMs >= 50 ? Severity.High : Severity.Medium,
$"{d.Disk.Volume} (datafiler): {Format.Number(d.Disk.AverageReadMs!.Value, 1)} ms i gennemsnit pr. læsning",
"Læsninger fra datafilerne på drevet tager i gennemsnit over 20 ms. På moderne storage (SSD/SAN) bør de ligge under "
+ "5-10 ms. Langsomme læsninger rammer direkte alle forespørgsler, der ikke finder data i hukommelsen (buffer pool), "
+ "så brugerne oplever langsomme skærmbilleder og rapporter.",
"Sammenlign med latensen i storage-laget (Windows' Avg. Disk sec/Read, SAN-statistik) for at se, om ventetiden " "Sammenlign med latensen i storage-laget (Windows' Avg. Disk sec/Read, SAN-statistik) for at se, om ventetiden "
+ "opstår i storage eller undervejs (HBA, multipath, VM-host). Undersøg også, om mængden af læsninger kan reduceres " + "opstår i storage eller undervejs (HBA, multipath, VM-host). Ser storage-laget hurtigt ud, er drevet overbelastet "
+ "med bedre indexes eller mere hukommelse.", + "af antallet af læsninger: se læse-IOPS og MB/s, og undersøg om de kan reduceres med bedre indexes eller mere hukommelse.",
[new Evidence("Fil", f.PhysicalName), new Evidence("Læsninger", Format.Number(f.Reads)), Evidence(d.Disk, d.Files, read: true, seconds),
new Evidence("Gns. latens", $"{Format.Number(f.AverageReadLatencyMs!.Value, 1)} ms"), subject: $"volume: {d.Disk.Volume} data",
new Evidence("Interval", $"{Format.Number((delta.To - delta.From).TotalMinutes)} min")],
subject: $"file: {f.DatabaseName}/{f.FileId}",
references: Docs.Learn("sys.dm_io_virtual_file_stats", "relational-databases/system-dynamic-management-views/sys-dm-io-virtual-file-stats-transact-sql")))); references: Docs.Learn("sys.dm_io_virtual_file_stats", "relational-databases/system-dynamic-management-views/sys-dm-io-virtual-file-stats-transact-sql"))));
} }
/// <summary>Drive totals plus the three slowest files, shared with the log write rule.</summary>
internal static Evidence[] Evidence(DiskActivity disk, IReadOnlyList<FileIoStat> files, bool read, double seconds)
{
var slowest = files
.Where(f => read ? f.Reads > 0 : f.Writes > 0)
.OrderByDescending(f => read ? f.AverageReadLatencyMs : f.AverageWriteLatencyMs)
.Take(3)
.Select(f => $"{f.DatabaseName} {Format.Number((read ? f.AverageReadLatencyMs : f.AverageWriteLatencyMs)!.Value, 1)} ms ({f.PhysicalName})");
return read
?
[
new("Drev", disk.Volume), new("Læsninger", Format.Number(disk.Reads)),
new("Gns. latens", $"{Format.Number(disk.AverageReadMs!.Value, 1)} ms"),
new("Læse-IOPS", Format.Number(disk.ReadIops ?? 0, 1)), new("Læst", $"{Format.Number(disk.ReadMbPerSecond ?? 0, 1)} MB/s"),
new("Langsomste filer", string.Join("; ", slowest)), new("Interval", $"{Format.Number(seconds / 60)} min"),
]
:
[
new("Drev", disk.Volume), new("Skrivninger", Format.Number(disk.Writes)),
new("Gns. latens", $"{Format.Number(disk.AverageWriteMs!.Value, 1)} ms"),
new("Skrive-IOPS", Format.Number(disk.WriteIops ?? 0, 1)), new("Skrevet", $"{Format.Number(disk.WriteMbPerSecond ?? 0, 1)} MB/s"),
new("Langsomste filer", string.Join("; ", slowest)), new("Interval", $"{Format.Number(seconds / 60)} min"),
];
}
} }
/// <summary> /// <summary>
/// Slow writes to transaction log files. /// Slow writes to the transaction log files on a drive (per drive, like <see cref="DataFileReadLatencyRule"/>).
/// </summary> /// </summary>
public sealed class LogFileWriteLatencyRule : Rule public sealed class LogFileWriteLatencyRule : Rule
{ {
public const int MinWrites = 1000; public const int MinWrites = 1000;
public override string Id => "SQLI-IO-002"; public override string Id => "SQLI-IO-002";
public override string Title => "Høj skrivelatens på logfiler"; public override string Title => "Høj skrivelatens på drev med logfiler";
public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews]; public override IReadOnlyCollection<Capability> RequiredCapabilities => [Capability.DynamicManagementViews];
public override RuleEvaluation Evaluate(RuleContext context) public override RuleEvaluation Evaluate(RuleContext context)
{ {
ArgumentNullException.ThrowIfNull(context); ArgumentNullException.ThrowIfNull(context);
var delta = FileIo.Delta(context); var delta = FileIo.Delta(context);
var seconds = (delta.To - delta.From).TotalSeconds;
return Result(delta.Files return Result(delta.Files
.Where(f => f.FileType == "LOG" && f.Writes >= MinWrites && f.AverageWriteLatencyMs >= 5) .Where(f => f.FileType == "LOG")
.Select(f => CreateFinding( .GroupBy(DiskVolumes.Of, StringComparer.OrdinalIgnoreCase)
f.AverageWriteLatencyMs >= 15 ? Severity.High : Severity.Medium, .Select(g => (Files: g.ToList(), Disk: DiskActivity.ByVolume(g, seconds)[0]))
$"{f.DatabaseName}: {Format.Number(f.AverageWriteLatencyMs!.Value, 1)} ms i gennemsnit pr. logskrivning", .Where(d => d.Disk.Writes >= MinWrites && d.Disk.AverageWriteMs >= 5)
.Select(d => CreateFinding(
d.Disk.AverageWriteMs >= 15 ? Severity.High : Severity.Medium,
$"{d.Disk.Volume} (logfiler): {Format.Number(d.Disk.AverageWriteMs!.Value, 1)} ms i gennemsnit pr. logskrivning",
"Hver commit venter på, at loggen er skrevet. Logskrivninger bør tage 1-2 ms; over 5 ms gør alle skrivende " "Hver commit venter på, at loggen er skrevet. Logskrivninger bør tage 1-2 ms; over 5 ms gør alle skrivende "
+ "transaktioner mærkbart langsommere.", + "transaktioner mærkbart langsommere, fx når brugerne bogfører eller gemmer.",
"Placér logfilen på storage med lav skrivelatens og write cache med batteribackup, adskilt fra datafilerne. " "Placér logfilerne på storage med lav skrivelatens og write cache med batteribackup, adskilt fra datafilerne. "
+ "Tjek, om andre filer eller VM'er deler samme volumen.", + "Tjek, om andre filer eller VM'er deler samme drev, og om skrive-MB/s nærmer sig det, drevet kan levere.",
[new Evidence("Fil", f.PhysicalName), new Evidence("Skrivninger", Format.Number(f.Writes)), DataFileReadLatencyRule.Evidence(d.Disk, d.Files, read: false, seconds),
new Evidence("Gns. latens", $"{Format.Number(f.AverageWriteLatencyMs!.Value, 1)} ms")], subject: $"volume: {d.Disk.Volume} log",
subject: $"file: {f.DatabaseName}/{f.FileId}",
references: Docs.Learn("sys.dm_io_virtual_file_stats", "relational-databases/system-dynamic-management-views/sys-dm-io-virtual-file-stats-transact-sql")))); references: Docs.Learn("sys.dm_io_virtual_file_stats", "relational-databases/system-dynamic-management-views/sys-dm-io-virtual-file-stats-transact-sql"))));
} }
} }

View file

@ -115,6 +115,11 @@
} }
</section> </section>
<section class="card">
<div class="card-head"><h2>Diskhastighed pr. drev</h2></div>
<DiskTable Disks="_analysis.Disks" ShowNormal="true" Empty="Ingen målinger pr. drev i tidsrummet (indsamles fra version 1.0.5)." />
</section>
<section class="card"> <section class="card">
<div class="card-head"><h2>I/O-latens pr. fil</h2></div> <div class="card-head"><h2>I/O-latens pr. fil</h2></div>
@if (_analysis.Files.Count == 0) @if (_analysis.Files.Count == 0)

View file

@ -75,6 +75,21 @@ else
} }
</section> </section>
<section class="card">
<div class="card-head"><h2>Diskhastighed pr. drev, seneste 24 timer</h2></div>
<DiskTable Disks="_disks" Empty="Endnu ingen målinger pr. drev (kræver to målinger efter opgradering til 1.0.5)." />
@if (_diskSeries.Count > 0)
{
<div class="grid2">
@foreach (var s in _diskSeries.Take(6))
{
<div class="panel"><Sparkline Points="s.MbPerSecond" Label="@($"{s.Volume} MB/s")" Decimals="1" /></div>
<div class="panel"><Sparkline Points="s.LatencyMs" Label="@($"{s.Volume} gns. latens (ms)")" Decimals="1" /></div>
}
</div>
}
</section>
<section class="card"> <section class="card">
<div class="card-head"><h2>I/O-latens pr. fil, seneste 24 timer</h2></div> <div class="card-head"><h2>I/O-latens pr. fil, seneste 24 timer</h2></div>
@if (_files.Count == 0) @if (_files.Count == 0)
@ -223,6 +238,8 @@ else
private IReadOnlyList<MetricPoint> _cpuSql = [], _cpuOther = [], _batches = [], _ple = []; private IReadOnlyList<MetricPoint> _cpuSql = [], _cpuOther = [], _batches = [], _ple = [];
private IReadOnlyList<WaitTotal> _waits = []; private IReadOnlyList<WaitTotal> _waits = [];
private IReadOnlyList<FileLatency> _files = []; private IReadOnlyList<FileLatency> _files = [];
private IReadOnlyList<SQLInsight.Reports.DiskComparison> _disks = [];
private IReadOnlyList<DiskSeries> _diskSeries = [];
private IReadOnlyList<StoredBlocking> _blocking = []; private IReadOnlyList<StoredBlocking> _blocking = [];
private IReadOnlyList<StoredDeadlock> _deadlocks = []; private IReadOnlyList<StoredDeadlock> _deadlocks = [];
@ -247,6 +264,8 @@ else
_ple = await Repository.GetMetricAsync(id, Metrics.PageLifeExpectancy, day); _ple = await Repository.GetMetricAsync(id, Metrics.PageLifeExpectancy, day);
_waits = await Repository.GetTopWaitsAsync(id, day, 12); _waits = await Repository.GetTopWaitsAsync(id, day, 12);
_files = await Repository.GetFileLatencyAsync(id, day); _files = await Repository.GetFileLatencyAsync(id, day);
_disks = (await Repository.GetDiskActivityAsync(id, day, DateTimeOffset.UtcNow)).Select(d => new SQLInsight.Reports.DiskComparison(d, null)).ToList();
_diskSeries = await Repository.GetDiskSeriesAsync(id, day);
_blocking = await Repository.GetBlockingAsync(id, day); _blocking = await Repository.GetBlockingAsync(id, day);
_deadlocks = await Repository.GetDeadlocksAsync(id, DateTimeOffset.UtcNow.AddDays(-7)); _deadlocks = await Repository.GetDeadlocksAsync(id, DateTimeOffset.UtcNow.AddDays(-7));
} }

View file

@ -0,0 +1,48 @@
@* Disk speed per drive: IOPS, MB/s and average latency, optionally against the normal for the same hours. *@
@if (Disks.Count == 0)
{
<p class="muted">@Empty</p>
}
else
{
<table>
<thead>
<tr>
<th>Drev</th><th class="num">Filer</th><th class="num">Læs IOPS</th><th class="num">Skriv IOPS</th>
<th class="num">Læs MB/s</th><th class="num">Skriv MB/s</th><th class="num">Gns. læs</th><th class="num">Gns. skriv</th>
</tr>
</thead>
<tbody>
@foreach (var d in Disks)
{
var w = d.Window;
<tr>
<td class="mono">@w.Volume</td>
<td class="num">@Files(w)</td>
<td class="num">@Display.Number(w.ReadIops ?? 0, 1)</td>
<td class="num">@Display.Number(w.WriteIops ?? 0, 1)</td>
<td class="num">@Display.Number(w.ReadMbPerSecond ?? 0, 1)</td>
<td class="num">@Display.Number(w.WriteMbPerSecond ?? 0, 1)</td>
<td class="num @(w.AverageReadMs >= 20 || Worse(w.AverageReadMs, d.Normal?.AverageReadMs) ? "bad" : "")">@Ms(w.AverageReadMs)@Normal(d.Normal?.AverageReadMs)</td>
<td class="num @(w.AverageWriteMs >= (w.DataFiles == 0 && w.LogFiles > 0 ? 5 : 20) || Worse(w.AverageWriteMs, d.Normal?.AverageWriteMs) ? "bad" : "")">@Ms(w.AverageWriteMs)@Normal(d.Normal?.AverageWriteMs)</td>
</tr>
}
</tbody>
</table>
<p class="muted">Latens er gennemsnitstiden pr. læsning/skrivning set fra SQL Server (sys.dm_io_virtual_file_stats). Datafiler bør ligge under 10-20 ms, logfiler under 5 ms.@(ShowNormal ? " I parentes: det normale for samme klokkeslæt de foregående uger." : "")</p>
}
@code {
[Parameter, EditorRequired] public IReadOnlyList<SQLInsight.Reports.DiskComparison> Disks { get; set; } = [];
[Parameter] public string Empty { get; set; } = "Ingen målinger pr. drev endnu.";
[Parameter] public bool ShowNormal { get; set; }
private static string Ms(double? value) => value is { } v ? $"{Display.Number(v, 1)} ms" : "–";
private string Normal(double? value) => ShowNormal && value is { } v ? $" ({Display.Number(v, 1)})" : "";
private static string Files(SQLInsight.Core.Data.DiskActivity d) =>
d.DataFiles + d.LogFiles == 0 ? "–" : $"{d.DataFiles} data · {d.LogFiles} log";
private static bool Worse(double? now, double? normal) => now is { } n && normal is > 0 and var m && n >= m * 2 && n - m >= 2;
}

View file

@ -0,0 +1,64 @@
using SQLInsight.Core.Data;
namespace SQLInsight.Core.Tests;
public class DiskTests
{
[Theory]
[InlineData(@"D:\SQLData\Sales.mdf", @"D:\")]
[InlineData(@"l:\Logs\Sales_log.ldf", @"L:\")]
[InlineData(@"\\nas01\sql\Sales.mdf", @"\\nas01\sql\")]
[InlineData("/var/opt/mssql/data/master.mdf", "/")]
public void Drive_is_derived_from_the_file_path(string physicalName, string volume) =>
Assert.Equal(volume, DiskVolumes.FromPhysicalName(physicalName));
[Fact]
public void Reported_volume_wins_over_the_path()
{
var mountPoint = File("Sales", 1, "ROWS", @"D:\Mounts\Data1\Sales.mdf") with { Volume = @"D:\Mounts\Data1\" };
Assert.Equal(@"D:\Mounts\Data1\", DiskVolumes.Of(mountPoint));
Assert.Equal(@"D:\", DiskVolumes.Of(mountPoint with { Volume = null }));
}
[Fact]
public void Files_are_summed_per_drive_with_rates_and_weighted_latency()
{
var files = new[]
{
File("Sales", 1, "ROWS", @"D:\Sales.mdf", reads: 3000, bytesRead: 300 * 1048576L, readStall: 30_000),
File("Stock", 1, "ROWS", @"D:\Stock.mdf", reads: 1000, bytesRead: 0, readStall: 50_000),
File("Sales", 2, "LOG", @"L:\Sales_log.ldf", writes: 6000, bytesWritten: 60 * 1048576L, writeStall: 12_000),
};
var disks = DiskActivity.ByVolume(files, seconds: 300);
Assert.Equal([@"D:\", @"L:\"], disks.Select(d => d.Volume));
var data = disks[0];
Assert.Equal(2, data.DataFiles);
Assert.Equal(0, data.LogFiles);
Assert.Equal(4000 / 300.0, data.ReadIops);
Assert.Equal(1.0, data.ReadMbPerSecond);
Assert.Equal(20, data.AverageReadMs); // 80 s over 4000 reads, not the average of 10 and 50 ms
Assert.Null(data.AverageWriteMs);
var log = disks[1];
Assert.Equal(20, log.WriteIops);
Assert.Equal(0.2, log.WriteMbPerSecond!.Value, 6);
Assert.Equal(2, log.AverageWriteMs);
}
private static FileIoStat File(string db, int id, string type, string path, long reads = 0, long bytesRead = 0, long readStall = 0,
long writes = 0, long bytesWritten = 0, long writeStall = 0) => new()
{
DatabaseName = db,
FileId = id,
FileType = type,
PhysicalName = path,
Reads = reads,
BytesRead = bytesRead,
ReadStallMs = readStall,
Writes = writes,
BytesWritten = bytesWritten,
WriteStallMs = writeStall,
};
}

View file

@ -103,6 +103,8 @@ public class CollectorTests(SqlServerFixture sql)
var pc2 = await session.CollectAsync(new PerformanceCounterCollector(), pc1); var pc2 = await session.CollectAsync(new PerformanceCounterCollector(), pc1);
Assert.Contains(io1.Snapshot.Files, f => f.DatabaseName == "master" && f.FileType == "LOG"); Assert.Contains(io1.Snapshot.Files, f => f.DatabaseName == "master" && f.FileType == "LOG");
// SQL Server 2022 has sys.dm_os_volume_stats: every file knows its drive (a mount point on Linux).
Assert.All(io1.Snapshot.Files, f => Assert.False(string.IsNullOrWhiteSpace(f.Volume)));
Assert.NotNull(io2.Delta); Assert.NotNull(io2.Delta);
Assert.NotNull(pc1.Snapshot.PageLifeExpectancySeconds); Assert.NotNull(pc1.Snapshot.PageLifeExpectancySeconds);
Assert.NotNull(pc1.Snapshot.MemoryGrantsPending); Assert.NotNull(pc1.Snapshot.MemoryGrantsPending);

View file

@ -146,6 +146,7 @@ public class TimeWindowAnalysisTests(SqlServerFixture sql) : IAsyncLifetime
var html = await client.GetStringAsync($"/instanser/{name}/analyse?{Window(from, to)}"); var html = await client.GetStringAsync($"/instanser/{name}/analyse?{Window(from, to)}");
Assert.Contains("SELECT * FROM SALESLINE", html, StringComparison.Ordinal); Assert.Contains("SELECT * FROM SALESLINE", html, StringComparison.Ordinal);
Assert.Contains("LCK_M_X", html, StringComparison.Ordinal); Assert.Contains("LCK_M_X", html, StringComparison.Ordinal);
Assert.Contains("Diskhastighed pr. drev", html, StringComparison.Ordinal);
Assert.Contains("class=\"unusual\"", html, StringComparison.Ordinal); Assert.Contains("class=\"unusual\"", html, StringComparison.Ordinal);
Assert.Contains("value=\"" + TimeWindow.ToInput(from, Copenhagen) + "\"", html, StringComparison.Ordinal); Assert.Contains("value=\"" + TimeWindow.ToInput(from, Copenhagen) + "\"", html, StringComparison.Ordinal);
@ -163,6 +164,51 @@ public class TimeWindowAnalysisTests(SqlServerFixture sql) : IAsyncLifetime
Assert.StartsWith("%PDF", System.Text.Encoding.ASCII.GetString(await pdf.Content.ReadAsByteArrayAsync())[..4], StringComparison.Ordinal); Assert.StartsWith("%PDF", System.Text.Encoding.ASCII.GetString(await pdf.Content.ReadAsByteArrayAsync())[..4], StringComparison.Ordinal);
} }
[Fact]
public async Task Disk_speed_is_summed_per_drive_and_kept_hourly()
{
var to = FloorHour(DateTimeOffset.UtcNow).AddHours(-1);
var from = to.AddHours(-1);
for (var at = from.AddMinutes(5); at <= to; at = at.AddMinutes(5))
{
await _repository.SaveCollectedAsync(_instanceId, "file-io", new CumulativeSample<FileIoSnapshot, FileIoDelta>(
new FileIoSnapshot { CapturedAt = at, SqlServerStartTime = DateTime.UtcNow.AddDays(-1), Files = [] },
new FileIoDelta
{
From = at.AddMinutes(-5),
To = at,
Files =
[
IoFile("AX30", 1, "ROWS", @"D:\AX30.mdf", reads: 3000, readStall: 90_000, bytesRead: 300 * 1048576L),
IoFile("AX30", 2, "LOG", @"L:\AX30_log.ldf", writes: 3000, writeStall: 3_000, bytesWritten: 30 * 1048576L),
],
}));
}
await _repository.RunMaintenanceAsync(new RetentionOptions(), DateTimeOffset.UtcNow);
var raw = await _repository.GetDiskActivityAsync(_instanceId, from, to);
var hourly = await _repository.GetDiskActivityHourlyAsync(_instanceId, from, to);
var analysis = await TimeWindowAnalysis.LoadAsync(_repository, (await _repository.GetInstanceAsync(@"AX01\AX30"))!, from, to, DateTimeOffset.UtcNow);
var data = raw.Single(d => d.Volume == @"D:\");
Assert.Equal(1, data.DataFiles);
Assert.Equal(30, data.AverageReadMs);
Assert.Equal(10, data.ReadIops); // 3000 reads per 300 s
Assert.Equal(1, data.ReadMbPerSecond); // 300 MB per 300 s
Assert.Equal(1, raw.Single(d => d.Volume == @"L:\").AverageWriteMs);
// The sample on the hour belongs to the next hour (as for waits): 11 samples of 300 s in this one.
Assert.Equal(11 * 300, hourly.Single(d => d.Volume == @"D:\").Seconds);
Assert.True(analysis.Disks.Single(d => d.Window.Volume == @"D:\").IsSlow);
Assert.False(analysis.Disks.Single(d => d.Window.Volume == @"L:\").IsUnusual);
}
private static FileIoStat IoFile(string db, int id, string type, string path, long reads = 0, long readStall = 0, long bytesRead = 0,
long writes = 0, long writeStall = 0, long bytesWritten = 0) => new()
{
DatabaseName = db, FileId = id, FileType = type, PhysicalName = path, Reads = reads, ReadStallMs = readStall, BytesRead = bytesRead,
Writes = writes, WriteStallMs = writeStall, BytesWritten = bytesWritten,
};
/// <summary>A busy hour: high CPU, lock waits and one heavy statement, against quiet same hours four weeks back.</summary> /// <summary>A busy hour: high CPU, lock waits and one heavy statement, against quiet same hours four weeks back.</summary>
private async Task SeedWindowAsync(DateTimeOffset from, DateTimeOffset to) private async Task SeedWindowAsync(DateTimeOffset from, DateTimeOffset to)
{ {

View file

@ -96,6 +96,22 @@ public class PerformanceRuleTests
} }
} }
[Fact]
public void Slow_data_files_on_one_drive_give_one_finding_for_the_drive()
{
var sales = File("ROWS", reads: 3000, readStallMs: 90_000); // 30 ms
var stock = File("ROWS", reads: 2000, readStallMs: 40_000) with { DatabaseName = "Stock", FileId = 1, PhysicalName = @"D:\Stock.mdf" };
var fast = File("ROWS", reads: 5000, readStallMs: 5_000) with { DatabaseName = "Fast", PhysicalName = @"E:\Fast.mdf" };
var result = Run(new DataFileReadLatencyRule(), With(Io(sales, stock, fast)));
var finding = Assert.Single(result.Findings);
Assert.Equal(@"volume: D:\ data", finding.Subject);
Assert.Contains(@"D:\ (datafiler): 26,0 ms", finding.Title, StringComparison.Ordinal); // 130 s over 5000 reads
Assert.Contains(finding.Evidence, e => e.Name == "Langsomste filer" && e.Value.StartsWith("Sales 30,0 ms", StringComparison.Ordinal));
Assert.Contains(finding.Evidence, e => e.Name == "Læse-IOPS");
}
private static CpuUtilization Cpu(int sql, int idle, int minutes = 15, string platform = "Windows") => new() private static CpuUtilization Cpu(int sql, int idle, int minutes = 15, string platform = "Windows") => new()
{ {
CapturedAt = Now, CapturedAt = Now,