Fase 4: experience across customers
Some checks are pending
CI / Build og unit tests (push) Waiting to run
CI / Integration (mcr.microsoft.com/mssql/server:2017-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2019-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2022-latest) (push) Blocked by required conditions
CI / Integration (mcr.microsoft.com/mssql/server:2025-latest) (push) Blocked by required conditions

The hub records when an instance changes build and works out which
findings went away within three days of an update and which were still
there a week later (version rules excluded). Staff see every effect in
the portal; effects seen at three or more customers, resolved more
often than not, are served to licensed agents as anonymous counts
(rule, major version, build, CU, customers, instances). The agent
fetches them every six hours and the web UI shows them on matching
findings: 'at 3 customers this went away after updating to CU21'.
Data processing documentation covers the aggregation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3EExwndy8Z22EK2Qpt6BK
This commit is contained in:
Claude 2026-09-29 10:33:37 +00:00
parent 7f640e74e5
commit 4269f6f9e8
No known key found for this signature in database
25 changed files with 1207 additions and 14 deletions

View file

@ -4,7 +4,7 @@
SQLInsight Pro er et overvågningsværktøj til Microsoft SQL Server, skrevet i C# på .NET. Det kører hos kunden og forlader ikke netværket, medmindre kunden selv vælger det. Værktøjet nøjes ikke med at vise tal. Det forklarer, hvad der er galt, hvorfor det er et problem, og hvordan det løses. Rådene har samme niveau som dem, man får fra en erfaren DBA.
> **Status:** Fase 1–3 er implementeret: agent, 61 regler (herunder Query Store, plananalyse, indexes, Always On og anomalier), notifikationer, lokalt repository og web-UI. Dertil kommer licenser, signerede opdateringer og SQLInsight Hub med portal til betalende kunder. Se [installation](docs/installation.md), [opdateringer](docs/opdatering.md), [databehandling](docs/databehandling.md), [Roadmap](#roadmap) og [CONTRIBUTING.md](CONTRIBUTING.md).
> **Status:** Fase 1–4 er implementeret: agent, 61 indbyggede regler plus community-regler, notifikationer, lokalt repository og web-UI, licenser, signerede opdateringer, SQLInsight Hub med portal, erfaring på tværs af kunder, PDF-rapporter og integration med Prometheus, Grafana og SCOM. Se [installation](docs/installation.md), [opdateringer](docs/opdatering.md), [integration](docs/integration.md), [rapporter](docs/rapporter.md), [egne regler](docs/regler.md), [databehandling](docs/databehandling.md), [Roadmap](#roadmap) og [CONTRIBUTING.md](CONTRIBUTING.md).
---
@ -203,7 +203,7 @@ Hvert **finding** indeholder:
- [ ] Log over medarbejderes adgang og tofaktorlogin i portalen
### Fase 4: Videre
- [ ] Anbefalinger på tværs af kunder: "denne CU har løst problemet hos andre"
- [x] Anbefalinger på tværs af kunder: Hubben ser, hvilke fund der forsvinder efter en opdatering, og agenterne viser "hos 4 kunder forsvandt dette efter CU21" på fundet. Deles kun samlet og anonymt fra mindst 3 kunder. Se [docs/hub.md](docs/hub.md#erfaring-på-tværs-af-kunder)
- [x] Rapporter (PDF) til ledelse og compliance: nøgletal, sammenfatning i almindeligt sprog, kontroller, instanser og alvorlige fund. Kan hentes i webbrugerfladen og sendes månedligt på mail. Se [docs/rapporter.md](docs/rapporter.md)
- [x] Mulighed for, at community selv bidrager med regler: regler som SQL i JSON (community via pull request og signeret regelpakke, lokale regler hos kunden), med validering og `rules test` mod en instans. Se [docs/regler.md](docs/regler.md)
- [x] API til integration med eksisterende monitorering: JSON-API og Prometheus-endpoint med API-tokens, Grafana-dashboard, og Windows' hændelseslog med faste event-id'er til SCOM. Se [docs/integration.md](docs/integration.md)

View file

@ -39,15 +39,26 @@ Data i hubben bruges kun til:
- at gøre SEOSofts support opmærksom på kritiske fund hos kunden
- at se, hvilke problemer der går igen på tværs af kunder, så regler, vejledninger og produkt kan forbedres (aggregeret)
- at give alle kunder **samlet, anonym erfaring** tilbage: "hos 4 kunder forsvandt dette fund efter opdatering til CU21" (se nedenfor)
Data sælges ikke og videregives ikke.
### Erfaring på tværs af kunder
Hubben registrerer, når en instans skifter build (fra og til version, CU-niveau og tidspunkt). Den ser derefter, hvilke fund der forsvandt kort efter, og hvilke der blev ved. Til agenterne deles kun:
- regel-id, SQL Server-hovedversion, build og CU-niveau
- antal kunder og antal instanser, hvor fundet forsvandt eller blev ved
Der deles aldrig kundenavne, instans- eller servernavne, databasenavne, målinger eller fundets tekster. En sammenhæng deles kun, når fundet er forsvundet hos mindst 3 forskellige kunder (`SQLInsight:Hub:Insights:MinCustomers`). Oplysningerne kan ikke føres tilbage til en bestemt kunde og er ikke personoplysninger. Det skal alligevel fremgå af databehandleraftalen, at kundens data indgår i denne statistik.
## Opbevaring
| Data | Slettes |
|---|---|
| Aktive fund | Når de er løst, og derefter efter 365 dage (medmindre de hører til en åben supportsag) |
| Log over modtagelser (tidspunkt, antal fund; ikke indholdet) | 90 dage |
| Instanser, der ikke har rapporteret | 180 dage, med deres fund |
| Instanser, der ikke har rapporteret | 180 dage, med deres fund og versionshistorik |
| Alt om en kunde | Når aftalen ophører, eller kunden beder om det (`delete-customer`) |
Fristerne kan ændres i hubbens konfiguration (se [hub.md](hub.md)). I kundens eget repository styres opbevaringen af `SQLInsight:Retention`.

View file

@ -51,6 +51,16 @@ dotnet publish src/SQLInsight.Hub -c Release -o /opt/sqlinsight/hub
Hubben kan køre som Windows-tjeneste eller systemd-enhed (`Type=notify`) på samme måde som agenten. Databaseskemaet oprettes og opgraderes ved start.
## Erfaring på tværs af kunder
Hubben registrerer, når en instans skifter build, og beregner, hvilke fund der forsvinder inden for 3 dage efter en opdatering, og hvilke der stadig findes efter 7 dage. Under **Opgraderinger** i portalen ses alle sammenhænge, også dem med få kunder. Sammenhænge med mindst 3 kunder, hvor fundet oftest forsvandt, hentes af agenterne (`GET /api/v1/insights`, med licens) og vises på fundet i kundens webbrugerflade.
```json
"Insights": { "MinCustomers": 3, "ResolvedWithin": "3.00:00:00", "PersistedAfter": "7.00:00:00" }
```
Regler om selve versionen (SQLI-VER-*) er udeladt, fordi de altid forsvinder ved en opdatering.
## Licenser
```bash

View file

@ -103,9 +103,51 @@ public sealed partial class HubSyncService(
}
}
if (_insightsFetchedAt is null || time.GetUtcNow() - _insightsFetchedAt >= InsightsInterval)
{
await FetchInsightsAsync(http, settings, cancellationToken).ConfigureAwait(false);
}
return sent;
}
/// <summary>How often aggregated experience from other customers is fetched.</summary>
public static readonly TimeSpan InsightsInterval = TimeSpan.FromHours(6);
private DateTimeOffset? _insightsFetchedAt;
/// <summary>Fetches "fixed elsewhere after updating to …" insights. Failing is harmless: the old ones stay.</summary>
public async Task<bool> FetchInsightsAsync(HttpClient http, HubOptions settings, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(http);
ArgumentNullException.ThrowIfNull(settings);
try
{
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(settings.Url!, "api/v1/insights"));
request.Headers.Authorization = new AuthenticationHeaderValue(AuthorizationScheme, settings.License);
using var response = await http.SendAsync(request, cancellationToken).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
return false;
}
var json = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
if (System.Text.Json.JsonSerializer.Deserialize<Contracts.HubInsights>(json, System.Text.Json.JsonSerializerOptions.Web) is null)
{
return false;
}
await repository.SaveHubInsightsAsync(json, time.GetUtcNow(), cancellationToken).ConfigureAwait(false);
_insightsFetchedAt = time.GetUtcNow();
return true;
}
catch (Exception ex) when (ex is HttpRequestException or System.Text.Json.JsonException or TaskCanceledException && !cancellationToken.IsCancellationRequested)
{
LogInsightsFailed(logger, ex.Message);
return false;
}
}
private LicenseStatus CheckLicense(HubOptions settings) =>
new LicenseVerifier(TrustedKeys.Embedded().With(settings.TrustedKeys), time).Check(settings.License);
@ -121,6 +163,9 @@ public sealed partial class HubSyncService(
[LoggerMessage(Level = LogLevel.Warning, Message = "Hub unreachable for {Instance}; will retry")]
private static partial void LogUnreachable(ILogger logger, Exception exception, string instance);
[LoggerMessage(Level = LogLevel.Information, Message = "Could not fetch insights from the hub: {Message}")]
private static partial void LogInsightsFailed(ILogger logger, string message);
[LoggerMessage(Level = LogLevel.Error, Message = "Hub sync round failed")]
private static partial void LogRoundFailed(ILogger logger, Exception exception);
}

View file

@ -0,0 +1,37 @@
namespace SQLInsight.Contracts;
/// <summary>
/// Aggregated experience across SQLInsight Hub customers, sent back to agents. Contains no customer, instance or
/// server names: only counts, and only when at least <see cref="MinCustomers"/> different customers are behind them.
/// </summary>
public sealed record HubInsights(DateTimeOffset GeneratedAt, int MinCustomers, IReadOnlyList<UpgradeInsight> Upgrades);
/// <summary>
/// After instances of SQL Server major version <see cref="Major"/> were updated to <see cref="ToBuild"/> or newer, findings
/// of <see cref="RuleId"/> they had went away within a few days on <see cref="Resolved"/> instances and stayed on
/// <see cref="Persisted"/>. Correlation, not proof of cause.
/// </summary>
public sealed record UpgradeInsight(string RuleId, int Major, string ToBuild, string? ToLevel, int Customers, int Resolved, int Persisted)
{
public double ResolvedShare => Resolved + Persisted == 0 ? 0 : (double)Resolved / (Resolved + Persisted);
}
public static class HubInsightMatcher
{
/// <summary>
/// Insights relevant to a finding: the same rule, the same major version, and a build newer than the instance's.
/// The one backed by most customers first.
/// </summary>
public static IReadOnlyList<UpgradeInsight> For(HubInsights? insights, string ruleId, string? productVersion)
{
if (insights is null || productVersion is null || !Version.TryParse(productVersion, out var current))
{
return [];
}
return insights.Upgrades
.Where(i => i.RuleId == ruleId && i.Major == current.Major && Version.TryParse(i.ToBuild, out var build) && build > current)
.OrderByDescending(i => i.Customers).ThenBy(i => Version.Parse(i.ToBuild))
.ToList();
}
}

View file

@ -19,35 +19,49 @@ public static class FindingsEndpoint
public static void Map(IEndpointRouteBuilder app) =>
app.MapPost("/api/v1/findings", HandleAsync).DisableAntiforgery().AllowAnonymous();
internal static async Task<IResult> HandleAsync(
HttpRequest request, HubDbContext db, IOptions<HubServerOptions> options, TimeProvider time, CancellationToken cancellationToken)
/// <summary>The license in the Authorization header, verified and not revoked; otherwise the response to send.</summary>
internal static async Task<(License? License, IResult? Rejected)> AuthorizeAsync(
HttpRequest request, HubDbContext db, HubServerOptions options, TimeProvider time, CancellationToken cancellationToken)
{
// 1. License: the hub is the authority; the agent's own view is never trusted.
var header = request.Headers.Authorization.ToString();
if (!header.StartsWith(AuthorizationScheme + " ", StringComparison.Ordinal))
{
return Problem(401, "Licens mangler", $"Authorization: {AuthorizationScheme} <licens> kræves.");
return (null, Problem(401, "Licens mangler", $"Authorization: {AuthorizationScheme} <licens> kræves."));
}
var keys = TrustedKeys.Embedded().With(options.Value.TrustedKeys);
var keys = TrustedKeys.Embedded().With(options.TrustedKeys);
var status = new LicenseVerifier(keys, time).Check(header[(AuthorizationScheme.Length + 1)..]);
if (status.State is LicenseState.Invalid or LicenseState.None)
{
return Problem(401, "Ugyldig licens", status.Message);
return (null, Problem(401, "Ugyldig licens", status.Message));
}
if (!status.AllowsFindingsSync)
{
return Problem(403, "Licensen giver ikke Hub-synkronisering", status.Message);
return (null, Problem(403, "Licensen giver ikke Hub-synkronisering", status.Message));
}
var license = status.License!;
var record = await db.Licenses.FindAsync([license.LicenseId], cancellationToken).ConfigureAwait(false);
var record = await db.Licenses.FindAsync([status.License!.LicenseId], cancellationToken).ConfigureAwait(false);
if (record?.RevokedAt is not null)
{
return Problem(403, "Licensen er spærret", record.RevokedReason ?? "Kontakt SQLInsight-support.");
return (null, Problem(403, "Licensen er spærret", record.RevokedReason ?? "Kontakt SQLInsight-support."));
}
return (status.License, null);
}
internal static async Task<IResult> HandleAsync(
HttpRequest request, HubDbContext db, IOptions<HubServerOptions> options, TimeProvider time, CancellationToken cancellationToken)
{
// 1. License: the hub is the authority; the agent's own view is never trusted.
var (license, rejected) = await AuthorizeAsync(request, db, options.Value, time, cancellationToken).ConfigureAwait(false);
if (license is null)
{
return rejected!;
}
LicenseRecord? record;
// 2. Body.
if (request.ContentLength > options.Value.MaxRequestBytes)
{
@ -135,6 +149,15 @@ public static class FindingsEndpoint
var active = 0;
if (apply)
{
if (instance.ProductVersion is { } previous && previous != submission.Instance.ProductVersion)
{
db.VersionChanges.Add(new InstanceVersionChange
{
InstanceId = instance.Id, FromVersion = previous, ToVersion = submission.Instance.ProductVersion,
ToLevel = submission.Instance.ProductUpdateLevel ?? submission.Instance.ProductLevel, ChangedAt = collectedAt,
});
}
instance.Release = submission.Instance.Release;
instance.ProductVersion = submission.Instance.ProductVersion;
instance.ProductLevel = submission.Instance.ProductLevel;

View file

@ -118,3 +118,15 @@ public sealed class StaffUser
public DateTime CreatedAt { get; set; }
public DateTime? LastLoginAt { get; set; }
}
/// <summary>An instance moved to another build (a CU, a GDR or an upgrade). The basis for "fixed after upgrading" insights.</summary>
public sealed class InstanceVersionChange
{
public long Id { get; set; }
public int InstanceId { get; set; }
public Instance? Instance { get; set; }
public required string FromVersion { get; set; }
public required string ToVersion { get; set; }
public string? ToLevel { get; set; }
public DateTime ChangedAt { get; set; }
}

View file

@ -12,6 +12,7 @@ public sealed class HubDbContext(DbContextOptions<HubDbContext> options) : DbCon
public DbSet<RuleState> RuleStates => Set<RuleState>();
public DbSet<SupportCase> SupportCases => Set<SupportCase>();
public DbSet<StaffUser> StaffUsers => Set<StaffUser>();
public DbSet<InstanceVersionChange> VersionChanges => Set<InstanceVersionChange>();
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
@ -60,6 +61,12 @@ public sealed class HubDbContext(DbContextOptions<HubDbContext> options) : DbCon
e.HasOne(c => c.Customer).WithMany(c => c.SupportCases).HasForeignKey(c => c.CustomerId);
});
modelBuilder.Entity<InstanceVersionChange>(e =>
{
e.HasOne(v => v.Instance).WithMany().HasForeignKey(v => v.InstanceId);
e.HasIndex(v => new { v.InstanceId, v.ChangedAt });
});
modelBuilder.Entity<StaffUser>(e =>
{
e.HasIndex(u => u.UserName).IsUnique();

View file

@ -0,0 +1,540 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
using SQLInsight.Hub.Data;
#nullable disable
namespace SQLInsight.Hub.Data.Migrations
{
[DbContext(typeof(HubDbContext))]
[Migration("20260929102409_VersionChanges")]
partial class VersionChanges
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasAnnotation("ProductVersion", "10.0.12")
.HasAnnotation("Relational:MaxIdentifierLength", 63);
NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder);
modelBuilder.Entity("SQLInsight.Hub.Data.Customer", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<int>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("text")
.HasColumnName("name");
b.HasKey("Id")
.HasName("pk_customers");
b.HasIndex("Name")
.IsUnique()
.HasDatabaseName("ix_customers_name");
b.ToTable("customers", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.HubFinding", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<string>("EvidenceJson")
.IsRequired()
.HasColumnType("jsonb")
.HasColumnName("evidence_json");
b.Property<string>("Explanation")
.IsRequired()
.HasColumnType("text")
.HasColumnName("explanation");
b.Property<DateTime>("FirstSeenAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("first_seen_at");
b.Property<int>("InstanceId")
.HasColumnType("integer")
.HasColumnName("instance_id");
b.Property<DateTime>("LastSeenAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("last_seen_at");
b.Property<string>("Recommendation")
.IsRequired()
.HasColumnType("text")
.HasColumnName("recommendation");
b.Property<DateTime?>("ResolvedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("resolved_at");
b.Property<string>("RuleId")
.IsRequired()
.HasColumnType("text")
.HasColumnName("rule_id");
b.Property<int>("Severity")
.HasColumnType("integer")
.HasColumnName("severity");
b.Property<string>("Subject")
.IsRequired()
.HasColumnType("text")
.HasColumnName("subject");
b.Property<int?>("SupportCaseId")
.HasColumnType("integer")
.HasColumnName("support_case_id");
b.Property<string>("Title")
.IsRequired()
.HasColumnType("text")
.HasColumnName("title");
b.HasKey("Id")
.HasName("pk_findings");
b.HasIndex("SupportCaseId")
.HasDatabaseName("ix_findings_support_case_id");
b.HasIndex("Severity", "ResolvedAt")
.HasDatabaseName("ix_findings_severity_resolved_at");
b.HasIndex("InstanceId", "RuleId", "Subject")
.IsUnique()
.HasDatabaseName("ix_findings_instance_id_rule_id_subject")
.HasFilter("resolved_at IS NULL");
b.ToTable("findings", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.Instance", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<int>("Id"));
b.Property<string>("AgentVersion")
.HasColumnType("text")
.HasColumnName("agent_version");
b.Property<int>("CustomerId")
.HasColumnType("integer")
.HasColumnName("customer_id");
b.Property<string>("Edition")
.HasColumnType("text")
.HasColumnName("edition");
b.Property<string>("InstanceKey")
.IsRequired()
.HasColumnType("text")
.HasColumnName("instance_key");
b.Property<DateTime?>("LastCollectedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("last_collected_at");
b.Property<DateTime?>("LastSubmissionAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("last_submission_at");
b.Property<string>("ProductLevel")
.HasColumnType("text")
.HasColumnName("product_level");
b.Property<string>("ProductUpdateLevel")
.HasColumnType("text")
.HasColumnName("product_update_level");
b.Property<string>("ProductVersion")
.HasColumnType("text")
.HasColumnName("product_version");
b.Property<string>("Release")
.HasColumnType("text")
.HasColumnName("release");
b.Property<string>("ServerName")
.HasColumnType("text")
.HasColumnName("server_name");
b.HasKey("Id")
.HasName("pk_instances");
b.HasIndex("CustomerId", "InstanceKey")
.IsUnique()
.HasDatabaseName("ix_instances_customer_id_instance_key");
b.ToTable("instances", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.InstanceVersionChange", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("ChangedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("changed_at");
b.Property<string>("FromVersion")
.IsRequired()
.HasColumnType("text")
.HasColumnName("from_version");
b.Property<int>("InstanceId")
.HasColumnType("integer")
.HasColumnName("instance_id");
b.Property<string>("ToLevel")
.HasColumnType("text")
.HasColumnName("to_level");
b.Property<string>("ToVersion")
.IsRequired()
.HasColumnType("text")
.HasColumnName("to_version");
b.HasKey("Id")
.HasName("pk_version_changes");
b.HasIndex("InstanceId", "ChangedAt")
.HasDatabaseName("ix_version_changes_instance_id_changed_at");
b.ToTable("version_changes", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.LicenseRecord", b =>
{
b.Property<string>("LicenseId")
.HasColumnType("text")
.HasColumnName("license_id");
b.Property<int>("CustomerId")
.HasColumnType("integer")
.HasColumnName("customer_id");
b.Property<DateTime>("ExpiresAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("expires_at");
b.Property<DateTime>("FirstSeenAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("first_seen_at");
b.Property<DateTime>("IssuedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("issued_at");
b.Property<DateTime>("LastSeenAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("last_seen_at");
b.Property<int>("MaxInstances")
.HasColumnType("integer")
.HasColumnName("max_instances");
b.Property<string>("Plan")
.IsRequired()
.HasColumnType("text")
.HasColumnName("plan");
b.Property<DateTime?>("RevokedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("revoked_at");
b.Property<string>("RevokedReason")
.HasColumnType("text")
.HasColumnName("revoked_reason");
b.HasKey("LicenseId")
.HasName("pk_licenses");
b.HasIndex("CustomerId")
.HasDatabaseName("ix_licenses_customer_id");
b.ToTable("licenses", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.RuleState", b =>
{
b.Property<int>("InstanceId")
.HasColumnType("integer")
.HasColumnName("instance_id");
b.Property<string>("RuleId")
.HasColumnType("text")
.HasColumnName("rule_id");
b.Property<string>("Outcome")
.IsRequired()
.HasColumnType("text")
.HasColumnName("outcome");
b.Property<string>("Reason")
.HasColumnType("text")
.HasColumnName("reason");
b.Property<DateTime>("UpdatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("updated_at");
b.HasKey("InstanceId", "RuleId")
.HasName("pk_rule_states");
b.ToTable("rule_states", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.StaffUser", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<int>("Id"));
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<string>("DisplayName")
.IsRequired()
.HasColumnType("text")
.HasColumnName("display_name");
b.Property<bool>("IsActive")
.HasColumnType("boolean")
.HasColumnName("is_active");
b.Property<DateTime?>("LastLoginAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("last_login_at");
b.Property<string>("PasswordHash")
.IsRequired()
.HasColumnType("text")
.HasColumnName("password_hash");
b.Property<string>("UserName")
.IsRequired()
.HasColumnType("text")
.HasColumnName("user_name");
b.HasKey("Id")
.HasName("pk_staff_users");
b.HasIndex("UserName")
.IsUnique()
.HasDatabaseName("ix_staff_users_user_name");
b.ToTable("staff_users", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.Submission", b =>
{
b.Property<Guid>("SubmissionId")
.ValueGeneratedOnAdd()
.HasColumnType("uuid")
.HasColumnName("submission_id");
b.Property<bool>("Applied")
.HasColumnType("boolean")
.HasColumnName("applied");
b.Property<DateTime>("CollectedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("collected_at");
b.Property<int>("FindingCount")
.HasColumnType("integer")
.HasColumnName("finding_count");
b.Property<int>("InstanceId")
.HasColumnType("integer")
.HasColumnName("instance_id");
b.Property<DateTime>("ReceivedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("received_at");
b.HasKey("SubmissionId")
.HasName("pk_submissions");
b.HasIndex("ReceivedAt")
.HasDatabaseName("ix_submissions_received_at");
b.ToTable("submissions", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.SupportCase", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("integer")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<int>("Id"));
b.Property<DateTime?>("ClosedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("closed_at");
b.Property<DateTime>("CreatedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("created_at");
b.Property<int>("CustomerId")
.HasColumnType("integer")
.HasColumnName("customer_id");
b.Property<bool>("IsOpen")
.HasColumnType("boolean")
.HasColumnName("is_open");
b.Property<string>("Note")
.HasColumnType("text")
.HasColumnName("note");
b.Property<string>("Reference")
.IsRequired()
.HasColumnType("text")
.HasColumnName("reference");
b.Property<string>("Title")
.IsRequired()
.HasColumnType("text")
.HasColumnName("title");
b.HasKey("Id")
.HasName("pk_support_cases");
b.HasIndex("CustomerId")
.HasDatabaseName("ix_support_cases_customer_id");
b.ToTable("support_cases", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.HubFinding", b =>
{
b.HasOne("SQLInsight.Hub.Data.Instance", "Instance")
.WithMany("Findings")
.HasForeignKey("InstanceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_findings_instances_instance_id");
b.HasOne("SQLInsight.Hub.Data.SupportCase", "SupportCase")
.WithMany("Findings")
.HasForeignKey("SupportCaseId")
.OnDelete(DeleteBehavior.SetNull)
.HasConstraintName("fk_findings_support_cases_support_case_id");
b.Navigation("Instance");
b.Navigation("SupportCase");
});
modelBuilder.Entity("SQLInsight.Hub.Data.Instance", b =>
{
b.HasOne("SQLInsight.Hub.Data.Customer", "Customer")
.WithMany("Instances")
.HasForeignKey("CustomerId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_instances_customers_customer_id");
b.Navigation("Customer");
});
modelBuilder.Entity("SQLInsight.Hub.Data.InstanceVersionChange", b =>
{
b.HasOne("SQLInsight.Hub.Data.Instance", "Instance")
.WithMany()
.HasForeignKey("InstanceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_version_changes_instances_instance_id");
b.Navigation("Instance");
});
modelBuilder.Entity("SQLInsight.Hub.Data.LicenseRecord", b =>
{
b.HasOne("SQLInsight.Hub.Data.Customer", "Customer")
.WithMany("Licenses")
.HasForeignKey("CustomerId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_licenses_customers_customer_id");
b.Navigation("Customer");
});
modelBuilder.Entity("SQLInsight.Hub.Data.SupportCase", b =>
{
b.HasOne("SQLInsight.Hub.Data.Customer", "Customer")
.WithMany("SupportCases")
.HasForeignKey("CustomerId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_support_cases_customers_customer_id");
b.Navigation("Customer");
});
modelBuilder.Entity("SQLInsight.Hub.Data.Customer", b =>
{
b.Navigation("Instances");
b.Navigation("Licenses");
b.Navigation("SupportCases");
});
modelBuilder.Entity("SQLInsight.Hub.Data.Instance", b =>
{
b.Navigation("Findings");
});
modelBuilder.Entity("SQLInsight.Hub.Data.SupportCase", b =>
{
b.Navigation("Findings");
});
#pragma warning restore 612, 618
}
}
}

View file

@ -0,0 +1,51 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
#nullable disable
namespace SQLInsight.Hub.Data.Migrations
{
/// <inheritdoc />
public partial class VersionChanges : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "version_changes",
columns: table => new
{
id = table.Column<long>(type: "bigint", nullable: false)
.Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
instance_id = table.Column<int>(type: "integer", nullable: false),
from_version = table.Column<string>(type: "text", nullable: false),
to_version = table.Column<string>(type: "text", nullable: false),
to_level = table.Column<string>(type: "text", nullable: true),
changed_at = table.Column<DateTime>(type: "timestamp with time zone", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("pk_version_changes", x => x.id);
table.ForeignKey(
name: "fk_version_changes_instances_instance_id",
column: x => x.instance_id,
principalTable: "instances",
principalColumn: "id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "ix_version_changes_instance_id_changed_at",
table: "version_changes",
columns: new[] { "instance_id", "changed_at" });
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "version_changes");
}
}
}

View file

@ -194,6 +194,46 @@ namespace SQLInsight.Hub.Data.Migrations
b.ToTable("instances", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.InstanceVersionChange", b =>
{
b.Property<long>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("bigint")
.HasColumnName("id");
NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<long>("Id"));
b.Property<DateTime>("ChangedAt")
.HasColumnType("timestamp with time zone")
.HasColumnName("changed_at");
b.Property<string>("FromVersion")
.IsRequired()
.HasColumnType("text")
.HasColumnName("from_version");
b.Property<int>("InstanceId")
.HasColumnType("integer")
.HasColumnName("instance_id");
b.Property<string>("ToLevel")
.HasColumnType("text")
.HasColumnName("to_level");
b.Property<string>("ToVersion")
.IsRequired()
.HasColumnType("text")
.HasColumnName("to_version");
b.HasKey("Id")
.HasName("pk_version_changes");
b.HasIndex("InstanceId", "ChangedAt")
.HasDatabaseName("ix_version_changes_instance_id_changed_at");
b.ToTable("version_changes", (string)null);
});
modelBuilder.Entity("SQLInsight.Hub.Data.LicenseRecord", b =>
{
b.Property<string>("LicenseId")
@ -437,6 +477,18 @@ namespace SQLInsight.Hub.Data.Migrations
b.Navigation("Customer");
});
modelBuilder.Entity("SQLInsight.Hub.Data.InstanceVersionChange", b =>
{
b.HasOne("SQLInsight.Hub.Data.Instance", "Instance")
.WithMany()
.HasForeignKey("InstanceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired()
.HasConstraintName("fk_version_changes_instances_instance_id");
b.Navigation("Instance");
});
modelBuilder.Entity("SQLInsight.Hub.Data.LicenseRecord", b =>
{
b.HasOne("SQLInsight.Hub.Data.Customer", "Customer")

View file

@ -35,6 +35,9 @@ public sealed class HubServerOptions
public IList<string> KnownProxies { get; } = [];
public RetentionOptions Retention { get; set; } = new();
/// <summary>Upgrade insights shared with agents (aggregated, anonymous).</summary>
public Insights.InsightOptions Insights { get; set; } = new();
}
/// <summary>How long the hub keeps data it no longer needs. Applied once a day.</summary>

View file

@ -0,0 +1,85 @@
using Microsoft.EntityFrameworkCore;
using SQLInsight.Contracts;
using System.Globalization;
using SQLInsight.Hub.Data;
namespace SQLInsight.Hub.Insights;
public sealed class InsightOptions
{
/// <summary>Fewest different customers behind an insight before it is shared with other customers.</summary>
public int MinCustomers { get; set; } = 3;
/// <summary>A finding that went away this soon after an update counts as resolved by it.</summary>
public TimeSpan ResolvedWithin { get; set; } = TimeSpan.FromDays(3);
/// <summary>A finding still there this long after an update counts as not resolved by it.</summary>
public TimeSpan PersistedAfter { get; set; } = TimeSpan.FromDays(7);
}
/// <summary>
/// "This CU made the problem go away elsewhere": for every build change, which of the instance's active findings
/// disappeared shortly after, and which stayed.
/// </summary>
public sealed class InsightCalculator(HubDbContext db, InsightOptions options, TimeProvider time)
{
/// <summary>Rules about the version itself are always "fixed" by an update, so they say nothing.</summary>
private static readonly string[] Excluded = ["SQLI-VER-"];
/// <summary>All upgrade effects, also those from fewer customers than the sharing threshold (for staff).</summary>
public async Task<IReadOnlyList<UpgradeInsight>> UpgradeEffectsAsync(CancellationToken cancellationToken)
{
var now = time.GetUtcNow().UtcDateTime;
var changes = await db.VersionChanges.AsNoTracking()
.Select(v => new { v.InstanceId, v.Instance!.CustomerId, v.ToVersion, v.ToLevel, v.ChangedAt })
.ToListAsync(cancellationToken).ConfigureAwait(false);
var instanceIds = changes.Select(c => c.InstanceId).Distinct().ToList();
var findings = await db.Findings.AsNoTracking()
.Where(f => instanceIds.Contains(f.InstanceId))
.Select(f => new { f.InstanceId, f.RuleId, f.FirstSeenAt, f.ResolvedAt })
.ToListAsync(cancellationToken).ConfigureAwait(false);
var byInstance = findings.ToLookup(f => f.InstanceId);
var events = new List<(string RuleId, int Major, string ToBuild, string? ToLevel, int CustomerId, bool Resolved)>();
foreach (var change in changes)
{
if (!int.TryParse(change.ToVersion.Split('.')[0], NumberStyles.None, CultureInfo.InvariantCulture, out var major))
{
continue;
}
var watchedLongEnough = now - change.ChangedAt >= options.PersistedAfter;
foreach (var rule in byInstance[change.InstanceId]
.Where(f => f.FirstSeenAt < change.ChangedAt && (f.ResolvedAt is null || f.ResolvedAt >= change.ChangedAt))
.Where(f => !Excluded.Any(prefix => f.RuleId.StartsWith(prefix, StringComparison.Ordinal)))
.GroupBy(f => f.RuleId))
{
// Per instance and rule: resolved when every occurrence active at the update went away soon after.
var resolved = rule.All(f => f.ResolvedAt is { } at && at - change.ChangedAt <= options.ResolvedWithin);
var persisted = rule.Any(f => f.ResolvedAt is null ? watchedLongEnough : f.ResolvedAt - change.ChangedAt >= options.PersistedAfter);
if (resolved || persisted)
{
events.Add((rule.Key, major, change.ToVersion, change.ToLevel, change.CustomerId, resolved));
}
}
}
return events
.GroupBy(e => (e.RuleId, e.Major, e.ToBuild))
.Select(g => new UpgradeInsight(
g.Key.RuleId, g.Key.Major, g.Key.ToBuild, g.Select(e => e.ToLevel).FirstOrDefault(l => l is not null),
g.Where(e => e.Resolved).Select(e => e.CustomerId).Distinct().Count(),
g.Count(e => e.Resolved), g.Count(e => !e.Resolved)))
.OrderBy(i => i.RuleId, StringComparer.Ordinal).ThenBy(i => i.Major).ThenBy(i => i.ToBuild, StringComparer.Ordinal)
.ToList();
}
/// <summary>What may be shared with customers: enough different customers, and resolved more often than not.</summary>
public async Task<HubInsights> ForCustomersAsync(CancellationToken cancellationToken)
{
var shared = (await UpgradeEffectsAsync(cancellationToken).ConfigureAwait(false))
.Where(i => i.Customers >= options.MinCustomers && i.ResolvedShare > 0.5)
.ToList();
return new HubInsights(time.GetUtcNow(), options.MinCustomers, shared);
}
}

View file

@ -0,0 +1,43 @@
using Microsoft.Extensions.Options;
using SQLInsight.Contracts;
using SQLInsight.Hub.Api;
using SQLInsight.Hub.Data;
namespace SQLInsight.Hub.Insights;
/// <summary>GET /api/v1/insights for licensed agents. Computed at most once an hour.</summary>
public static class InsightsEndpoint
{
public static readonly TimeSpan CacheFor = TimeSpan.FromHours(1);
private static readonly SemaphoreSlim Gate = new(1, 1);
private static HubInsights? _cached;
public static void Map(IEndpointRouteBuilder app) =>
app.MapGet("/api/v1/insights", async (HttpRequest request, HubDbContext db, IOptions<HubServerOptions> options, InsightCalculator calculator, TimeProvider time, CancellationToken ct) =>
{
var (license, rejected) = await FindingsEndpoint.AuthorizeAsync(request, db, options.Value, time, ct).ConfigureAwait(false);
if (license is null)
{
return rejected!;
}
await Gate.WaitAsync(ct).ConfigureAwait(false);
try
{
if (_cached is null || time.GetUtcNow() - _cached.GeneratedAt > CacheFor)
{
_cached = await calculator.ForCustomersAsync(ct).ConfigureAwait(false);
}
return Results.Ok(_cached);
}
finally
{
Gate.Release();
}
}).AllowAnonymous();
/// <summary>For tests and after a change of settings.</summary>
public static void ClearCache() => _cached = null;
}

View file

@ -18,6 +18,7 @@ public sealed class HubMaintenance(HubDbContext db, IOptions<HubServerOptions> o
var silentBefore = now - retention.SilentInstances;
var silent = db.Instances.Where(i => i.LastSubmissionAt != null && i.LastSubmissionAt < silentBefore).Select(i => i.Id);
await db.RuleStates.Where(r => silent.Contains(r.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.VersionChanges.Where(v => silent.Contains(v.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.Submissions.Where(s => silent.Contains(s.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.Findings.Where(f => silent.Contains(f.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
var instances = await db.Instances.Where(i => i.LastSubmissionAt != null && i.LastSubmissionAt < silentBefore).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
@ -49,6 +50,7 @@ public sealed class HubMaintenance(HubDbContext db, IOptions<HubServerOptions> o
await using var transaction = await db.Database.BeginTransactionAsync(cancellationToken).ConfigureAwait(false);
var instances = db.Instances.Where(i => i.CustomerId == customer.Id).Select(i => i.Id);
await db.RuleStates.Where(r => instances.Contains(r.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.VersionChanges.Where(v => instances.Contains(v.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.Submissions.Where(s => instances.Contains(s.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.Findings.Where(f => instances.Contains(f.InstanceId)).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);
await db.Instances.Where(i => i.CustomerId == customer.Id).ExecuteDeleteAsync(cancellationToken).ConfigureAwait(false);

View file

@ -0,0 +1,48 @@
@page "/portal/indsigt"
@attribute [Authorize]
@using SQLInsight.Contracts
@using SQLInsight.Hub.Insights
@inject InsightCalculator Calculator
@inject InsightOptions Options
<PageTitle>Opgraderinger · SQLInsight Hub</PageTitle>
<h1>Hvad forsvinder efter en opdatering?</h1>
<p class="lead">
For hver instans, der har skiftet build: hvilke fund forsvandt inden for @Options.ResolvedWithin.TotalDays.ToString("0", CultureInfo.InvariantCulture) dage,
og hvilke var der stadig efter @Options.PersistedAfter.TotalDays.ToString("0", CultureInfo.InvariantCulture) dage. Det er sammenhæng, ikke bevis.
Rækker med mindst @Options.MinCustomers kunder, hvor fundet oftest forsvandt, deles anonymt med kundernes agenter.
</p>
<section class="card">
@if (_effects.Count == 0)
{
<p class="empty">Ingen instanser har skiftet build endnu.</p>
}
else
{
<table>
<thead><tr><th>Regel</th><th>Version</th><th>Opdateret til</th><th class="num">Kunder</th><th class="num">Forsvandt</th><th class="num">Blev</th><th class="num">Andel</th><th>Deles</th></tr></thead>
<tbody>
@foreach (var e in _effects)
{
var shared = e.Customers >= Options.MinCustomers && e.ResolvedShare > 0.5;
<tr>
<td class="mono">@e.RuleId</td>
<td class="num">@e.Major</td>
<td>@e.ToBuild @(e.ToLevel is { } level ? $"({level})" : "")</td>
<td class="num">@e.Customers</td>
<td class="num">@e.Resolved</td>
<td class="num">@e.Persisted</td>
<td class="num">@e.ResolvedShare.ToString("P0", CultureInfo.GetCultureInfo("da-DK"))</td>
<td>@(shared ? "Ja" : "Nej")</td>
</tr>
}
</tbody>
</table>
}
</section>
@code {
private IReadOnlyList<UpgradeInsight> _effects = [];
protected override async Task OnInitializedAsync() => _effects = await Calculator.UpgradeEffectsAsync(CancellationToken.None);
}

View file

@ -7,6 +7,7 @@
<span class="nav-label">Kunder</span>
<NavLink href="/portal" Match="NavLinkMatch.All">Overblik</NavLink>
<NavLink href="/portal/fund">Alle fund</NavLink>
<NavLink href="/portal/indsigt">Opgraderinger</NavLink>
<NavLink href="/portal/sager">Supportsager</NavLink>
<NavLink href="/portal/licenser">Licenser</NavLink>
</nav>

View file

@ -61,6 +61,8 @@ public static class HubProgram
}
});
builder.Services.AddScoped<HubMaintenance>();
builder.Services.AddSingleton(settings.Insights);
builder.Services.AddScoped<SQLInsight.Hub.Insights.InsightCalculator>();
builder.Services.AddHostedService<HubRetentionService>();
builder.Services.AddScoped<StaffAccounts>();
builder.Services.AddScoped<PortalQueries>();
@ -124,6 +126,7 @@ public static class HubProgram
app.MapGet("/", () => Results.Redirect("/portal"));
app.MapGet("/health", () => Results.Ok(new { status = "ok" }));
FindingsEndpoint.Map(app);
SQLInsight.Hub.Insights.InsightsEndpoint.Map(app);
PortalEndpoints.Map(app);
app.MapRazorComponents<PortalApp>();
return app;

View file

@ -68,6 +68,23 @@ public sealed partial class InsightRepository
""", new { outboundId, error, now = Utc(now), next = nextAttemptAt is { } n ? Utc(n) : (DateTime?)null }).ConfigureAwait(false);
}
public async Task SaveHubInsightsAsync(string json, DateTimeOffset fetchedAt, CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);
await connection.ExecuteAsync("""
MERGE insight.HubInsights AS t USING (SELECT 1 AS Id) AS s ON t.Id = s.Id
WHEN MATCHED THEN UPDATE SET FetchedAt = @at, Json = @json
WHEN NOT MATCHED THEN INSERT (Id, FetchedAt, Json) VALUES (1, @at, @json);
""", new { json, at = Utc(fetchedAt) }).ConfigureAwait(false);
}
/// <summary>The last insights fetched from the hub, as JSON (SQLInsight.Contracts.HubInsights), or null.</summary>
public async Task<string?> GetHubInsightsJsonAsync(CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);
return await connection.QuerySingleOrDefaultAsync<string>("SELECT Json FROM insight.HubInsights WHERE Id = 1;").ConfigureAwait(false);
}
public async Task SetHubErrorAsync(string error, DateTimeOffset now, CancellationToken cancellationToken = default)
{
await using var connection = await OpenAsync(cancellationToken).ConfigureAwait(false);

View file

@ -0,0 +1,7 @@
/* Aggregated, anonymous experience from SQLInsight Hub ("fixed elsewhere after updating to …"). One row. */
CREATE TABLE insight.HubInsights
(
Id tinyint NOT NULL CONSTRAINT PK_HubInsights PRIMARY KEY CONSTRAINT CK_HubInsights_Single CHECK (Id = 1),
FetchedAt datetime2(0) NOT NULL,
Json nvarchar(max) NOT NULL
);

View file

@ -54,6 +54,20 @@
}
</section>
@if (_elsewhere.Count > 0 && _finding.ResolvedAt is null)
{
var best = _elsewhere[0];
<section class="panel insight">
<h2>Erfaring fra SQLInsight-kunder</h2>
<p>
Hos <strong>@best.Customers kunder</strong> forsvandt dette fund efter opdatering til
<strong>@(best.ToLevel is { } level ? $"{level} ({best.ToBuild})" : best.ToBuild)</strong>
(@best.Resolved af @(best.Resolved + best.Persisted) instanser). Instansen kører @_version.
</p>
<p class="muted">Tal fra SQLInsight Hub, samlet anonymt på tværs af mindst @_minCustomers kunder. Det er en sammenhæng, ikke en garanti: læs anbefalingen ovenfor, og test opdateringen først.</p>
</section>
}
@if (_finding.References.Count > 0)
{
<section class="panel">
@ -71,6 +85,21 @@
[Parameter] public long Id { get; set; }
private StoredFinding? _finding;
private IReadOnlyList<SQLInsight.Contracts.UpgradeInsight> _elsewhere = [];
private string? _version;
private int _minCustomers;
protected override async Task OnInitializedAsync() => _finding = await Repository.GetFindingAsync(Id);
protected override async Task OnInitializedAsync()
{
_finding = await Repository.GetFindingAsync(Id);
if (_finding is null || await Repository.GetHubInsightsJsonAsync() is not { } json)
{
return;
}
var insights = System.Text.Json.JsonSerializer.Deserialize<SQLInsight.Contracts.HubInsights>(json, System.Text.Json.JsonSerializerOptions.Web);
_version = (await Repository.GetInstanceAsync(_finding.InstanceName))?.ProductVersion;
_minCustomers = insights?.MinCustomers ?? 0;
_elsewhere = SQLInsight.Contracts.HubInsightMatcher.For(insights, _finding.RuleId, _version);
}
}

View file

@ -6,6 +6,7 @@
<ProjectReference Include="..\SQLInsight.Core\SQLInsight.Core.csproj" />
<ProjectReference Include="..\SQLInsight.Repository\SQLInsight.Repository.csproj" />
<ProjectReference Include="..\SQLInsight.Reports\SQLInsight.Reports.csproj" />
<ProjectReference Include="..\SQLInsight.Contracts\SQLInsight.Contracts.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Authentication.Negotiate" />

View file

@ -160,3 +160,4 @@ details .script pre { max-height: 32rem; }
.report-links { display: flex; gap: .6rem; flex-wrap: wrap; }
a.button { display: inline-block; background: var(--accent); color: #fff; padding: .5rem 1rem; border-radius: 8px; font-weight: 600; }
a.button:hover { filter: brightness(1.08); }
.panel.insight { border-left: 4px solid var(--accent); }

View file

@ -105,4 +105,46 @@ public class HubEndToEndTests(SqlServerFixture sql, HubFixture hub) : IClassFixt
Assert.Equal(0, calls);
Assert.Contains("https", (await _repository.GetHubStatusAsync())!.LastError, StringComparison.Ordinal);
}
[Fact]
public async Task Experience_from_other_customers_reaches_the_customers_web_ui()
{
for (var i = 0; i < 3; i++)
{
await HubInsightTests.UpgradeAsync(hub, $"Erfaring {i}", ["SQLI-CFG-005"], []);
}
SQLInsight.Hub.Insights.InsightsEndpoint.ClearCache();
var options = new HubOptions { Enabled = true, Url = new Uri("http://localhost/"), License = hub.License("Modtager A/S") };
options.TrustedKeys[HubFixture.KeyId] = hub.PublicKey;
var sync = new HubSyncService(Options.Create(options), _repository, new TestServerClients(() => hub.Factory.CreateClient()), TimeProvider.System, NullLogger<HubSyncService>.Instance);
Assert.True(await sync.FetchInsightsAsync(hub.Factory.CreateClient(), options, CancellationToken.None));
var instanceId = await _repository.UpsertInstanceAsync("prod-sql01", new SQLInsight.Core.Instances.InstanceInfo
{
ServerName = "SQL01", Version = SQLInsight.Core.Instances.SqlServerVersion.Parse(HubInsightTests.OldBuild), ProductLevel = "RTM", ProductUpdateLevel = "CU20",
Edition = "Enterprise Edition", EngineEdition = SQLInsight.Core.Instances.EngineEdition.Enterprise,
}, "SqlClient", false, DateTimeOffset.UtcNow);
await _repository.SaveRuleResultsAsync(instanceId,
[
new RuleResult
{
RuleId = "SQLI-CFG-005", RuleTitle = "Plan cache", Outcome = RuleOutcome.Findings,
Findings = [new SQLInsight.Core.Findings.Finding { RuleId = "SQLI-CFG-005", Severity = SQLInsight.Core.Findings.Severity.Medium, Title = "Plan cache er fyldt med engangsplaner", Explanation = "e", Recommendation = "r" }],
},
], DateTimeOffset.UtcNow);
var findingId = (await _repository.GetActiveFindingsAsync(instanceId)).Single().FindingId;
await using var web = new Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory<Program>().WithWebHostBuilder(b =>
{
Microsoft.AspNetCore.Hosting.HostingAbstractionsWebHostBuilderExtensions.UseEnvironment(b, "Development");
b.UseSetting("SQLInsight:Repository:ConnectionString", sql.DatabaseConnectionString(_database));
b.UseSetting("SQLInsight:Web:Authentication", "None");
});
var html = await web.CreateClient().GetStringAsync($"/fund/{findingId}");
Assert.Contains("Erfaring fra SQLInsight-kunder", html, StringComparison.Ordinal);
Assert.Contains("Hos <strong>3 kunder</strong>", html, StringComparison.Ordinal);
Assert.Contains("CU21 (15.0.4415.2)", html, StringComparison.Ordinal);
}
}

View file

@ -0,0 +1,123 @@
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using SQLInsight.Contracts;
using SQLInsight.Hub.Data;
using SQLInsight.Hub.Insights;
namespace SQLInsight.IntegrationTests;
[Trait("Category", "Integration")]
public class HubInsightTests(HubFixture hub) : IClassFixture<HubFixture>
{
public const string OldBuild = "15.0.4385.2";
public const string NewBuild = "15.0.4415.2";
internal static FindingsSubmission Submission(string build, string level, DateTimeOffset at, params string[] rules) => new()
{
SubmissionId = Guid.NewGuid(),
AgentVersion = "1.0.0",
CollectedAt = at,
Instance = new InstanceSnapshot { InstanceKey = "sql01", Release = "SQL Server 2019", ProductVersion = build, ProductUpdateLevel = level },
Rules = rules.Select(r => new RuleOutcomeDto { RuleId = r, Outcome = "Findings" }).ToList(),
Findings = rules.Select(r => new FindingDto
{
RuleId = r, Subject = "", Severity = "High", Title = r, Explanation = "e", Recommendation = "r", FirstSeenAt = at.AddHours(-1),
}).ToList(),
};
internal static async Task SubmitAsync(HttpClient client, string license, FindingsSubmission submission)
{
using var request = new HttpRequestMessage(HttpMethod.Post, "/api/v1/findings") { Content = JsonContent.Create(submission) };
request.Headers.Authorization = new AuthenticationHeaderValue("SQLInsight-License", license);
(await client.SendAsync(request)).EnsureSuccessStatusCode();
}
/// <summary>A customer whose instance had the rules' findings on the old build, then updated: <paramref name="fixedRules"/> went away.</summary>
internal static async Task UpgradeAsync(HubFixture hub, string customer, string[] before, string[] after)
{
var client = hub.Factory.CreateClient();
var license = hub.License(customer);
var t0 = DateTimeOffset.UtcNow.AddMinutes(-30);
await SubmitAsync(client, license, Submission(OldBuild, "CU20", t0, before));
await SubmitAsync(client, license, Submission(NewBuild, "CU21", t0.AddMinutes(15), after));
}
private async Task<HubInsights> InsightsAsync(string license)
{
InsightsEndpoint.ClearCache();
using var request = new HttpRequestMessage(HttpMethod.Get, "/api/v1/insights");
request.Headers.Authorization = new AuthenticationHeaderValue("SQLInsight-License", license);
var response = await hub.Factory.CreateClient().SendAsync(request);
response.EnsureSuccessStatusCode();
return (await response.Content.ReadFromJsonAsync<HubInsights>())!;
}
[Fact]
public async Task Findings_that_went_away_after_an_update_at_three_customers_are_shared()
{
var tag = Guid.NewGuid().ToString("N")[..6];
for (var i = 0; i < 3; i++)
{
await UpgradeAsync(hub, $"Kunde {tag} {i}", ["SQLI-CFG-005", "SQLI-VER-002", "SQLI-IDX-001"], ["SQLI-IDX-001"]);
}
await UpgradeAsync(hub, $"Kunde {tag} x", ["SQLI-CFG-003"], []);
await UpgradeAsync(hub, $"Kunde {tag} y", ["SQLI-CFG-003"], []);
var insights = await InsightsAsync(hub.License($"Kunde {tag} 0"));
var shared = Assert.Single(insights.Upgrades, u => u.RuleId == "SQLI-CFG-005");
Assert.Equal((15, NewBuild, "CU21", 3, 3, 0), (shared.Major, shared.ToBuild, shared.ToLevel, shared.Customers, shared.Resolved, shared.Persisted));
Assert.DoesNotContain(insights.Upgrades, u => u.RuleId == "SQLI-VER-002");
Assert.DoesNotContain(insights.Upgrades, u => u.RuleId == "SQLI-CFG-003"); // only two customers
Assert.DoesNotContain(insights.Upgrades, u => u.RuleId == "SQLI-IDX-001"); // still there
Assert.Equal(3, insights.MinCustomers);
}
[Fact]
public async Task Findings_still_there_a_week_after_the_update_count_against_it()
{
var tag = Guid.NewGuid().ToString("N")[..6];
for (var i = 0; i < 3; i++)
{
await UpgradeAsync(hub, $"Stadig {tag} {i}", ["SQLI-STAT-001"], ["SQLI-STAT-001"]);
}
await using var scope = hub.Factory.Services.CreateAsyncScope();
var db = scope.ServiceProvider.GetRequiredService<HubDbContext>();
var instances = db.Instances.Where(i => i.Customer!.Name.StartsWith($"Stadig {tag}")).Select(i => i.Id);
await db.VersionChanges.Where(v => instances.Contains(v.InstanceId)).ExecuteUpdateAsync(v => v.SetProperty(x => x.ChangedAt, x => x.ChangedAt.AddDays(-10)));
await db.Findings.Where(f => instances.Contains(f.InstanceId)).ExecuteUpdateAsync(f => f.SetProperty(x => x.FirstSeenAt, x => x.FirstSeenAt.AddDays(-11)));
var effects = await scope.ServiceProvider.GetRequiredService<InsightCalculator>().UpgradeEffectsAsync(CancellationToken.None);
var stat = Assert.Single(effects, e => e.RuleId == "SQLI-STAT-001");
Assert.Equal((0, 3), (stat.Resolved, stat.Persisted));
Assert.DoesNotContain((await InsightsAsync(hub.License($"Stadig {tag} 0"))).Upgrades, u => u.RuleId == "SQLI-STAT-001");
}
[Fact]
public async Task Insights_require_a_license()
{
Assert.Equal(HttpStatusCode.Unauthorized, (await hub.Factory.CreateClient().GetAsync("/api/v1/insights")).StatusCode);
}
[Fact]
public void Matcher_picks_newer_builds_of_the_same_major_version()
{
var insights = new HubInsights(DateTimeOffset.UtcNow, 3,
[
new UpgradeInsight("SQLI-CFG-005", 15, "15.0.4415.2", "CU21", 3, 3, 0),
new UpgradeInsight("SQLI-CFG-005", 15, "15.0.4430.1", "CU24", 5, 5, 1),
new UpgradeInsight("SQLI-CFG-005", 16, "16.0.4175.1", "CU20", 9, 9, 0),
]);
Assert.Equal(["CU24", "CU21"], HubInsightMatcher.For(insights, "SQLI-CFG-005", "15.0.4385.2").Select(i => i.ToLevel));
Assert.Equal(["CU24"], HubInsightMatcher.For(insights, "SQLI-CFG-005", "15.0.4420.2").Select(i => i.ToLevel));
Assert.Empty(HubInsightMatcher.For(insights, "SQLI-CFG-004", "15.0.4385.2"));
Assert.Empty(HubInsightMatcher.For(null, "SQLI-CFG-005", "15.0.4385.2"));
}
}