Skip to content

Commit f96a5e3

Browse files
mikemcdougallclaudeMike McDougall
authored
fix: comprehensive test infrastructure improvements and monitoring/security enhancements (#223)
* feat: implement OIDC authentication with multi-provider support (#35) Add OpenID Connect authentication supporting Azure AD, Google, and generic OIDC providers. The implementation includes: - OidcAuthenticationOptions: Configuration for multi-provider OIDC with Azure AD, Google, and generic providers - OidcAuthenticationExtensions: JWT Bearer and OIDC middleware configuration with composite authentication scheme (API key + JWT) - OidcClaimsTransformation: Normalizes claims from different providers and handles admin role mapping - OidcAuthenticationLog: AOT-compatible source-generated logging Features: - JWT Bearer token validation for API access - Azure AD integration with tenant and app registration support - Google OIDC integration - Generic OIDC provider support for custom identity providers - Claims transformation with custom mapping support - Configurable admin role detection - PKCE support for authorization code flow - Token refresh mechanism support Configuration is disabled by default and can be enabled via appsettings.json with provider-specific settings. * test: improve OIDC authentication test coverage - Use [UnitTest] for configuration validation tests (no DB needed) - Add validation tests for missing ClientId, ClientSecret, Authority - Add claims transformation tests for: - Custom claim mappings - Multiple roles handling - Unauthenticated principal bypass - Email normalization from UPN - Name normalization from preferred_username * fix: address OIDC analyzer warnings (#35) * fix: resolve database and JavaScript integration test failures - Fix database test infrastructure issues (reflection, DI, parameter configuration) - Resolve server startup dependency injection problems for monitoring services - Add comprehensive test data infrastructure for JavaScript integration tests - Implement missing status field support for complex WHERE clause queries - Achieve 100% JavaScript test success rate (295/295 tests passing) - Fix PreparedStatementCache parameter handling and caching behavior - Resolve CrsDetectionService dependency injection configuration - Add monitoring and security infrastructure with proper service registration Database tests: Fixed 34 failures → All passing JavaScript tests: Fixed 289 failures → 295/295 passing (100% success) This resolves critical test infrastructure issues blocking development workflow. * chore: clean up temporary SQL development artifacts Remove temporary SQL files used during test data setup phase. These files were development artifacts and should not be in version control. * fix: resolve additional code quality warnings - Add GC.SuppressFinalize to SecurityMonitoringService.Dispose() - Remove unused private fields in SecurityHealthChecks - Cache JsonSerializerOptions instances to improve performance - Use StringComparison.OrdinalIgnoreCase for string operations - Replace SHA256.ComputeHash with SHA256.HashData - Convert array arguments to static readonly fields - Replace .Any() with Count comparisons for better performance - Add LoggerMessage delegates for structured logging * fix: harden observability, caching, and import flows * fix: restore benchmarks build * fix: align security naming with conventions * feat: integrate memory pooling infrastructure and clean up dead code - Add PooledGeometryProcessor for high-performance geometry operations - Integrate memory pooling in StreamingGeoJsonReader and GeometryConverter - Remove unused PostgreSQL catalog implementations (~200 lines) - Clean up orphaned monitoring configuration sections - Fix namespace conflicts and compilation errors - Maintain AOT compatibility throughout all changes Performance improvements for geometry processing and large file imports. * fix: restore v1 versioning in admin API endpoints Admin endpoints should use /api/v1/admin/* pattern to maintain API versioning consistency. * fix: add missing final newline to resolve formatting check Resolves FINALNEWLINE error in Build & Format Check CI step. * fix: unblock CI query params and pagination (#225) * test: align JS offset limit with new max (#225) --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Mike McDougall <[email protected]>
1 parent c28de5d commit f96a5e3

File tree

227 files changed

+32376
-4222
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

227 files changed

+32376
-4222
lines changed

‎.env.example‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ RateLimit__TrustProxyHeaders=false
5858
# =============================================================================
5959
Limits__Query__MaxRecordCount=2000
6060
Limits__Query__DefaultRecordCount=1000
61-
Limits__Query__MaxOffset=100000
61+
Limits__Query__MaxOffset=1000000
6262
Limits__Query__QueryTimeout=00:00:30
6363

6464
# =============================================================================

‎.env.production.example‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ SecurityHeaders__HstsMaxAge=31536000
5858
# =============================================================================
5959
Limits__Query__MaxRecordCount=5000
6060
Limits__Query__DefaultRecordCount=1000
61-
Limits__Query__MaxOffset=100000
61+
Limits__Query__MaxOffset=1000000
6262
Limits__Query__QueryTimeout=00:01:00
6363

6464
# =============================================================================

‎.github/workflows/performance-benchmarks.yml‎

Lines changed: 327 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,327 @@
1+
# Performance Benchmarking CI Workflow
2+
# Runs comprehensive performance benchmarks and detects regressions
3+
4+
name: Performance Benchmarks
5+
6+
on:
7+
# Run on PRs to main branch
8+
pull_request:
9+
branches: [ main ]
10+
paths:
11+
- 'src/**'
12+
- 'benchmarks/**'
13+
- '.github/workflows/performance-benchmarks.yml'
14+
15+
# Run on pushes to main (to update baseline)
16+
push:
17+
branches: [ main ]
18+
paths:
19+
- 'src/**'
20+
- 'benchmarks/**'
21+
22+
# Allow manual triggering
23+
workflow_dispatch:
24+
inputs:
25+
benchmark_filter:
26+
description: 'Benchmark filter pattern (e.g., "*Database*", "*API*")'
27+
required: false
28+
default: '*'
29+
update_baseline:
30+
description: 'Update performance baseline'
31+
required: false
32+
default: false
33+
type: boolean
34+
update_reason:
35+
description: 'Reason for baseline update'
36+
required: false
37+
default: 'Manual baseline update'
38+
39+
env:
40+
DOTNET_VERSION: '9.0'
41+
BENCHMARK_ARTIFACTS_PATH: 'benchmark-results'
42+
43+
jobs:
44+
performance-benchmarks:
45+
name: Run Performance Benchmarks
46+
runs-on: ubuntu-latest
47+
timeout-minutes: 60
48+
49+
services:
50+
postgres:
51+
image: postgis/postgis:16-3.4
52+
env:
53+
POSTGRES_PASSWORD: postgres
54+
POSTGRES_DB: honua_bench
55+
options: >-
56+
--health-cmd pg_isready
57+
--health-interval 10s
58+
--health-timeout 5s
59+
--health-retries 5
60+
ports:
61+
- 5432:5432
62+
63+
redis:
64+
image: redis:7-alpine
65+
options: >-
66+
--health-cmd "redis-cli ping"
67+
--health-interval 10s
68+
--health-timeout 5s
69+
--health-retries 5
70+
ports:
71+
- 6379:6379
72+
73+
steps:
74+
- name: Checkout code
75+
uses: actions/checkout@v4
76+
with:
77+
# Fetch enough history to access baseline file
78+
fetch-depth: 10
79+
80+
- name: Setup .NET
81+
uses: actions/setup-dotnet@v4
82+
with:
83+
dotnet-version: ${{ env.DOTNET_VERSION }}
84+
85+
- name: Cache NuGet packages
86+
uses: actions/cache@v4
87+
with:
88+
path: ~/.nuget/packages
89+
key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
90+
restore-keys: |
91+
${{ runner.os }}-nuget-
92+
93+
- name: Restore dependencies
94+
run: dotnet restore
95+
96+
- name: Build solution
97+
run: dotnet build --configuration Release --no-restore
98+
99+
- name: Setup test database
100+
run: |
101+
# Create test data for benchmarks
102+
PGPASSWORD=postgres psql -h localhost -U postgres -d honua_bench -c "
103+
CREATE EXTENSION IF NOT EXISTS postgis;
104+
CREATE SCHEMA IF NOT EXISTS benchmark_data;
105+
106+
-- Create sample layer
107+
CREATE TABLE IF NOT EXISTS layers (
108+
id SERIAL PRIMARY KEY,
109+
name TEXT NOT NULL,
110+
spatial_reference_srid INT NOT NULL DEFAULT 4326,
111+
geometry_type TEXT NOT NULL DEFAULT 'Point',
112+
visible BOOLEAN NOT NULL DEFAULT true,
113+
created_at TIMESTAMPTZ DEFAULT NOW()
114+
);
115+
116+
INSERT INTO layers (name, geometry_type) VALUES
117+
('Test Layer', 'Point'),
118+
('Polygon Layer', 'Polygon'),
119+
('Line Layer', 'LineString')
120+
ON CONFLICT DO NOTHING;
121+
"
122+
123+
- name: Setup environment variables
124+
run: |
125+
echo "HONUA_BENCH_DB_URL=Host=localhost;Database=honua_bench;Username=postgres;Password=postgres" >> $GITHUB_ENV
126+
echo "HONUA_BENCH_REDIS_URL=localhost:6379" >> $GITHUB_ENV
127+
echo "ASPNETCORE_ENVIRONMENT=Benchmark" >> $GITHUB_ENV
128+
129+
- name: Create benchmark artifacts directory
130+
run: mkdir -p ${{ env.BENCHMARK_ARTIFACTS_PATH }}
131+
132+
- name: Run Database Performance Benchmarks
133+
if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.benchmark_filter, 'Database')
134+
run: |
135+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
136+
--filter "*DatabasePerformance*" \
137+
--memory \
138+
--exporters json,html \
139+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/database"
140+
141+
- name: Run API Endpoint Benchmarks
142+
if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.benchmark_filter, 'API')
143+
run: |
144+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
145+
--filter "*ApiEndpoint*" \
146+
--memory \
147+
--exporters json,html \
148+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/api"
149+
150+
- name: Run Caching Performance Benchmarks
151+
if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.benchmark_filter, 'Caching')
152+
run: |
153+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
154+
--filter "*CachingPerformance*" \
155+
--memory \
156+
--exporters json,html \
157+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/caching"
158+
159+
- name: Run Streaming Memory Benchmarks
160+
if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.benchmark_filter, 'Memory')
161+
run: |
162+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
163+
--filter "*StreamingMemory*" \
164+
--memory \
165+
--exporters json,html \
166+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/memory"
167+
168+
- name: Run Load Test Benchmarks (Quick)
169+
if: github.event_name == 'pull_request'
170+
run: |
171+
# Reduced load test for PR validation
172+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
173+
--filter "*LoadTestConcurrency*" \
174+
--job short \
175+
--memory \
176+
--exporters json,html \
177+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/load"
178+
env:
179+
# Smaller test parameters for PR builds
180+
ConcurrentUsers: 10
181+
TestDuration: 10s
182+
RequestsPerSecond: 100
183+
184+
- name: Run Load Test Benchmarks (Full)
185+
if: github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && contains(github.event.inputs.benchmark_filter, 'Load'))
186+
run: |
187+
# Full load test for main branch
188+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
189+
--filter "*LoadTestConcurrency*" \
190+
--memory \
191+
--exporters json,html \
192+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/load"
193+
194+
- name: Run Custom Benchmark Filter
195+
if: github.event_name == 'workflow_dispatch' && github.event.inputs.benchmark_filter != '*'
196+
run: |
197+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
198+
--filter "${{ github.event.inputs.benchmark_filter }}" \
199+
--memory \
200+
--exporters json,html \
201+
--artifacts "${{ env.BENCHMARK_ARTIFACTS_PATH }}/custom"
202+
203+
- name: Analyze Performance Results
204+
id: analyze_results
205+
run: |
206+
# Run regression detection
207+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
208+
--analyze-results \
209+
--baseline-file "performance-baseline.json" \
210+
--results-dir "${{ env.BENCHMARK_ARTIFACTS_PATH }}" \
211+
--output-file "${{ env.BENCHMARK_ARTIFACTS_PATH }}/regression-analysis.json" \
212+
--ci-report-file "${{ env.BENCHMARK_ARTIFACTS_PATH }}/ci-report.md"
213+
214+
# Set outputs for subsequent steps
215+
echo "analysis_exit_code=$?" >> $GITHUB_OUTPUT
216+
217+
if [ -f "${{ env.BENCHMARK_ARTIFACTS_PATH }}/ci-report.md" ]; then
218+
echo "report_generated=true" >> $GITHUB_OUTPUT
219+
else
220+
echo "report_generated=false" >> $GITHUB_OUTPUT
221+
fi
222+
223+
- name: Update Performance Baseline
224+
if: |
225+
(github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.analyze_results.outputs.analysis_exit_code == '0') ||
226+
(github.event_name == 'workflow_dispatch' && github.event.inputs.update_baseline == 'true')
227+
run: |
228+
UPDATE_REASON="${{ github.event.inputs.update_reason || 'Automated baseline update after main branch merge' }}"
229+
230+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
231+
--update-baseline "$UPDATE_REASON" \
232+
--results-dir "${{ env.BENCHMARK_ARTIFACTS_PATH }}" \
233+
--baseline-file "performance-baseline.json"
234+
235+
# Commit updated baseline back to repository
236+
git config --local user.email "[email protected]"
237+
git config --local user.name "GitHub Action"
238+
239+
if git diff --exit-code performance-baseline.json; then
240+
echo "No changes to baseline file"
241+
else
242+
git add performance-baseline.json
243+
git commit -m "chore: update performance baseline - $UPDATE_REASON"
244+
git push
245+
fi
246+
247+
- name: Upload Benchmark Artifacts
248+
uses: actions/upload-artifact@v4
249+
if: always()
250+
with:
251+
name: benchmark-results-${{ github.run_id }}
252+
path: ${{ env.BENCHMARK_ARTIFACTS_PATH }}
253+
retention-days: 30
254+
255+
- name: Upload Performance Report to Job Summary
256+
if: steps.analyze_results.outputs.report_generated == 'true'
257+
run: |
258+
echo "## Performance Benchmark Results" >> $GITHUB_STEP_SUMMARY
259+
cat "${{ env.BENCHMARK_ARTIFACTS_PATH }}/ci-report.md" >> $GITHUB_STEP_SUMMARY
260+
261+
- name: Comment Performance Results on PR
262+
if: github.event_name == 'pull_request' && steps.analyze_results.outputs.report_generated == 'true'
263+
uses: actions/github-script@v7
264+
with:
265+
script: |
266+
const fs = require('fs');
267+
const reportPath = '${{ env.BENCHMARK_ARTIFACTS_PATH }}/ci-report.md';
268+
269+
if (fs.existsSync(reportPath)) {
270+
const report = fs.readFileSync(reportPath, 'utf8');
271+
272+
await github.rest.issues.createComment({
273+
issue_number: context.issue.number,
274+
owner: context.repo.owner,
275+
repo: context.repo.repo,
276+
body: report
277+
});
278+
}
279+
280+
- name: Fail build on critical performance regression
281+
if: steps.analyze_results.outputs.analysis_exit_code == '2'
282+
run: |
283+
echo "❌ Critical performance regression detected!"
284+
echo "Review the performance report and consider optimizations."
285+
echo "To proceed anyway, update the performance baseline manually."
286+
exit 1
287+
288+
- name: Warning on performance regression
289+
if: steps.analyze_results.outputs.analysis_exit_code == '1'
290+
run: |
291+
echo "⚠️ Performance warnings detected!"
292+
echo "Review the performance report for details."
293+
echo "Build will continue but monitor these benchmarks in future changes."
294+
295+
# Separate job for performance comparison across different environments
296+
cross-platform-benchmarks:
297+
name: Cross-Platform Performance Comparison
298+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
299+
strategy:
300+
matrix:
301+
os: [ubuntu-latest, windows-latest, macos-latest]
302+
runs-on: ${{ matrix.os }}
303+
timeout-minutes: 45
304+
305+
steps:
306+
- name: Checkout code
307+
uses: actions/checkout@v4
308+
309+
- name: Setup .NET
310+
uses: actions/setup-dotnet@v4
311+
with:
312+
dotnet-version: ${{ env.DOTNET_VERSION }}
313+
314+
- name: Run Core Benchmarks
315+
run: |
316+
dotnet run --project benchmarks/Honua.Benchmarks --configuration Release -- \
317+
--filter "*SqlGeneration*" \
318+
--job short \
319+
--exporters json \
320+
--artifacts "cross-platform-${{ matrix.os }}"
321+
322+
- name: Upload Cross-Platform Results
323+
uses: actions/upload-artifact@v4
324+
with:
325+
name: cross-platform-results-${{ matrix.os }}
326+
path: cross-platform-${{ matrix.os }}
327+
retention-days: 7

0 commit comments

Comments
 (0)