-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Improve write-tests-agent with best practices #33860
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Improve write-tests-agent with best practices #33860
Conversation
Add comprehensive reinforcement guidelines to prevent common mistakes: 1. Prefer C# Over XAML - Default to .cs files for most UI tests - Only use XAML for binding/template/style/resource tests - Includes clear examples and decision criteria 2. Use Test Helper Base Classes - Document TestShell, TestContentPage, TestNavigationPage - Show helper methods and when to use each - Prevent creating Shell tests from scratch 3. Avoid Obsolete APIs - Map obsolete → modern APIs (Application.MainPage → Window.Page) - Cover Frame → Border deprecation - Include modern threading API guidance 4. Use UITest Optimized Controls - Always use UITestEntry, UITestEditor, UITestSearchBar - Controls provide IsCursorVisible to prevent flaky screenshots - Cursor blinking causes visual test failures - Includes examples and location of controls 5. Check Similar Tests for Patterns - Add bash commands to search for similar tests - Encourage reusing established patterns - Help agent learn from existing examples These additions address real scenarios encountered during test creation and will help the agent make better decisions automatically.
78a224b to
e9cd1d9
Compare
PureWeen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review: Content is Valuable, but Wrong Location
Thanks for contributing these best practices! The guidance is practical and addresses real issues we've seen. However, the content should be moved to a different file.
The Content is Good ✅
| Section | Value |
|---|---|
| 1. Prefer C# Over XAML | ✅ Reduces unnecessary XAML complexity |
| 2. Use Test Helper Base Classes | ✅ Documents TestShell, TestContentPage - not well documented elsewhere |
| 3. Avoid Obsolete APIs | ✅ Prevents compiler warnings |
| 4. Use UITest Optimized Controls | ✅ Prevents flaky tests from cursor blinking |
| 5. Check Similar Tests | ✅ Encourages pattern reuse |
The Location is Wrong ❌
Current location: .github/agents/write-tests-agent.md
Problem: This is an agent file, not a guidance file. Agents should be lightweight dispatchers that invoke skills.
| File Type | Purpose | Size Guideline |
|---|---|---|
Agent (.github/agents/*.md) |
Dispatch to skills, minimal logic | <100 lines ideal |
Skill (.github/skills/*/SKILL.md) |
Detailed workflow, procedures | <500 lines |
Instructions (.github/instructions/*.md) |
Reference guidance, auto-loaded by glob | Unlimited |
The agent file is now 322 lines - way over the ideal size for an agent (~50-100 lines).
Recommended Fix
Move all 5 sections to .github/instructions/uitests.instructions.md
This is the correct location because:
- Auto-loaded for relevant files - Has
applyTo: "src/Controls/tests/TestCases.Shared.Tests/**,src/Controls/tests/TestCases.HostApp/**" - Reference material - Best practices are reference guidance, not workflow
- Already exists - This file is the canonical place for UI test guidance
- Agent should stay thin - write-tests-agent should just dispatch to skills
Suggested Structure in uitests.instructions.md
Add a new section near the end:
## Best Practices
### Prefer C# Over XAML
[Your content here]
### Use Test Helper Base Classes
[Your content here]
### Avoid Obsolete APIs
[Your content here]
### Use UITest Optimized Controls
[Your content here]
### Check Similar Tests for Patterns
[Your content here]Summary
- ✅ Content: Valuable, keep it
- ❌ Location: Move to
uitests.instructions.md - ❌ Agent size: Revert agent to its original thin state
Please move the content and I'll be happy to approve. Thanks for the contribution!
Address review feedback from PR dotnet#33860: - Move all 5 best practice sections to uitests.instructions.md - Reduce agent file from 322 lines to 100 lines - Agent now references instructions file for best practices - Follows proper separation of concerns (agent=dispatch, instructions=guidance) Benefits: - Instructions auto-load when working on test files (applyTo pattern) - Best practices always accessible, not just when agent invoked - Agent remains thin dispatcher as intended - Follows established architectural patterns in repository Content preserved: 1. Prefer C# Over XAML 2. Use Test Helper Base Classes (TestShell, TestContentPage, etc.) 3. Avoid Obsolete APIs (Application.MainPage → Window.Page) 4. Use UITest Optimized Controls (UITestEntry, UITestEditor, UITestSearchBar) 5. Check Similar Tests for Patterns All examples, tables, and bash commands intact - just moved to correct location.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds comprehensive best practices guidance to the write-tests-agent to help it avoid common mistakes when creating UI tests. The changes provide clear decision criteria, examples, and search commands to help the agent make better architectural decisions automatically.
Changes:
- Added "Prefer C# Over XAML" guideline with clear decision criteria and examples
- Documented test helper base classes (TestShell, TestContentPage, TestNavigationPage) with usage examples
- Mapped obsolete APIs to modern equivalents (Application.MainPage → Window.Page, Frame → Border, Device.BeginInvokeOnMainThread → modern threading APIs)
- Added UITest optimized controls guidance (UITestEntry, UITestEditor, UITestSearchBar) to prevent flaky screenshot tests
- Included bash commands to search for similar tests and reuse established patterns
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
.github/instructions/uitests.instructions.md |
Added comprehensive Best Practices section with 5 subsections covering C# vs XAML preference, test helper base classes, obsolete API avoidance, UITest optimized controls, and pattern reuse guidance |
.github/agents/write-tests-agent.md |
Added reference to the new best practices section in the uitests.instructions.md file with summary of key topics |

Add comprehensive reinforcement guidelines to prevent common mistakes:
Prefer C# Over XAML
Use Test Helper Base Classes
Avoid Obsolete APIs
Use UITest Optimized Controls
Check Similar Tests for Patterns
These additions address real scenarios encountered during test creation and will help the agent make better decisions automatically.