test: add testcases for autoqa

[Minh141120]

This pull request introduces new automated test prompts for verifying various features of the Jan application. These tests are focused on ensuring the presence and correctness of specific UI elements and functionality in the application. Below is a summary of the most important changes, grouped by the type of functionality being tested.

Tests for Settings Panel Verification:

• Added a test to verify that all expected model providers are listed in the Model Providers section of the Settings panel.
• Added a test to validate that hardware information is displayed correctly in the Hardware section of the Settings panel, including details about the operating system, CPU, memory, and GPUs.
• Added a test to ensure that enabling Experimental Features reveals the MCP Servers section in the Settings sidebar and that its content displays correctly.

Tests for Assistant and Extensions:

• Added a test to confirm that a default assistant named Jan is present on the Assistants screen, with a description starting with "Jan is a helpful desktop assistant...".
• Added a test to verify that the Extensions section in Settings lists four specific extensions: Jan Assistant, Conversational, Download Manager, and llama.cpp Inference Engine.

---

Important

Add test cases for Jan application to verify UI elements and functionality, including settings, assistant presence, hardware info, experimental features, and extensions.

• Tests for Settings Panel Verification:
  • Verify model providers listed in Model Providers section.
  • Validate hardware information in Hardware section, including OS, CPU, memory, and GPUs.
  • Ensure enabling Experimental Features reveals MCP Servers section.
• Tests for Assistant and Extensions:
  • Confirm default assistant Jan is present with correct description.
  • Verify Extensions section lists Jan Assistant, Conversational, Download Manager, and llama.cpp Inference Engine.

^{This description was created by for e070b48. You can customize this summary. It will automatically update as commits are pushed.}

[ellipsis-dev]

Caution

Changes requested ❌

Reviewed everything up to f9d025b in 1 minute and 40 seconds. Click for details.

• Reviewed 173 lines of code in 5 files
• Skipped 0 files when reviewing.
• Skipped posting 2 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. autoqa/tests/new-user/4-hardware-info.txt:45

• Draft comment:
  Use plain ASCII quotes instead of curly quotes in the GPU message. Replace “No GPUs detected” with "No GPUs detected".
• Reason this comment was not posted:
  Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% The comment appears to be completely incorrect - it's suggesting to replace quotes with identical quotes. The file already uses plain ASCII quotes. This seems like a false positive from an automated tool that got confused. The comment would make no changes to the code. Could there be some invisible Unicode characters that aren't showing up in the diff view? Could the quotes look plain but actually be special characters? Even if there were invisible characters, the suggestion shows exactly the same text as what's already there, so the comment would not fix any issues. The comment should be deleted as it suggests making a change that would result in identical text, and appears to be a false positive.

2. autoqa/tests/new-user/4-hardware-info.txt:45

• Draft comment:
  Typographical error: Replace the curly quotation marks around “No GPUs detected” with plain ASCII double quotes ("No GPUs detected") in order to adhere to the plain ASCII character requirement.
• Reason this comment was not posted:
  Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% The comment is suggesting a fix for a problem that doesn't exist - the code already uses plain ASCII quotes. The comment is therefore incorrect and misleading. Additionally, even if there were curly quotes, this would be a very minor issue that doesn't affect functionality. Could I be missing some hidden Unicode characters that aren't visible in the plain text view? Could there be a reason the automated tool detected curly quotes that I can't see? Even if there were hidden characters, the file is clearly readable and the quotes appear correct in the plain text view. The automated tool seems to have made a false positive detection. The comment should be deleted because it's incorrect - the code already uses plain ASCII quotes, and the comment is suggesting a fix for a non-existent problem.

Workflow ID: wflow_mNxEpSKZVD2tYxB0

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}

[ellipsis-dev]

Replace the non-ASCII arrow in 'Settings → Extensions' with an ASCII equivalent like 'Settings -> Extensions'.

Suggested change

	You are going to test the Jan application by verifying the available extensions listed under Settings → Extensions.
	You are going to test the Jan application by verifying the available extensions listed under Settings -> Extensions.

[ellipsis-dev]

Important

Looks good to me! 👍

Reviewed e070b48 in 1 minute and 9 seconds. Click for details.

• Reviewed 63 lines of code in 5 files
• Skipped 0 files when reviewing.
• Skipped posting 5 draft comments. View those below.
• Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.

1. autoqa/tests/new-user/2-providers-available.txt:5

• Draft comment:
  Good addition: the step to dismiss the 'Help Us Improve Jan' dialog ensures the UI is unobstructed for testing. Ensure this step gracefully handles the case where the dialog does not appear.
• Reason this comment was not posted:
  Comment looked like it was already resolved.

2. autoqa/tests/new-user/3-default-jan-assistant.txt:5

• Draft comment:
  Consistent inclusion: dismissing the 'Help Us Improve Jan' dialog before verifying the assistant ensures a clear test state.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50% This comment seems to be purely informative, as it doesn't provide a specific suggestion or request for action. It doesn't align with the rules for good comments, which should either suggest a code change, ask for confirmation on a specific point, or highlight a potential issue.

3. autoqa/tests/new-user/4-hardware-info.txt:5

• Draft comment:
  The added step to dismiss the help dialog helps ensure hardware info is visible without interference. Confirm that the test handles the absence of the dialog.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50% The comment is asking the PR author to confirm that the test handles a specific case, which violates the rule against asking for confirmation or ensuring behavior. It does not provide a specific code suggestion or ask for a specific test to be written.

4. autoqa/tests/new-user/5-enable-mcp-server.txt:5

• Draft comment:
  Including the dialog dismissal step is good to avoid UI interference when enabling Experimental Features. Ensure this does not impact subsequent interactions.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50% The comment is asking the PR author to ensure that the change does not impact subsequent interactions, which violates the rule against asking for confirmation or ensuring behavior. It does not provide a specific suggestion or point out a specific issue.

5. autoqa/tests/new-user/6-extensions.txt:5

• Draft comment:
  The initial step to dismiss the 'Help Us Improve Jan' popup is a sensible addition, ensuring that extension verification isn’t blocked by unexpected dialogs.
• Reason this comment was not posted:
  Comment did not seem useful. Confidence is useful = 0% <= threshold 50% This comment is purely informative, explaining why a change is sensible. It doesn't provide a suggestion, ask for confirmation, or point out a potential issue.

Workflow ID: wflow_z9wtQn78FFkZX7KH

^{You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.}