Skip to content

Allow for user feedback #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 7, 2024
Merged

Allow for user feedback #15

merged 5 commits into from
Nov 7, 2024

Conversation

ajbozarth
Copy link
Collaborator

@ajbozarth ajbozarth commented Oct 30, 2024

This adds a UI for users to provide feedback on the Qiskit Code Assistant as well as specific prompts.

Included features

  • Feedback status bar icon: When clicked opens a feedback dialog for general feedback on the extension or model. The text field is immediately focused and and supports multiline input.
    Screenshot 2024-10-30 at 5 14 43 PM
    Screenshot 2024-10-31 at 8 35 53 PM
  • Prompt specific feedback: Once a prompt response has been received a feedback button is added to the Notebook cell toolbar. When clicked a feedback dialog connected to the most recently received prompt response is opened. The prompt response does not necessarily need to have been in the current cell, or even file.
    Screenshot 2024-10-30 at 5 14 54 PM
    Screenshot 2024-10-30 at 5 15 21 PM
    Screenshot 2024-10-31 at 8 35 46 PM
  • After feedback is successfully sent to the service a toast notification is triggered
    Screenshot 2024-10-30 at 5 18 38 PM

Current Limitations

  1. The prompt feedback button is only available in notebooks and not the text editor.
    • This is due to not having a good location to put the button
    • Workaround: Due to the global context noted in limitation 2, users can switch to a notebook after receiving a prompt in the text editor and submit review via the feedback button.
  2. The prompt feedback button is not tied to the current cell specifically and is just tied to the most recent prompt response returned to the current instance of jupyterlab.
    • This is a solvable limitation, but would require an excessive amount of code to address (would at least more than triple the amount of code added in this PR)
    • Note: This scope is still limited to the users current browser tab and not across all users connected to the current Jupyter lab instance
  3. Thumbsup/thumbsdown buttons are not surfaced in the cell toolbar but will be in the dialog form.
    • Adding these would be straight forward, but with the above limitation related to scope and context I believe it would be very confusing to the user what they're responding to.
  4. If a user select a thumbsup/thumbsdown they can't de-select it
    • This is due to using a radio input for the form field, the field also still show the radio buttons to see which is selected since browsers convert unicode to emojis, which don't have a good "select" visual

Open Questions

  1. Is the issue of scope and context in limitation 2 a deal breaker or can well move forward with the prompt feedback button have a "global" context?
  2. Is my judgment on issue 3 valid or do we still want to implement it with global context?

This includes a initial working version of the feedback UX
@ajbozarth ajbozarth added the enhancement New feature or request label Oct 30, 2024
@ajbozarth ajbozarth self-assigned this Oct 30, 2024
@ajbozarth
Copy link
Collaborator Author

@cbjuan @vabarbosa This is just an initial draft, but outside of fixing the dialog UI it's fully featured and complete. If you could take a look, I included a detailed description and called out specific questions I'd like answers to.

@ajbozarth
Copy link
Collaborator Author

  1. The prompt feedback button is not tied to the current cell specifically and is just tied to the most recent prompt response returned to the current instance of jupyterlab.

After a worrying thought entered my head I tested how far the scope go on this and confirmed it is limited to a specific user instance of jupyterlab and not a server instance. Ie the scope is shared across every open notebook, but not between the same jupyterlab instance opened in another browser tab. Relatedly it does not persist on page reload.

@ajbozarth
Copy link
Collaborator Author

I have just pushed the final updates and updated the top description with new screen shots, descriptions, and limitations

@ajbozarth ajbozarth marked this pull request as ready for review November 1, 2024 01:51
@ajbozarth
Copy link
Collaborator Author

@cbjuan I just pushed the update to include the input/output values as now allowed by the service https://qiskit-code-assistant.quantum.ibm.com/docs#/Feedback/submit_feedback_feedback_post

Copy link
Member

@cbjuan cbjuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, so approving

I'd love to test it before publishing a new version with this. Also, could you take a look to this PR, @vabarbosa ?

@ajbozarth
Copy link
Collaborator Author

I'd love to test it before publishing a new version with this.

@cbjuan so do you want to test this PR by building it locally or do you mean we should merge and release an rc for testing via pip install wheel file?

@cbjuan
Copy link
Member

cbjuan commented Nov 7, 2024

I was thinking on testing it locally, but you're right, probably is cleaner to publish it as a rc and test it from there. Let's do it after @vabarbosa review

Copy link
Collaborator

@vabarbosa vabarbosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a couple minor comments, but nothing blocking

Copy link
Collaborator

@vabarbosa vabarbosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@ajbozarth ajbozarth merged commit a3b4a17 into main Nov 7, 2024
6 checks passed
@ajbozarth ajbozarth deleted the feedback branch November 7, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants