feat: add submitterEmail column to form response for efficient duplicate checking#27303
feat: add submitterEmail column to form response for efficient duplicate checking#27303joeauyeung wants to merge 11 commits intomainfrom
Conversation
…ate checking - Add submitterEmail column to App_RoutingForms_FormResponse and App_RoutingForms_QueuedFormResponse models - Add index on submitterEmail for efficient querying - Update RoutingFormResponseRepository to save submitterEmail when creating form responses - Add findResponseWithBookingByEmail method to query by email directly - Update hasDuplicateSubmission to use email-based query instead of iterating through all responses - Extract and save submitter email in handleResponse when recording form responses This improves performance by querying directly on the indexed email column instead of fetching all form responses within a time window and iterating through them to find matching emails. Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
…ter query performance Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
…ions Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
graphite-app
bot
added
enterprise
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
E2E results are ready! |
Udit-takkar
left a comment
Udit-takkar
left a comment
There was a problem hiding this comment.
@joeauyeung What is apps/web/app/api/routing-forms/queued-response/route.ts path?
That endpoint doesn't set submitterEmail when calling recordFormResponse
Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
| response, | ||
| // We use the queuedFormResponse's chosenRouteId because that is what decided routed team members | ||
| chosenRouteId: queuedFormResponse.chosenRouteId, | ||
| submitterEmail, |
There was a problem hiding this comment.
@Udit-takkar adding the submitter email here as well
Udit-takkar
left a comment
Udit-takkar
left a comment
There was a problem hiding this comment.
LGTM.
I'll wait for Hariom's approval
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
hariombalhara
left a comment
hariombalhara
left a comment
There was a problem hiding this comment.
Devin has valid concern around existing Form Responses without submitterEmail. We need to tackle that.
…uery Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
Co-Authored-By: joe@cal.com <j.auyeung419@gmail.com>
Devin AI is resolving merge conflictsThis PR has merge conflicts with the Devin will:
If you prefer to resolve conflicts manually, you can close the Devin session and handle it yourself. |
…-response-submitter-email
Devin AI is resolving merge conflictsThis PR has merge conflicts with the Devin will:
If you prefer to resolve conflicts manually, you can close the Devin session and handle it yourself. |
…-response-submitter-email Co-Authored-By: unknown <>
3 participants
What does this PR do?
Optimizes the form submitted no event webhook trigger by storing the submitter email directly in the form response record, enabling efficient database queries instead of iterating through all responses.
Before: When checking for duplicate form submissions, the code fetched all form responses within a 60-minute window and iterated through each response's JSON to find matching emails.
After: The submitter email is extracted and stored in a dedicated indexed column when the form is submitted, allowing direct database queries by email.
Changes
submitterEmailcolumn toApp_RoutingForms_FormResponseandApp_RoutingForms_QueuedFormResponsemodels(formId, submitterEmail, createdAt)for optimal query performancefindResponseWithBookingByEmailrepository method for direct email-based querieshandleResponse.tsto extract and save submitter email when recording form responseshasDuplicateSubmissionto use the new email-based queryUpdates since last revision
getSubmitterEmailfunction fromformSubmissionValidation.tsinstead of having a duplicateextractSubmitterEmailfunction inhandleResponse.tsgetSubmitterEmailaccept a more genericResponseWithValuetype to work with both the webhook response format and the handleResponse formatsubmitterEmailparameter in repository method callssubmitterEmailto composite index on(formId, submitterEmail, createdAt)for better query performance since queries always filter by formId firstgetSubmitterEmailandgetSubmitterNamefunctions covering edge cases (empty responses, numeric values, arrays, multiple emails, different response formats)submitterEmailextraction toapps/web/app/api/routing-forms/queued-response/route.tswhich was missing the new parameter when callingrecordFormResponsesubmitterEmailin lowercase and remove case-insensitive query mode for better index utilization (addresses reviewer feedback)Mandatory Tasks (DO NOT REMOVE)
How should this be tested?
submitterEmailcolumn is populated in the database in lowercaseHuman Review Checklist
getSubmitterEmailfunction finds the first field value containing "@". Verify this matches the expected behavior for identifying submitter emails (note: could match non-email fields like Twitter handles).ResponseWithValuetype uses[key: string]: unknownto allow flexibility - verify this doesn't cause issues with type safety.submitterEmailas NULL.mode: "insensitive"). Verify this is acceptable for all use cases.(formId, submitterEmail, createdAt)is optimal for the query pattern (equality on formId, equality on email, range on createdAt).Checklist
Link to Devin run: https://app.devin.ai/sessions/b837bc0ab2e647eb9374765432bebcef
Requested by: @joeauyeung