-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
370 add the data quality report to the csv as a separate sheet #372
370 add the data quality report to the csv as a separate sheet #372
Conversation
…s-a-separate-sheet Additional changes: - Refactor CSV handling (necessary due to circular imports): reorganize csv-related functions into a dedicated csv module and update imports
Note to self: Due to a circular import when importing project.npda.general_functions.csv_upload, I have:
|
…s-a-separate-sheet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looking very nice @reecehill
Seen on STAGING (created by @reecehill and merged by @eatyourpeas 7 minutes and 41 seconds ago) Please check your changes! |
This pull request includes significant refactoring and enhancements to the CSV handling functionality in the
project/npda/general_functions
module. The changes primarily focus on organizing the CSV-related functions into a dedicated subdirectory, adding new functionalities, and improving existing ones.Key changes include:
Refactoring and Reorganization:
project/npda/general_functions/csv
and updated import paths accordingly. [1] [2] [3] [4] [5] [6]New Functionality:
download_file
to handle file downloads and refactoreddownload_csv
to use this function. Introduced a newdownload_xlsx
function for downloading XLSX files. [1] [2]Enhancements:
csv_parse
function to parse CSV files into pandas DataFrames, handling various edge cases and data type conversions.write_errors_to_xlsx
function to generate an XLSX file highlighting validation errors in the CSV data.Bug Fixes:
serialize_error
function to ensure keys are correctly serialized as integers.Additional Improvements: