Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate CSV files pre-flight #2244

Open
terrazoon opened this issue Jan 3, 2025 · 1 comment
Open

Validate CSV files pre-flight #2244

terrazoon opened this issue Jan 3, 2025 · 1 comment
Assignees

Comments

@terrazoon
Copy link
Contributor

terrazoon commented Jan 3, 2025

If the header column is not on run one, delete blank lines until it is.

If the number of columns in the rows don't match the number of columns in the header, reject the file.

If there is a BOM present in a csv file, remove it before uploading to S3. Use lstrip('\ufeff')

@terrazoon terrazoon converted this from a draft issue Jan 3, 2025
@terrazoon terrazoon moved this from ⬇ Up-Next to 🏗 In progress (WIP: ≤ 3 per person) in Notify.gov product board Jan 8, 2025
@terrazoon terrazoon moved this from 🏗 In progress (WIP: ≤ 3 per person) to 👀 In review in Notify.gov product board Jan 8, 2025
@terrazoon terrazoon moved this from 👀 In review to ✅ Done in Notify.gov product board Jan 8, 2025
@terrazoon terrazoon moved this from ✅ Done to ⬇ Up-Next in Notify.gov product board Jan 8, 2025
@terrazoon terrazoon changed the title Strip BOMs before uploading to S3 Validate CSV files pre-flight Jan 14, 2025
@terrazoon
Copy link
Contributor Author

There is a report that a user was able to upload a csv file that contained strings instead of phone numbers and that this file was accepted and we tried to process it, resulting all the messages failing.

I tried to repro with a simple CSV file with the same problem, but when I tried to upload it was rejected in the UI with the message "You need to fix 1 phone number".

We need the service_id and job_id to investigate further, so we can download the user's csv file.

@terrazoon terrazoon moved this from ⬇ Up-Next to ⏱ Blocked/Waiting in Notify.gov product board Jan 16, 2025
@terrazoon terrazoon self-assigned this Jan 16, 2025
@terrazoon terrazoon moved this from ⏱ Blocked/Waiting to 🏗 In progress (WIP: ≤ 3 per person) in Notify.gov product board Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏗 In progress (WIP: ≤ 3 per person)
Development

No branches or pull requests

1 participant