Hi folks,
I'm currently working on a script that would automatically generate CSV
files for my staff to upload into Serials Solutions to update our local
holdings. There is one part of the script that seems to keep failing. The
main script [1] currently creates one big CSV file which according to my
staff will create problems with the uploading process in Serials Solutions.
The current path I am on is to split this big CSV file into smaller files
for easier uploading. I found a small csv splitter script and modified it
to include quoting=csv.QUOTE_MINIMAL in the csv.writer call. [2] Created a
module from said code. So far, so good.
When I call the csv_splitter.split in my main script, the function splits
the main CSV file into smaller files (default 10000 rows per file);
however, the last split file is missing at least 15-25 rows. I tested
different CSV files with the function and have seen the same issue of the
split file ending mid-row. I've uploaded the main CSV file and the last
output file from the function [3] - you will see that in line 9987 of the
output_5.csv file ends in the middle of the title. Again, I tested
different CSV files with this script and it consistently leaves out the
last few rows from the master CSV file.
Am I missing something in the csv_splitter code that would cause an abrupt
end of writing rows to the output files?
[1] Testing environment: Ubuntu 14.04.2 LTS, Python 2.7 -
https://gist.github.com/yo-bj/7224738839ff73837da6
[2] https://gist.github.com/yo-bj/dea88123600d9a74b8ec
[3]
https://www.dropbox.com/sh/su81bcw7aiajepw/AAC_GjzAH7s1BbUwkF-Kf93Ha?dl=0
Thanks,
Becky
-------------------------------------------------------
Becky Yoose
Discovery and Integrated Systems Librarian
Grinnell College
|