 |
 |
Hi folks, I'm currently working on a script that would automatically generate CSV files for my staff to upload into Serials Solutions to update our local holdings. There is one part of the script that seems to keep failing. The main script [1] currently creates one big CSV file which according to my staff will create problems with the uploading process in Serials Solutions. The current path I am on is to split this big CSV file into smaller files for easier uploading. I found a small csv splitter script and modified it to include quoting=csv.QUOTE_MINIMAL in the csv.writer call. [2] Created a module from said code. So far, so good. When I call the csv_splitter.split in my main script, the function splits the main CSV file into smaller files (default 10000 rows per file); however, the last split file is missing at least 15-25 rows. I tested different CSV files with the function and have seen the same issue of the split file ending mid-row. I've uploaded the main CSV file and the last output file from the function [3] - you will see that in line 9987 of the output_5.csv file ends in the middle of the title. Again, I tested different CSV files with this script and it consistently leaves out the last few rows from the master CSV file. Am I missing something in the csv_splitter code that would cause an abrupt end of writing rows to the output files? [1] Testing environment: Ubuntu 14.04.2 LTS, Python 2.7 - https://gist.github.com/yo-bj/7224738839ff73837da6 [2] https://gist.github.com/yo-bj/dea88123600d9a74b8ec [3] https://www.dropbox.com/sh/su81bcw7aiajepw/AAC_GjzAH7s1BbUwkF-Kf93Ha?dl=0 Thanks, Becky ------------------------------------------------------- Becky Yoose Discovery and Integrated Systems Librarian Grinnell College