 |
 |
You are likely running into an AFS limitation. Why do I keep getting [Errno 27] File too large? The cause of this error is that AFS has a limit on the total length of the names of all files in a directory. If the file names are short, you can write approximately 64,000 files to an AFS directory. If the file names are long, you can hit the directory limit much sooner. https://sites.google.com/a/stanford.edu/rcpedia/how-do-i/why-do-i-keep-getti ng-errno-27-file-too-large --tr -----Original Message----- From: Code for Libraries <[log in to unmask]> On Behalf Of Eric Lease Morgan Sent: Thursday, June 16, 2022 11:55 AM To: [log in to unmask] Subject: [CODE4LIB] file too large I need help uncompressing a tar file. 8-D I have a 67 GB tar file. I believe it contains about 700,000 files equally distributed between two subdirectories. When I try to uncompress the tar file, I eventually get a repeated "file too large" error looking like this: tar: document_parses/pdf_json/40afe13d64d6a5ef6640537e9f2334c0d86dfa88.json: Cannot open: File too large tar uncompresses many of the files, but not all of them. It seems to get stuck on the file names that are rather long. In the subdirectories, where the files are being stored, the number of files is above 28,000. The files are being saved on an AFS file system. I can't believe the file names are too long since all of the files have same length of file name. Nor, is my file system (quota) full. What is going on here? What file is too large? -- Eric Morgan University of Notre Dame