Internet Archive keeps some samples uploaded for training on Archive-It:
https://archive.org/details/sample-warc-file
Webrecorder also maintains some examples in the pywb repo:
https://github.com/webrecorder/pywb
And Common Crawl links out to a raw example that can be easily repackaged (I just tried it myself):
https://commoncrawl.org/blog/web-archiving-file-formats-explained
I think that in general many of these examples are going for size efficiency rather than being comprehensive examples of what WARC can and can't do. They do illustrate the basic structure well enough.
If you're really looking to play, I have two suggestions for simply and easily making your own WARCs:
1.
If you have access to Wget<https://www.gnu.org/software/wget/>, it's had support for making WARCs since 1.14.
Resources:
https://wiki.archiveteam.org/index.php/Wget_with_WARC_output
https://salivity.github.io/wget/article/how-to-save-wget-downloads-to-a-warc-file
1.
For a simpler option with less technical overhead, you can use ArchiveWeb.page<https://webrecorder.net/archivewebpage/> which also supports the newer WACZ format.
I'd grab some simple websites at first - definitely don't go for anything that is hosting a lot of PDFs. This is surprisingly common and will increase the time to create your file by quite a bit.
Sincerely,
Aurora Charlow
[Ohio University Logo]
Aurora Charlow, she/they
DIGITAL ARCHIVIST
UNIVERSITY LIBRARIES
MAHN CENTER FOR ARCHIVES AND SPECIAL COLLECTIONS, PRESERVATION & DIGITAL INITIATIVES
EMAIL
[log in to unmask]<mailto:[log in to unmask]>
PHONE
740.593.0055
________________________________
From: Code for Libraries <[log in to unmask]> on behalf of Eric Lease Morgan <[log in to unmask]>
Sent: Thursday, June 18, 2026 12:11 PM
To: [log in to unmask] <[log in to unmask]>
Subject: Re: [CODE4LIB] [External] [CODE4LIB] warc files
On Jun 18, 2026, at 10:34 AM, Charlow, Aurora <[log in to unmask]> wrote:
> I'd look at this repo as a first stop: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fiipc%2Fawesome-web-archiving%23tools--software&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564341986%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WVqi6weUkxUUFjboclSLPE4AstQtcm8HXnbamDDM16I%3D&reserved=0<https://github.com/iipc/awesome-web-archiving#tools--software>
On Jun 18, 2026, at 11:22 AM, Ed Summers <[log in to unmask]> wrote:
> Down at the bottom of the Awesome page that Aurora mentioned is a Public Data section, which includes probably the largest publicly available pool of WARC data: CommonCrawl:
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata.commoncrawl.org%2F&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564367707%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=%2BRwelB9gVwyw9JBXL1zoUmSSfvfONYJBLPEmNd82sCc%3D&reserved=0<https://data.commoncrawl.org/>
>
> The other one is the End of Term Archive:
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Feotarchive.org%2F&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564379823%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=ZcO%2FZOS3ETf2aaESEmw8xxIRVbaTBBp7P1x%2BX5a2Lis%3D&reserved=0<https://eotarchive.org/>
>
> Both projects hast their WARC data on Amazon S3.
>
> The Internet Archive is probably sitting on the largest pile of WARC data, but they generally don’t make it available to the public.
>
> You’ll know you’ve caught the web archiving bug when you start creating your own WARC data using some of the tools on that Awesome list. Browsertrix [1] is a fun one to try: open source and available as a service. ArchiveBox [2] is another open source project for archiving the web that will write WARC files. Probably one of the most CLI friendly way to create WARC data is using wget to crawl some region of the web. Or you can try Harvard’s scoop tool. There are a lot of fun things to try!
>
> //Ed
>
> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.browsertrix.com%2F&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564391329%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=zLMoH%2BpcI6wP1h0cwuG8BnFYuxzonwbmM7syX3O5Ts0%3D&reserved=0<https://docs.browsertrix.com/>
> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchivebox.io%2F&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564404826%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=j7aJk7mvvjfGKSBw%2FPyrCCLozGYUnR5iFrsQeLlIwZg%3D&reserved=0<https://archivebox.io/>
> [3] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Fsoftware%2Fwget%2Fmanual%2F&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564418799%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Mr9Z%2BE6H3Np1bFfznIU8%2FoG1siVeV9HaJBWqYaCrA68%3D&reserved=0<https://www.gnu.org/software/wget/manual/>
> [4] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fharvard-lil%2Fscoop%23readme&data=05%7C02%7Ccharlowa%40OHIO.EDU%7C6896bcef3480441a923c08decd546881%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C639173959564432099%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=RrnS9b85Qe42Cf3E24k0C0O9lNRH9za7P4UOnlrVzGw%3D&reserved=0<https://github.com/harvard-lil/scoop#readme>
Thank you for the prompt replies, but I still don't see any URLs pointing to warc files, sans warc files terabytes or petabyte in size. There are many descriptions of sites that have been converted into warc files, but no access to the warc files themselves. I'd like to play with a dozen or so warc files, not of my own design, that are in the few gigabyte size range. What am I missing? --E "Still Embarrassed" M
|