 |
 |
Hi, Kyle – I would echo the sentiment to put a copy in the dark cloud (e.g. Amazon Glacier). Last time I checked Oracle had the cheapest dark cloud. Go with whatever is cheap and have a plan to pull out when pricing changes. There is a dizzying array of local storage options. For data this size many will steer you towards "enterprise" storage solutions, such as a SAN. For preservation/archiving alone, this is overkill. What might work well is a computer with a ton of hard drives in it. I recommend using consumer level hard drives since they are cheap and big. Like all hard drives, they are unreliable too. Plan for some drives failing. 45drives.com uses the design that the cloud hosting company Backblaze puts out. Unlike most enterprise level products, you can see pricing without asking for a quote. http://www.45drives.com/products/storage/order.php?id=S45-lite&config=01&model=S45-lite-01&code=Slite&software=Default&type=storage 200T of drives would be about 10k, less if you use fewer/bigger drives. Add in ~5k for the machine itself. It may make sense to take a hybrid approach. You could store locally and put a copy in the dark cloud for safekeeping. If you are only going to have one copy, a reputable cloud storage provider isn't a bad choice. More than one copy is better. The best path forward depends on what your IT situation looks like. What existing resources, if any, can be leveraged? (IT staff? server room? etc) Please report back with what you end up going with. Tom On Thu, Mar 1, 2018 at 2:36 PM, Kyle Banerjee <[log in to unmask]> wrote: > On Thu, Mar 1, 2018 at 11:11 AM, Kyle Breneman <[log in to unmask]> > wrote: > >> 1. A combination of both. Around 60TB already digitized, with over 100TB >> more footage still in analog form. >> 2. Portable hard drives, a RAID, a Mac Pro tower. >> 3. No, we do not have a digitization workflow defined yet. >> 4. Define "quick access." I think that we do want "quick access" unless >> we also have something like a local NAS for immediate access to the files >> and then AWS strictly as secondary backup copies. >> > > Unless you move your computing to Amazon as well, I wouldn't think quick > access would be viable. First of all, S3 is slow and works very differently > than filesystems people are accustomed to interacting with -- I'd never use > it for video processing. Even if that weren't a factor, the network > latency and high bandwidth charges would be deal busters. Even just storing > the stuff in S3 so it can quickly be accessed is going to be high -- 200TB > is going to run you over 5 grand a month. > > I'd be much more tempted to use Amazon for preservation only. I would guess > the Snowball devices and service fees to get your data up there and sit in > an S3 bucket for one day while it gets transitioned to Glacier to run about > a grand, with monthly charges after that running a little over $800. In > case of disaster, you can recover the videos affordably enough (a few > grand) via Snowball. > > For ongoing workflow, there would be the question of how to get material up > there. Uploading huge files doesn't work well so you'd probably want to > ship drives or use Snowball which will be significant. > > You could do backup yourself as Chris describes, but don't underestimate > labor, facilities, verifying backups, bandwidth/logistics of getting the > info to multiple places, etc. Glacier will protect the integrity of your > data which provides much more protection than simply backing up to tape or > drive and trusting all will be good. > > kyle