|
|
All the news now and in your digital future
OT: Who's using Blu-ray for archiving?
Posted by John K
|
Re: OT: Who's using Blu-ray for archiving? June 29, 2010 05:25PM |
I don't consider BluRay to be a feasible archival format, because the storage per disc is too small. I know of a house that used to archive to DVDs a long time ago, and frankly, when I saw the stack of DVDs... Let's just say that if you had to search through that pile to find a project, it's a daunting prospect. Although BluRay stores more than 5 times the data of a DVD, you need to consider that you're moving into a tapeless world, and more storage is required as a result. So it is still not practical.
If you can't afford an LTO, I'd say go on RAID 1s.
www.strypesinpost.com
If you can't afford an LTO, I'd say go on RAID 1s.
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? June 29, 2010 07:02PM |
Just a reminder: Nobody has the foggiest idea what the shelf life of a recordable Blu Ray disc is. For that matter, nobody really knows what the shelf life of DVD-R media is either. Testing by NIST and other groups has been all over the map, with accelerated aging studies coming up with numbers of anywhere from five to two hundred years.
I wouldn't trust optical media for any data I really, totally, definitely, for sure wanted to preserve indefinitely.
I wouldn't trust optical media for any data I really, totally, definitely, for sure wanted to preserve indefinitely.
|
Re: OT: Who's using Blu-ray for archiving? June 29, 2010 07:07PM |
>Since i do my shooting on Sony XDCAM EX at 35Mbs i use Blu-Ray BD-R 25 GB
About half a year ago, I may have agreed. Thing is, I know of a place that shoots largely on the Sony EX cameras, they opted for BluRay as an archival format, and after a few months of hectic production, they have some 3-4 TBs of footage sitting on hard drives, most of them XDCAM EX card archives. Although the data on each SxS card may fit onto a BluRay, it is going to take a long time to burn all that onto discs. If you do it religiously after every shoot, and you're able to make it work, more power to you.
www.strypesinpost.com
About half a year ago, I may have agreed. Thing is, I know of a place that shoots largely on the Sony EX cameras, they opted for BluRay as an archival format, and after a few months of hectic production, they have some 3-4 TBs of footage sitting on hard drives, most of them XDCAM EX card archives. Although the data on each SxS card may fit onto a BluRay, it is going to take a long time to burn all that onto discs. If you do it religiously after every shoot, and you're able to make it work, more power to you.
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? June 30, 2010 01:49PM |
Just to add to the points.
Optical media, and solid state drives are not considered archival formats, because in the case of optical media, storage per disc is too low. For solid state drives, the price for storage is too high. This is not taking into consideration whether these formats can preserve the integrity of your data for a substantial amount of time (eg. 20 years). Granted, even if the data is intact, you may not have the means to play them, as digital codecs may become obsolete.
Although the price of traditional hard drives may have fallen, they are simply not reliable enough, partly because they contain too many moving parts, data can become corrupt, and the hard drive may fail beyond a certain time span. We tend to assume the lifespan of a single hard drive to be between 3-5 years, and mainly because of this, whether it exists in a RAID or as a solitary drive, it cannot be considered an archival format.
An LTO deck may be pricier (too pricey for some of us), but if you have a substantial amount of data to archive (eg. >20 TBs), LTOs actually is the cheaper and more reliable option.
www.strypesinpost.com
Optical media, and solid state drives are not considered archival formats, because in the case of optical media, storage per disc is too low. For solid state drives, the price for storage is too high. This is not taking into consideration whether these formats can preserve the integrity of your data for a substantial amount of time (eg. 20 years). Granted, even if the data is intact, you may not have the means to play them, as digital codecs may become obsolete.
Although the price of traditional hard drives may have fallen, they are simply not reliable enough, partly because they contain too many moving parts, data can become corrupt, and the hard drive may fail beyond a certain time span. We tend to assume the lifespan of a single hard drive to be between 3-5 years, and mainly because of this, whether it exists in a RAID or as a solitary drive, it cannot be considered an archival format.
An LTO deck may be pricier (too pricey for some of us), but if you have a substantial amount of data to archive (eg. >20 TBs), LTOs actually is the cheaper and more reliable option.
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? June 30, 2010 02:05PM |
It is the nature of modern video archiving that a) nothing is for certain and b) materials need to be updated constantly. It's no longer as simple as outputting to a Beta or DigiBeta tape and that's it. And frankly, these days, we'd probably look back on the Betacam days as an extremely lossy, insufficient format for archiving.
I remember talking to an engineer about compression and he talked about how the major studios go crazy with trying to archive their classic films. The technology changes every three years and suddenly they have a whole catalogue of films that need to be remastered to the newest, best-looking format.
Digital technology changes rapidly, but formats still stick around. H.264, after all, is now seven years old. And MPEG-1 has been around for something like 22 years. The problem, of course, is that digital formats have a hard ceiling in terms of quality. You can rescan a good film print 40 years later and it will yield new visual information to take advantage of superior technology; an MPEG-1 will not look any better if you reconverted it to a 24K hyper-HD frame size -- and will in fact look a lot worse.
As for long-term endurance, the nature of any tapeless medium is that the archives are going to have to be checked, tested and possibly transferred regularly. A drive sitting on a shelf for six months is not guaranteed to survive. And none of these new file-based archival storage media has been around long enough for us to know whether they'll really last 20 years, or 100, or just two. Plus, digital information is more likely to degrade in ways that render the whole thing unsalvageable. If you get a part of an analog video cassette warped, demagnetized or dirty, you can rescue the rest. If a digital file is corrupted, very often that's it.
Imagine the headache of maintaining a large archive of P2, R3D, or AVCHD raw media...not only would the drives need to survive, but to ensure the files are still truly intact, you might even have to run them through the conversion process periodically just to make sure the file structure is still sound, that the video and audio are as they were, and so on.
www.derekmok.com
I remember talking to an engineer about compression and he talked about how the major studios go crazy with trying to archive their classic films. The technology changes every three years and suddenly they have a whole catalogue of films that need to be remastered to the newest, best-looking format.
Digital technology changes rapidly, but formats still stick around. H.264, after all, is now seven years old. And MPEG-1 has been around for something like 22 years. The problem, of course, is that digital formats have a hard ceiling in terms of quality. You can rescan a good film print 40 years later and it will yield new visual information to take advantage of superior technology; an MPEG-1 will not look any better if you reconverted it to a 24K hyper-HD frame size -- and will in fact look a lot worse.
As for long-term endurance, the nature of any tapeless medium is that the archives are going to have to be checked, tested and possibly transferred regularly. A drive sitting on a shelf for six months is not guaranteed to survive. And none of these new file-based archival storage media has been around long enough for us to know whether they'll really last 20 years, or 100, or just two. Plus, digital information is more likely to degrade in ways that render the whole thing unsalvageable. If you get a part of an analog video cassette warped, demagnetized or dirty, you can rescue the rest. If a digital file is corrupted, very often that's it.
Imagine the headache of maintaining a large archive of P2, R3D, or AVCHD raw media...not only would the drives need to survive, but to ensure the files are still truly intact, you might even have to run them through the conversion process periodically just to make sure the file structure is still sound, that the video and audio are as they were, and so on.
www.derekmok.com
|
Re: OT: Who's using Blu-ray for archiving? June 30, 2010 02:50PM |
>You can rescan a good film print 40 years later
The counter argument to that, is there are quite a number of people who have U-matic, S-VHS or Hi-8 tapes without a functioning deck. In a few good years, we can add Beta-SP to the list. Last year, I had a friend who asked me if he knew if anyone still had a DAT player so he could recapture the audio from an old project. And I didn't know who still keeps one of those machines. Film never really had competing technology for a very long time. Film editing on the other hand, has been taken over by NLEs such as FCP and Avid, so we know a flatbed editor isn't worth as much these days.
In world of audio recording, the old analog 1 inch open reel multi track recorders (eg. the Ampex from the 70s), has largely been replaced by digital open reel formats, and today, the DAW. Those machines are expensive, and used to cost between 5-6 digits. Storing audio recordings on those formats, isn't a pain free process either. Every few months, those analog tapes had to be loaded back on the open reel machine, fully re-wound to prevent the magnetic audio information from printing over.
Regarding digital formats, we talk about Mpeg 1, but that was a widely used end user format, and to some degree, I still occasionally send that out today. There are digital formats that we hardly hear of today- cinepak, sorenson.. How about the proprietary digital formats like microcosm. Red Giant acquired Microcosm from Digital Anarchy about two years ago, and today, it has still not been re-released. Will these formats survive for the next 20 years? And will you be able to run those files on OS 10.9 Turkish Angora?
The bottom line is that you never know what is going to happen, but if I was to make an educated choice, and I had some 3TBs of material to archive, short of making crayon drawings on wood, I would go for the LTO, and if that's not in the budget, I would get it archived onto a RAID 1 drive, and run systematic checks every 6 months. On a smaller scale, it would be optical media and hard drive.
>If a digital file is corrupted, very often that's it.
It depends on what kind of corruption.
[www.aeroquartet.com]
Heck, these days, Jon Chappell can get a corrupted FCP project up and running.
You have Disk Warrior, Data Rescue, etc... They aren't fool proof, because you need to know what kind of corruption occurred. Even film prints get mouldy if they are not stored properly.
I know of a company that had a D-beta player (Sony J30), and there was one day, one of the staff tried playing a digital beta tape from the archives, and the tape snapped, and the player had to be sent for servicing. From memory, the cost of servicing the player, was in the region of US$5K.
www.strypesinpost.com
The counter argument to that, is there are quite a number of people who have U-matic, S-VHS or Hi-8 tapes without a functioning deck. In a few good years, we can add Beta-SP to the list. Last year, I had a friend who asked me if he knew if anyone still had a DAT player so he could recapture the audio from an old project. And I didn't know who still keeps one of those machines. Film never really had competing technology for a very long time. Film editing on the other hand, has been taken over by NLEs such as FCP and Avid, so we know a flatbed editor isn't worth as much these days.
In world of audio recording, the old analog 1 inch open reel multi track recorders (eg. the Ampex from the 70s), has largely been replaced by digital open reel formats, and today, the DAW. Those machines are expensive, and used to cost between 5-6 digits. Storing audio recordings on those formats, isn't a pain free process either. Every few months, those analog tapes had to be loaded back on the open reel machine, fully re-wound to prevent the magnetic audio information from printing over.
Regarding digital formats, we talk about Mpeg 1, but that was a widely used end user format, and to some degree, I still occasionally send that out today. There are digital formats that we hardly hear of today- cinepak, sorenson.. How about the proprietary digital formats like microcosm. Red Giant acquired Microcosm from Digital Anarchy about two years ago, and today, it has still not been re-released. Will these formats survive for the next 20 years? And will you be able to run those files on OS 10.9 Turkish Angora?
The bottom line is that you never know what is going to happen, but if I was to make an educated choice, and I had some 3TBs of material to archive, short of making crayon drawings on wood, I would go for the LTO, and if that's not in the budget, I would get it archived onto a RAID 1 drive, and run systematic checks every 6 months. On a smaller scale, it would be optical media and hard drive.
>If a digital file is corrupted, very often that's it.
It depends on what kind of corruption.
[www.aeroquartet.com]
Heck, these days, Jon Chappell can get a corrupted FCP project up and running.
You have Disk Warrior, Data Rescue, etc... They aren't fool proof, because you need to know what kind of corruption occurred. Even film prints get mouldy if they are not stored properly.
I know of a company that had a D-beta player (Sony J30), and there was one day, one of the staff tried playing a digital beta tape from the archives, and the tape snapped, and the player had to be sent for servicing. From memory, the cost of servicing the player, was in the region of US$5K.
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? June 30, 2010 04:57PM |
In some places that I work at, they use LTO. And that's what I always recommend if there is a substantial amount of material that will be archived.
For me, I don't have the $5K to invest into an archival system, and by the time I archive enough material to make an LTO deck more viable, we will probably be seeing an LTO-9 system. So I use RAID 1 external drives. And everything else is backed up, including my own system drive.
www.strypesinpost.com
For me, I don't have the $5K to invest into an archival system, and by the time I archive enough material to make an LTO deck more viable, we will probably be seeing an LTO-9 system. So I use RAID 1 external drives. And everything else is backed up, including my own system drive.
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? June 30, 2010 05:06PM |
Every now and then when I feel like checking what's on the drives, which usually happens every few months. I don't use a lot of storage, because most of what I cut are in various production houses.
www.strypesinpost.com
www.strypesinpost.com
|
Re: OT: Who's using Blu-ray for archiving? July 01, 2010 02:53AM |
I already have some bare drives laying around.... maybe it's time for a Drobo. 
_______________________________________
SCQT! Self-contained QuickTime ? pass it on!
_______________________________________
SCQT! Self-contained QuickTime ? pass it on!
|
Re: OT: Who's using Blu-ray for archiving? July 01, 2010 10:31AM |
>The deal killer for us, however, was IBM's migration plan. Every 2 years, we'd have to invest in
>the next generation of tape
If you're really obsessive about it, I would say upgrade every 2-3 generations, so compatibility is maintained.
www.strypesinpost.com
>the next generation of tape
If you're really obsessive about it, I would say upgrade every 2-3 generations, so compatibility is maintained.
www.strypesinpost.com
Sorry, only registered users may post in this forum.
