Nikon1201

BackBlaze is unlimited storage

Nikon1201

BackBlaze is unlimited storage

pebble70 Loc: Winchester, MA USA

I too am also a pro photographer.
I used a 3tb mirrored external harddrive that is 3 tb plus a mirror image . so, a back up to the back up.
it recently crashed and i was told there are three levels of recovery $100////500.00 ///or clean room recovery 1500.00
I have since purchased 2 8tb external harddrives it's a little more work; but I do not leave these either powered up or connected to the computer.
So they only run when I want them to, or download what I want them to...

seems to be working
Ken

TheShoe Loc: Lacey, WA

Any that you archive should be in multiple files that are (a) kept in separate locations, and (b) not on the RAID array. Its functions are as backup and speed of reading or writing. Speed is unimportant for those files that are inactive.

hpucker99 Loc: Anchorage, Alaska

Tis is the first I have heard of this type of problem. I already have a RAID but will look into getting another. Do you remember who the manufacturer of the enclosure and drives were?

dyximan

I want to thank all of you for your time and effort in helping me to understand the differences and make an informed decision

TriX Loc: Raleigh, NC

Actually, RAIDs leverage the performance of multiple drives to increase speed, reliability, or both. Yes, you can drop a RAID box, but you can also drop drives, displays, computers, etc. by the same token, a flood will likely disable one as will a fire, but exactly the same can be said of the computer attachéd to the RAID and any drives directly connected to it. Net-net, none of these are valid reasons to avoid the advantages of a RAID array.

There are two ways to implement a RAID. You can either use the built-in functionality of your OS (Windows has that functionality built-in, I can’t speak to Macs) with the drives located either externally or in your enclosure. Alternately, you can purchase a seperate RAID enclosure with a RAID controller built-in. I would agree that the RAID controller is one additional potential point of failure, but so is the disc controller card or circuitry in your computer, so no difference there, and if the controller fails, your data is not lost, just unavailable until you replace the controller card. Also RAIDs typically allow hot plugging and automatic rebuilds of failed drives.

If you use a RAID (and RAID levels like RAID 3 aren’t available), you can choose:

RAID 1 (simple mirroring). You get no increase in speed (unless you can read different files from both mirrors simultaneously), but you get n+1 redundancy. The cost is you pay for 2x the usable space

RAID 0+1 (striping and mirroring) you get both a 2x speed increase and n+1 redundancy. Same disadvantage as RAID 1 - you pay for 2x the usable capacity. This is the classic favorite of DBAs (data base administrators), but you’ll need at least 4 drives to implement this.

RAID 5 (block striped with rotating parity) there may or may not be a speed increase since the actual purpose of RAID 5 is to allow multiple clients to read different files simultaneously, but you do get redundancy with less “parity overhead”. For example, in a 4 drive group, you may only use 25% of the available capacity as parity (drive failure protection), allowing you to use 75% for available storage. If you lose a drive, you put in a spare, and the RAID rebuilds the data onto the new drive. This can take quite a while with the newer large capacity drives, and you’re vulnerable to a second drive failure during this time, which may take days. To protect against this, some RAIDs allow you to use double parity (sometimes called RAID 6), but this means you pay for an extra drive that is protection, not usable space.

Finally, double drive failures DO happen. It may be a nearby lightening hit, a power surge, a cooling failure or malware that propagates/corrupts both drives, and this is what a seperate backup or DR copy (preferably in a seperate physical location) is for.

Good luck with whatever you decide, and keep your precious data safe with multiple seperate copies.

Neilhunt

Synology DS418 NAS Disk station, 4-Bay, 2GB DDR4 (Diskless) + 2 of WD Red 8TB NAS Hard Disk Drive - 5400 RPM Class SATA 6 Gb/s 128MB Cache 3.5 Inch - WD80EFZX
* https://smile.amazon.com/gp/product/B075N17DM6/ref=oh_aui_detailpage_o05_s00?ie=UTF8&psc=1
* https://smile.amazon.com/gp/product/B01BYLY4DM/ref=oh_aui_detailpage_o04_s01?ie=UTF8&psc=1

Not the cheapest, but good quality.
Four-bay enclosure, so room to expand.
Fill two bays with 8TB discs, RAID1, almost 8TB of storage.

In a couple of years, fill the other two bays with 16TB or 32TB discs, which will be on the affordable curve by then.

Cheese

I have a WD My Cloud EX2 Ultra. Solidly built, easy to setup and use. I have had it about a year and no problems whatsoever.

jdubu Loc: San Jose, CA

I have 2 Synology NAS RAID 1 set ups. One for our office and one for the home. Office has all our business files and the home one has all my photography files. I keep a separate hard drive copy of each Synology located in the opposite location from the original NAS.

Both Synologys auto backup to a different Synology that my sister has in her home linked through her network. Her Synology for her business backs up to a Synology I keep at my house for her safety. I do not use any cloud backup, but do upload for client access with authorized access.

blackest Loc: Ireland

This is an rsync method, its run on an android tv with a 4tb drive running Linux (it uses less than 10 watts of power an hour about 1.68 KWatts a week or less)
my lightroom files are stored on a nas with several drives attached the lightroom folder is on 4tbdata it contains all my lightroom photo's and a backup of the catalog.

SSH lets me login to the nas remotely and the android tv box


#!/bin/sh

rsync -avz -e "ssh -i /home/backup/cron/androidtv-rsync-key" backup@192.168.3.253:/media/4tbdata/Lightroom/ /media/iPad/itunesmedia/Lightroom/

This command is a little complicated the file transfer in encrypted so I don't have to supply a password:
"ssh -i /home/backup/cron/androidtv-rsync-key"

Supplies a preshared key from the androidtv box which is authenticated by the nas

backup@192.168.3.253:/media/4tbdata/Lightroom/
My nas sits on my lan at 192.168.3.253 the user that logs in is called backup (best practice is to use a user with only permissions to do the task and unable to do anything else). It's a usb harddrive and it mounts in /media with a mountpoint of 4tbdata which has a folder called lightroom in it.

/media/iPad/itunesmedia/Lightroom/

this is the destination folder another usb drive mountpoint is called iPad. I use this location for backing up both my itunesmedia and Lightroom, basically all the important stuff goes there. The command copies any changed file to the androidtv box. If I add new files to lightroom it sees them and copies them to the backup, if i alter a file in lightroom it copies the blocks that have changed, (with a dng file thats usually just the xmp data from processing or any keywords added). For the first backup i actually attached the two drives to a tower system which has 2 usb3 ports, this took around 4 hours, doing it over the lan would have taken a week (unfortunately the androidtv box only has 100mb ethernet, even gigabit would take a fair while).

The second backup and subsequent backup is much faster as only the differences need to be transferred.


*/2 * * * /home/backup/cron/LRBackup.sh

On Linux there is a process called cron and to add tasks you use

crontab -e

that line basically says run that script (LRBackup.sh) every 2 hours (on the even hours)
so at 2am (and 4am and 6am ...) the androidtv box logs in to the nas box looks to see if anything new has come in and copies it over.

While you might not have cron windows has task schedular and rsync. So you can run this on windows or a mac too the main difference is that you use more electricity but you can schedule your pc or mac to wakeup and do the task and go back to sleep again (it mostly is done in less than 5 minutes).

The basic backup script is ok but it only adds to the backup, what if I delete a file on the nas? It still exists on the androidtv backup.

The following is a script I called deletebooks.sh

rsync -zav --delete --backup --backup-dir=/media/iPad/deleted/Books/ backup@192.168.3.253:/media/3tbdata/itunesmedia/Books/ /media/iPad/itunesmedia/Books/

This i run on the android tv after logging into it via ssh (an ssh session is just a terminal / command line). When I run this command it connects to the nas asks for a password and it copies changed files as before. It could be fully automated like the other one but this command is a tad dangerous.

Any files that are not on my nas are now deleted from the backup on the androidtv, but it doesn't do a straight delete of the files instead it moves them to a folder called Books in a folder called deleted. (named that because thats where i put my deleted files).

Its quite easy to add duplicates to itunes say i have 2 epub copies of "the print.epub" by Ansel Adams
well itunes makes a folder called books and inside that a folder for each author e.g /Books/Ansel Adams/
"the print.epub"
"the print 2.epub"

when i delete the 2nd copy of "the print" my backup on the androidtv still has both when i run the script "the print 2.epub"
appears in /media/deleted/Books/Ansel Adams/the print 2.epub

If i have messed up in itunes and i really didn't mean to delete that version I can still recover it. But mostly that /deleted/Books/ folder can be safely deleted.

If i was using this script automatically i'd probably have /deleted/todays date/Books/ that way it would reflect when things got deleted but its just a second chance so mostly i will delete the deleted items once i'm sure i dont need them.

The backup drive is also a network fileshare so I can mount it on my mac I can also rename the folders so /deleted/Books/ might become /deleted/2018-2-12Books/ that way I don't need to cull that folder straight away.

Whys that drive called iPad? well one of the things i wanted to be able to do was transfer files to and from my ipad without using my mac or uploading and downloading via the internet. As a network share I can access it from the iPad and so without hooking the ipad to my mac, say I feel like reading "The Print" I can navigate to /itunesmedia/Books/Ansel Adams/the print.epub and download and open it in ibooks on the iPad. Once I've done with it on the iPad, I can just delete it. knowing I have 2 copies still on my nas and on my android TV.

I can also upload photos from my iPad to the AndroidTV and later run Lightroom on my mac import them into lightroom (which stores them on my NAS which is backed up every 2 hours to the Android TV Lightroom folder).

And there is a little more too. I configured the android tv box so its wifi forms an adhoc network called androidtv and computers and my iPad can join this network. If i'm home I can use my regular network now i made /media the fileshare for my lan which means any drive i plug in to my Android TV box is automatically shared on my network. So if I put an sd card reader in one of the usb slots the card is mounted automatically so when I connect to the share on my iPad it see's both the card and the USB hard drive so it is as simple as selecting the DCIM folder on the card and copying that to the USB hard drive. Any card type as long as I have a card reader for it.

Final bonus I have a 5600mah powerbank (also a handy torch) I can plug the androidTv into that and run it that way (it should be ok for a couple of hours at least, I can recharge the powerbank anywhere there is a usb port. e.g the usb port I have plugged into my Car.

I can also run kodi on the androidtv box and have a hdmi cable plugged into a tv, or just use the iPad to play any media file on there.

I paid about $50 for the android tv box it uses an Amlogic 905 cpu the Amlogic 905x is also good but the 905w is not so great. Many Android TV boxes use Rockchip processors and they are difficult to get working so avoid those. I bought a 16GB class 10 microSD card to run android on for around $16 you can use as small as 4GB cards but class 10 is needed. If I remove the microsd card it boots into android OS. It can still read cards and the external hard drives. So you could possibly plug the hdmi into a hotel tv or stream media to a chrome-cast.

Cpu is a quad core arm cpu with 1 GB ram it's really over kill for backup duties but it can play 4k movies too... If you have a netflix subscription you can use it for that too. if you plug in a mouse and keyboard you can pretty much run any linux program. Box is roughly the size of a portable hard drive but much lighter. Oh and if I want i can grab the hard drive from it and take it with me and my laptop i just need to tell lightroom where my photos are and it will work with my raw files as normal.

So thats what $70 buys you (actually the only thing new i bought was the micro sd card for $16 I had everything else).

Oh most of the backup programs are rsync based e.g Carbon Copy Cloner if you run that on a mac and open the activity monitor you will see one or two processes called rsync. rsync is free too, but the gui interfaces to things like carbon copy cloner are more reassuring even if all they actually do is build a command line for rsync.

This can work over the internet too, but to say store files with a relative they need to have a public ip address. Because my internet provider doesnt give me a public ip address its rather difficult to access my nas from outside my lan.

Thats my backup solution

KC Thompson

Thanks for the good thoughts, but sadly, I'm not among the majority of the population with high speed internet. It may be 20 years old, but DSL is all that's offered in my part of the country - and I'm truly in the country. Downloading my files from CrashPlan would have taken several years. I went with them initially because they offered a specially configured disk to get my huge files uploaded efficiently. Backup occurred quite fine until the day of the crash - at which point I learned they'd changed policies from providing a disk to only streaming my files back to me, leaving me bereft of any possibility of recovery. I'm still mad - and until I'm included along with the majority of the population's internet capabilities as you described, I'm stuck with redundant hard drives as a solution to protecting my work -- a solution that I recommend to others that may not have the huge pipes for streaming from a cloud that you are evidently lucky enough to have.

jdubu Loc: San Jose, CA

Then self contained NAS systems is a great way for you to go. You control everything on site and your backup plan can be other hard drives, maybe Blu-ray discs as a different media backup. All within your LAN, no need for internet speed issues. If you do use NAS in a Raid setup, plan for the future of your storage, not for what you will have in the next year or so. Some NAS systems in the beginning had a maximum HD size.

TriX Loc: Raleigh, NC

Makes sense to me, and I’d be unhappy with crashplan also. Many/most major cloud providers such as Backblaze will send you either a HD or USB memory sticks if the recovery download will take too long.

blackest Loc: Ireland

I like raid its a nice way to be sure of short term backup , i mean if I copy a card to a raid1 drive there are 2 copies from the get go as well as the card and if things work out you can read faster too. But for redundant backup single drives with contents duplicated can be more flexible. for one thing they can be in separate locations. maybe you get a power spike from lightening this might take out both raid drives being hooked up on the same controller... Maybe better to have an online and offline backup hooking up the off line backup periodically to keep the third backup up to date.