Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Need a lot of help on storage and backup
Page <<first <prev 4 of 6 next> last>>
Jan 7, 2021 13:49:18   #
In-lightened Loc: Kansas City
 
Is anybody using GoodSync to backup external drives? And using Lightroom?

Reply
Jan 7, 2021 13:51:51   #
Rmccully
 
I keep my Lightroom Catalog on a Drobo where it produces a "duplicate" copy. If a drive in the Drobo goes down, the files are recovered and protected within the Drobo. I back up the Drobo to Backblaze cloud storage. I also back up the Drobo to another 10 TB hard drive. If one of the systems goes down, I still have two backups and one is in the cloud.

Reply
Jan 7, 2021 14:28:25   #
jdm Loc: Cape Cod
 
Gene51 wrote:
Aside from the mostly excellent advice you've already received here, may I suggest that you turn on "automatically write metadata changes into xmp" in Lightroom's Edit->Catalog Settings->Metadata tab. This will create a tiny "sidecar" file that has the same filename as your image file with an ".xmp" extension. This file contains all of your metadata and represents a snapshot of the changes. So even if you for some reason lose your catalog to file corruption or accidental deletion, you won't lose your edits.
Aside from the mostly excellent advice you've alre... (show quote)


Thank you Gene. Done.

I must say that you always make such valuable contributions in your comments to posts on this site. Thank you for being so generous with your time and talents. Regards, JDM

Reply
 
 
Jan 7, 2021 14:36:14   #
TriX Loc: Raleigh, NC
 
In-lightened wrote:
Trix....when backing up to Amazon...are files that you delete in your workflow (that you previously loaded to Amazon) deleted on your Amazon copy?


That depends on how your backup SW is configured. Amazon doesn’t delete anything unless you or your SW deletes it. Be sure you know how your BU SW handles deletions and if it implements versioning (keeping old versions of files that are changed or deleted).

Reply
Jan 7, 2021 14:39:58   #
eeengineer Loc: Madison, WI
 
rmalarz wrote:
JDM, I'd suggest a NAS unit (Network Attached Storage). That along with some backup software would be suitable for quite a bit of time into the future. I'd get one that is expandable. That way as you increase your amount of files, you can add more drives.
--Bob


I have a NAS for my Photo and computer backup. I use Synology DS 1618+ 6 drive with 4 - 10TB drives in a S1(Raid5) configuration with 1 spare. This is overkill for you at first. But if you are shooting that many pictures, you need to have room to grow. I am using 11.3 out of 26.2 TB. This gives me room to grow and I can add another drive to it. Synology has very easy to use programs that make it easy to map your folders to your computers. They also have many other programs to use the space. They even have a video security program. I advise going with the Plus version because it is commercial quality. I got mine on Amazon as a "used" in very good condition. That meant it was returned to Amazon and maybe had a scratch on it. It sits in a closet next to my internet switches, so who cares. Research Synology and all of their programs. It is a very good company that has been around a long time.

Reply
Jan 7, 2021 14:44:36   #
ColonelButler Loc: Niagara-on-the-Lake ON Canada
 
Just a comment here in cloud backup choice. Lots of reputable large providers out there such as Apple, Amazon, Google etc. There are other medium size providers of the service such as Backblaze that I use for a reasonable fee of about $50 per year for unlimited storage (I have 3+Tb at present).

The thing that sold me was the recovery service they provide. For a refundable deposit, they will provide you with a copy of your files via the mail on an external hard drive which would do the restore on your new drive at several orders of magnitude less time. It took me 21 days to upload my files over the internet when I first started using the service. If you are using an internet based recovery, consider how long it would take to restore your files.

I have an iMac and back up the computer drive with Time Machine on an external drive. My media files are stored on two synchronized external drives. Backblaze provides a backup of the media files. Therefore I have two copies at home and one external.

Reply
Jan 7, 2021 14:46:27   #
jdm Loc: Cape Cod
 
ColonelButler wrote:
Just a comment here in cloud backup choice. Lots of reputable large providers out there such as Apple, Amazon, Google etc. There are other medium size providers of the service such as Backblaze that I use for a reasonable fee of about $50 per year for unlimited storage (I have 3+Tb at present).

The thing that sold me was the recovery service they provide. For a refundable deposit, they will provide you with a copy of your files via the mail on an external hard drive which would do the restore on your new drive at several orders of magnitude less time. It took me 21 days to upload my files over the internet when I first started using the service. If you are using an internet based recovery, consider how long it would take to restore your files.

I have an iMac and back up the computer drive with Time Machine on an external drive. My media files are stored on two synchronized external drives. Backblaze provides a backup of the media files. Therefore I have two copies at home and one external.
Just a comment here in cloud backup choice. Lots o... (show quote)


Thank you, Colonel. You an a few others have recommended Backblaze. I'm going to look into this. Recovery is key in the event something happens. I thank you! JDM

Reply
 
 
Jan 7, 2021 14:51:09   #
TriX Loc: Raleigh, NC
 
ColonelButler wrote:
Just a comment here in cloud backup choice. Lots of reputable large providers out there such as Apple, Amazon, Google etc. There are other medium size providers of the service such as Backblaze that I use for a reasonable fee of about $50 per year for unlimited storage (I have 3+Tb at present).

The thing that sold me was the recovery service they provide. For a refundable deposit, they will provide you with a copy of your files via the mail on an external hard drive which would do the restore on your new drive at several orders of magnitude less time. It took me 21 days to upload my files over the internet when I first started using the service. If you are using an internet based recovery, consider how long it would take to restore your files.

I have an iMac and back up the computer drive with Time Machine on an external drive. My media files are stored on two synchronized external drives. Backblaze provides a backup of the media files. Therefore I have two copies at home and one external.
Just a comment here in cloud backup choice. Lots o... (show quote)


Just to clarify. Unless you have a symmetrical internet connection such as Google Fibre, your download speed is likely to be MUCH higher than your upload speed. For example, Spectrum Comcast Service her is 10 Mb up and 200 Mb Dow, so while it may take you weeks to initially seed the cloud, your download may be much faster (although having a HD with all your Data is certainly faster if you have lots of data). If you want to estimate the speed (and it can change substantially depending on whether you are moving large or small files), assume that the actual download speed is 1/2 theoretical, then divide by 8 to get MBytes/sec and multiply by 3600 to get a rough estimate of the speed in MBytes/hour. For example, using the 200 Mb/sec download speed I mentioned previously, that’s 100Mb/8=12.5MBytes/sec or 45 GBytes/hour. Again depending on file size and other load on the connection, your speed may vary substantially.

Reply
Jan 7, 2021 15:35:23   #
goldenyears Loc: Lake Osewgo
 
jdm wrote:
I’m looking for help to add additional storage and backup for my lightroom catalogue.
~JDM


Apparently all serious photographers use a catalogue so I guess I should find out about catalogues and see what I'm missing.

I've been accumulating tens of thousands of images since 1998 and rarely have a problem finding an image. Why does someone use a catalogue instead of just saving meaningfully named images in meaningfully named folders?

Can cataloged images be accessed without using cataloging software?

Are catalogues limited in how many images they hold? If so, can you have multiple catalogues?

Can a catalogue be copied to a backup disk like any other folder or file?

Are there stand-alone cataloging programs, or are they part of photo editing software? If you catalogue your photos in one system can you easily convert them to a different system.

Reply
Jan 7, 2021 16:10:41   #
jdm Loc: Cape Cod
 
goldenyears wrote:
Apparently all serious photographers use a catalogue so I guess I should find out about catalogues and see what I'm missing.

I've been accumulating tens of thousands of images since 1998 and rarely have a problem finding an image. Why does someone use a catalogue instead of just saving meaningfully named images in meaningfully named folders?

Can cataloged images be accessed without using cataloging software?

Are catalogues limited in how many images they hold? If so, can you have multiple catalogues?

Can a catalogue be copied to a backup disk like any other folder or file?

Are there stand-alone cataloging programs, or are they part of photo editing software? If you catalogue your photos in one system can you easily convert them to a different system.
Apparently all serious photographers use a catalog... (show quote)


Goldenyears,

Goodness, I'm just a person who loves photography. I use Lightroom to store and organize my photos and the language of "catalog" comes from there, referring to all of your many photos. I then organize my photos into what Lightroom calls "collections," which are what I think you call "meaningfully named folders."

I don't think one can access the catalogue outside of Lightroom but, to be honest, someone with more than my technical ability may know more on that. I only have around 6,000 images now, after a recent culling, so I've never tested the size limit of the catalogue. (I'm completely impressed that you have 10s of thousands and no difficulty finding one. Clearly, your system works for you.) I do believe that you can have multiple catalogues, but I do not.

You can copy a catalogue to one or several back up sources, which was, in fact, the nature of my inquiry here to begin with.

I don't know the answer to your later question about stand alone catalogues. I only know of software based ones, such as Lightroom, Bridge, and others.

Your last question about converting a catalogue from one system to another is a really good one. A friend of mine just switched from Mac to PC and her iPhotos went with the switch; she's looking for help getting her catalogue onto her PC. Please let me know if you learn anything on that. Regards, JDM

Reply
Jan 7, 2021 16:48:56   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
goldenyears wrote:
Apparently all serious photographers use a catalogue so I guess I should find out about catalogues and see what I'm missing.

I've been accumulating tens of thousands of images since 1998 and rarely have a problem finding an image. Why does someone use a catalogue instead of just saving meaningfully named images in meaningfully named folders?

I also have a lot of digital images collected since 1999. In addition I have a lot of family photos scanned to produce digital files. They date back to the 1860s and some documents back to 1830. As I age, my memory has degraded so I have trouble remembering how to find things. I use Lightroom, which has a catalog based on a database. I can use keywords and collections to find things and labels to manage my workflow.

As far as storing your files in meaningfully named folders, that has some advantages. I do both. I find the catalog easier to use, but I'm used to working with databases. On the other hand, my family hasn't a clue about databases or Lightroom so that method of finding things would be useless for them. I use a folder-based system so they have a chance of finding the family photos I assume they will be interested in. Of course even that will not do them any good if they do not know that the photos exist or where they might be found. So I have to keep my family appraised of the pile of family photos that are available, but will last only a short time after I have kicked off unless someone takes them over.

More information about my organizing system can be found at https://www.uglyhedgehog.com/user-page?upnum=1595

goldenyears wrote:
Can cataloged images be accessed without using cataloging software?

Database catalogs will need the database software to access them. Folder catalogs can be accessed by normal computer search techniques.

goldenyears wrote:
Are catalogues limited in how many images they hold? If so, can you have multiple catalogues?

There may be limitations on the number of images but I believe they are very large (assuming your computer has enough memory).
You can have as many catalogues as you wish. But since you can search within a catalog based on specific terms you can use a single catalog to hold multiple subjects.

When I started to set up my catalog I had 65,000 images. A lot of them were duplicates or different versions due to different editing sessions. It was a real mess. The first thing I did was to take all my images and split them up by year. I produced one catalog for each year. Then I could go through the images one year at a time so I had smaller numbers to deal with. I could delete duplicates and useless versions or use virtual copies for different edits. When I got through all the images (took a couple months working in free time, a couple hours at a time), I just took the first catalog, changed the name to "Master Catalog 2010" and merged all the other catalogs into that. Since then I just copy the catalog every January and change the name to the current year.

goldenyears wrote:
Can a catalogue be copied to a backup disk like any other folder or file?

A catalog is a file. It can be copied, backed up, and distributed among several disks.

goldenyears wrote:
Are there stand-alone cataloging programs, or are they part of photo editing software? If you catalogue your photos in one system can you easily convert them to a different system.

There are stand-alone cataloging programs that are used in business to catalog documents, which include photos as a subset. The program I use, Lightroom, is only for photos. Not all editing programs include a cataloging system. The ones that do have systems that are proprietary, and are not compatible with other programs.

PS: I'm an amateur photographer. I have rarely been accused of being serious. But I do find the catalog useful.

Reply
 
 
Jan 7, 2021 17:13:17   #
Harry0 Loc: Gardena, Cal
 
jdm wrote:
I’m looking for help to add additional storage and backup for my lightroom catalogue. (My apologies in advance for not being that technical … hence why I’m reaching out to you.)

I’ve been shooting with Nikon D750 and just got the D850 for Christmas. Due to the file size of the images of the 850, I started thinking about (and worrying about) storage, which got me then thinking about backup, too. I fear that both situations are inadequate.

Here’s my setup:
~MacBook Pro (2019)
~Lightroom Photoshop with all images stored on external drive (WD 2TB My Passport Ultra; 7000+ images)
~Backup to Time Machine on external drive (again a WD 2TB My Passport Ultra … I’ve assumed all along that this backs up my lightroom catalogue residing on the other external drive … is that true?)

Please advise on storage space and backup with any/all recommendations. I am an Amazon Prime Member so I have access to their free storage. I have OneDrive and storage on Microsoft360. I’d also be willing to subscribe to a backup service if it makes the most sense. I have no idea how to go about copying or uploading my lightroom catalogue from the external hard drive to a new backup location.

I know I’m asking a lot. I’m hoping there’s a kind soul out there in the mood to take on a challenge!

Many thanks for your help and patience!

~JDM
I’m looking for help to add additional storage and... (show quote)


I picked up a 10tb Western Digital Element,
And it works well for my uses.

Reply
Jan 7, 2021 19:46:10   #
goldenyears Loc: Lake Osewgo
 
jdm wrote:
More information about my organizing system can be found at https://www.uglyhedgehog.com/user-page?upnum=1595JDM


Very thorough and informative. Thank you, and everyone who replied.

I use two simple programs to help me locate and backup hundreds of folders containing +/- 75,000 images. Most of the images are sent to me for posting on a website so it is a very fluid process to manage. I work on it daily. My primary level of organization is in folders named for the sender.

One program I use is for finding files. It searches indexes of connected storage devices by a portion of the file name, i.e. "Boeing 787", or "N2744A." Results take only seconds, and can be sorted by date, device, folder or file name, and renamed, opened, deleted, copied, or moved.

The other program I use for backup. It lists files by name in two storage locations side by side, according to criteria. The display shows matching, differing, or "orphan" files. The newer of the listed files can be bulk copied from one side to the other. Files can be individually renamed, opened, deleted, copied, or moved.

Reply
Jan 7, 2021 21:50:26   #
A. T.
 
jdm wrote:
I’m looking for help to add additional storage and backup for my lightroom catalogue. (My apologies in advance for not being that technical … hence why I’m reaching out to you.)

I’ve been shooting with Nikon D750 and just got the D850 for Christmas. Due to the file size of the images of the 850, I started thinking about (and worrying about) storage, which got me then thinking about backup, too. I fear that both situations are inadequate.

Here’s my setup:
~MacBook Pro (2019)
~Lightroom Photoshop with all images stored on external drive (WD 2TB My Passport Ultra; 7000+ images)
~Backup to Time Machine on external drive (again a WD 2TB My Passport Ultra … I’ve assumed all along that this backs up my lightroom catalogue residing on the other external drive … is that true?)

Please advise on storage space and backup with any/all recommendations. I am an Amazon Prime Member so I have access to their free storage. I have OneDrive and storage on Microsoft360. I’d also be willing to subscribe to a backup service if it makes the most sense. I have no idea how to go about copying or uploading my lightroom catalogue from the external hard drive to a new backup location.

I know I’m asking a lot. I’m hoping there’s a kind soul out there in the mood to take on a challenge!

Many thanks for your help and patience!

~JDM
I’m looking for help to add additional storage and... (show quote)


My friend this is the place to ask questions and get the answer, so never apologize or feel uneasy about asking them. Now, I will give my two cents worth of information that has helped me. I have Macs and I use time machine; however, time machine is a backup but you should never rely on time machine for safe keeping of your photos. There's a saying, "two copies is one and three is two". I have a RAID (Redundant Array of Independent Drives) system. I also use an off site subscription backup (Backblaze). Keep in mind that all hard drives have a life expectancy so don't think that just because you use hard drives you are forever safe. Those drives need to be updated over time, meaning those photos need to placed on new drives. Now I'm certain that everyone has their own method of photo protection and I'm just giving you my system. I will tell you this, you should have at least three copies of all your photos with one being an off site facility such as Backblaze. FYI, I have a D850 and you are absolutely correct; the file size is the same as a medium format camera, huge. Good luck.

Reply
Jan 7, 2021 22:28:38   #
TriX Loc: Raleigh, NC
 
A. T. wrote:
My friend this is the place to ask questions and get the answer, so never apologize or feel uneasy about asking them. Now, I will give my two cents worth of information that has helped me. I have Macs and I use time machine; however, time machine is a backup but you should never rely on time machine for safe keeping of your photos. There's a saying, "two copies is one and three is two". I have a RAID (Redundant Array of Independent Drives) system. I also use an off site subscription backup (Backblaze). Keep in mind that all hard drives have a life expectancy so don't think that just because you use hard drives you are forever safe. Those drives need to be updated over time, meaning those photos need to placed on new drives. Now I'm certain that everyone has their own method of photo protection and I'm just giving you my system. I will tell you this, you should have at least three copies of all your photos with one being an off site facility such as Backblaze. FYI, I have a D850 and you are absolutely correct; the file size is the same as a medium format camera, huge. Good luck.
My friend this is the place to ask questions and g... (show quote)


👍👍 Spot on. 3 copies - working, backup, and an off-site disaster recovery (DR) copy.

Reply
Page <<first <prev 4 of 6 next> last>>
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.