Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Best Post Processing Setup
Apr 22, 2018 15:21:05   #
neillaubenthal
 
Hi…in the never ending quest to make a better post processing setup I figured I would ask here to see what others might be doing that I could borrow for my images.

We live full time in our RV and travel continuously…so my main computer is a laptop. It's a Retina MacBook Pro…but this isn't a Mac vs Windows thing so let's not devolve ourselves into that debate nobody ever wins. I've got a 512 GB SSD in the laptop which is obviously too small to keep all the original image files on. I shoot RAW with a D7500 and use Lightroom CC Classic, Aurora HDR, and Luminar 2018 (all up to date) to manage and process the images. Library management is all handled by LR with the other two getting exported images for processing which then auto re-import back into LR.

I've only got one LR catalog…it goes back to 2010 or so, the first couple of years were imported from Aperture when it was abandoned but everything from about 2014 has always been in LR. The catalog points to a year/month folder arrangement over on a NAS connected to our RV network with about 1.5TB of images in it.

When I import into LR all the image files go over to the NAS and only the preview files and catalog file itself are on the laptop. I've got plenty of backup systems in place for this arrangement…an extra NAS drive in the RV and periodic clones of the NAS to drives that live in the car and truck that tows the RV.

The problem is that importing takes a long time as does anything that requires accessing the NAS via wifi from the laptop…resulting in slow export/import/move between images, etc. So I've been looking…for a year+…for a better method. I've already done most of the stuff with the LR catalog that one gets from googling how to speed up LR…all of that helped some but not significantly.

I tried keeping the originals for a few collections on the laptops SSD…and amazingly enough those are much quicker to access.

About the best idea I've come up with so far is to just put a bigger SSD in the laptop and keep the original images there…but that means upgrading to a 2 TB SSD or just keep the current year's photos on the laptop which means I could get by with a 1 TB SSD. I can make a duplicate copy of the images as they're imported over on the NAS so I've got backup there and between those, Apple's Time Machine backup and a weekly clone of the laptop SSD along with the existing backup strategy I'll have plenty of in RV and out of RV copies of the files. However…I haven't figured out a really good scheme for moving the older year's photos off of the laptop as time goes on…obviously LR can move them…but then I'll end up with 2 copies of them on the NAS…the original one that was created during the original import process and the copy that was created when the LR originals were moved from the laptop to the NAS…unless there's a way to tell LR to now use the NAS copy as the "original" rather than the one on the SSD so that the copy on the SSD can be manually deleted after the re-referencing is complete…not to mention the time lag to copy last year's say 500 GB of images over from the SSD to the NAS.

Has anybody already solved this problem or am I looking in the right direction already…or if there are multiple solutions folks have figured out toss 'em all in here so we can all be smarter. Heck…maybe having multiple catalogs is a better idea…although if they all refer back to the same folder on either the SSD or NAS I'm not sure if there's any advantage either way.

Reply
Apr 22, 2018 20:20:27   #
toxdoc42
 
I just purchased a 4 TB USB hard drive from Costco for about $80 and I am using that for everything that I can, I also have a slimmer 2 TB external drive that I carry with me when I travel to use with my Surface.

Reply
Apr 23, 2018 07:42:01   #
PhotosByCat Loc: Baltimore, OH
 
External hard drives can be fairly inexpensive on sale. That is what I use for main, backup, and finals. I have 3 separate drives. One has all my original raw and/or jpeg files. One has the exact files plus all PS adjustments I make. This is the one I organize so I can find a pic quickly. The third one has ONLY my final jpeg version on it, also organized by subject matter. Hope this helps you.

Reply
 
 
Apr 23, 2018 08:52:16   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
neillaubenthal wrote:
Hi…in the never ending quest to make a better post processing setup I figured I would ask here to see what others might be doing that I could borrow for my images.

We live full time in our RV and travel continuously…so my main computer is a laptop. It's a Retina MacBook Pro…but this isn't a Mac vs Windows thing so let's not devolve ourselves into that debate nobody ever wins. I've got a 512 GB SSD in the laptop which is obviously too small to keep all the original image files on. I shoot RAW with a D7500 and use Lightroom CC Classic, Aurora HDR, and Luminar 2018 (all up to date) to manage and process the images. Library management is all handled by LR with the other two getting exported images for processing which then auto re-import back into LR.

I've only got one LR catalog…it goes back to 2010 or so, the first couple of years were imported from Aperture when it was abandoned but everything from about 2014 has always been in LR. The catalog points to a year/month folder arrangement over on a NAS connected to our RV network with about 1.5TB of images in it.

When I import into LR all the image files go over to the NAS and only the preview files and catalog file itself are on the laptop. I've got plenty of backup systems in place for this arrangement…an extra NAS drive in the RV and periodic clones of the NAS to drives that live in the car and truck that tows the RV.

The problem is that importing takes a long time as does anything that requires accessing the NAS via wifi from the laptop…resulting in slow export/import/move between images, etc. So I've been looking…for a year+…for a better method. I've already done most of the stuff with the LR catalog that one gets from googling how to speed up LR…all of that helped some but not significantly.

I tried keeping the originals for a few collections on the laptops SSD…and amazingly enough those are much quicker to access.

About the best idea I've come up with so far is to just put a bigger SSD in the laptop and keep the original images there…but that means upgrading to a 2 TB SSD or just keep the current year's photos on the laptop which means I could get by with a 1 TB SSD. I can make a duplicate copy of the images as they're imported over on the NAS so I've got backup there and between those, Apple's Time Machine backup and a weekly clone of the laptop SSD along with the existing backup strategy I'll have plenty of in RV and out of RV copies of the files. However…I haven't figured out a really good scheme for moving the older year's photos off of the laptop as time goes on…obviously LR can move them…but then I'll end up with 2 copies of them on the NAS…the original one that was created during the original import process and the copy that was created when the LR originals were moved from the laptop to the NAS…unless there's a way to tell LR to now use the NAS copy as the "original" rather than the one on the SSD so that the copy on the SSD can be manually deleted after the re-referencing is complete…not to mention the time lag to copy last year's say 500 GB of images over from the SSD to the NAS.

Has anybody already solved this problem or am I looking in the right direction already…or if there are multiple solutions folks have figured out toss 'em all in here so we can all be smarter. Heck…maybe having multiple catalogs is a better idea…although if they all refer back to the same folder on either the SSD or NAS I'm not sure if there's any advantage either way.
Hi…in the never ending quest to make a better post... (show quote)


You don't want to hear this, but using your MBP with a NAS is not a system that will work for live active storage. You are better daisy-chaining some Thunderbolt 2 drives, which would be considerably faster than a NAS.
It would be even faster if your computer could accommodate more storage. I routinely configure laptops with 4 drives - two blazingly fast PCIe NVMe m.2 drives and two SSDs (or for a less expensive setup, two 7200rpm mechanical drives) all inside the box.

A NVMe m.2 can attain speeds of up to 3500 MB/s, compared to SSD over SATA which gives you 500 MB/s and a fast SATA III 7200rpm drive which maxes out at between 100 and 200MB/s

Thunderbolt 3 gives you 5 GB/s, Thunderbolt 2 maxes out at 2.5 GB/s and Tbolt 1 gives you a theoretical max of 1.25 GB/s

Using a fast SSD with Tbolt 3 would be almost as fast as having it in the laptop. That is your fastest option.

NAS is great for archiving and backup, and for shared storage if you are on a small network. It is not intended for active storage, unless you are using a QNap device with has Thunderbolt 2 (and many 3 at this point) interface and is not a NAS, but rather a DAS or direct access storage device. NAS is network attached storage - implying that it works on gigabit ethernet, which is considerably slower.

I would avoid the 4 TB/$80 drives like the plague. They are the mfgr's cheapest drives and more often than I'd consider reasonable never make it to the end of their warranty period. You are better off getting a WD Black, or HGST Ultrastar drive and putting it into an accessory drive cabinet that you can buy for $20 or so.These drives are typically warrantied for 5 yrs.

Reply
Apr 23, 2018 12:47:19   #
Ron Dial Loc: Cuenca, Ecuador
 
You did not mention how you are backing up the duplicates of your images. The issue you are dealing with is common to all digital photographers. Size of Hard Drives, speed, indexing so you can find the image you need. And since you are living in an RV, I assume you have no place for off-site storage. These are issues you need to deal with. Multiple Hard Drives is probably the solution you will find most workable. Consider a fire resistant box for your back-up copies. Not perfect, but probably the best. Keep the box in a different part of the RV, away from your primary work place. You do not want to lose everything to a fire.

Reply
Apr 24, 2018 01:05:33   #
wesm Loc: Los Altos CA
 
Gene51 wrote:
...I routinely configure laptops with 4 drives - two blazingly fast PCIe NVMe m.2 drives and two SSDs


Could you do this with a MacBook Pro? I don't think so, but I'm not sure.

I agree that a thunderbolt connection to an SSD external drive is probably the best option for the OP.

Reply
Apr 24, 2018 04:54:20   #
Ron Dial Loc: Cuenca, Ecuador
 
I know MAC is a popular platform and their new Thunderbolt connection is very fast and all. But consider that a more middle of the road connection might be the right choice. After all it was MAC that originated Firewire and all of my Firewire connection devices for MACs are not SOL. For a back up system you have to consider what will be the future interface, and be prepared to convert everything to a new interface if that happens.

Reply
 
 
Apr 24, 2018 05:14:07   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
wesm wrote:
Could you do this with a MacBook Pro? I don't think so, but I'm not sure.

I agree that a thunderbolt connection to an SSD external drive is probably the best option for the OP.


No, I use custom gaming/engineering workstation laptops from Sager Notebooks. They work out to the same price as Macbook Pros, after you end up buying all the external and internal storage you need. If the MBP has 16 gb ram (minimum) and Thunderbolt 3, you can get very close to the performance of the custom units, it will just be slightly less convenient if you have a bunch of external Thunderbolt 3 hard drives. This is the main difference between closed architecture (Mac) and open architecture (Windows).

Reply
Apr 24, 2018 05:24:24   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
Ron Dial wrote:
I know MAC is a popular platform and their new Thunderbolt connection is very fast and all. But consider that a more middle of the road connection might be the right choice. After all it was MAC that originated Firewire and all of my Firewire connection devices for MACs are not SOL. For a back up system you have to consider what will be the future interface, and be prepared to convert everything to a new interface if that happens.


It doesn't matter, firewire may be great, but Jobs killed it by overcharging for licensing for it's use, making it less desirable for a mfgr to implement than USB - USB requires CPU cycles to run, Firewire has zero impact on CPU use.

The nice thing about using your own external cases, you can keep the drives and change the cases when the interface changes. When you buy the "packaged backup solutions" sometimes the drives are very difficult to remove from the cases.

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.