Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Post-Processing Digital Images
Google imagery progress...
Oct 4, 2021 08:05:17   #
Rongnongno Loc: FL
 
I have been following articles published by google and their imagery progress is rather incredible.

Click to see the article. Check the movie withing the article...

I would like to see this implemented in a software. Does any of you know if there is one using this technology?

Reply
Oct 4, 2021 08:23:24   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
Did a quick scan of the article. I don't know anyone who uses this yet but it seemed to me that the high resolution result is pretty much made up. There is no guarantee that it represents the way the image should look. How can you produce a real image when the missing data are not there? (Missing data being the interpolation between given pixels).

The third paragraph of the article ends with the sentence "This synthesis procedure can be interpreted as an optimization algorithm that follows the gradient of the data density to produce likely samples." The key words here are "likely samples".

Reply
Oct 4, 2021 10:34:28   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
Here is a part of the second clip. It's an animated gif so I loaded it into Photoshop and extracted the 32x32 and 256x256 images of the streetcar. I then enlarged the image so they were both the same size.

The rightmost window seen above the streetcar looks like a featureless black blob in the 32x32 view. It's about 40 pixels wide by 30 high. In the 256x256 view its a similar size but now you can see the mullions, which appear to me to be extracted out of nothing. If you look at the other windows, the mullions show a different pattern. Normally I would expect the mullions to be in a similar configuration in adjacent windows.

It appears to me that this is AI adding things that aren't there.

32x32
32x32...

256x256
256x256...

32x32 Windows
32x32 Windows...
(Download)

256x256 windows
256x256 windows...
(Download)

Reply
 
 
Oct 5, 2021 03:29:28   #
Rongnongno Loc: FL
 
It appears you are dead set against it.

Any AI that enlarges an image creates pixels to fill the void.

Google is by far more realistic than any other - including PS CC -.

Also, Google uses a specific technic that does not employ pixels but noise. This appears to remove most of the artifacts left when upping an image resolution.

As too adding you are not paying attention.
The top metal bar is removed or appears a ghost, as does the lower one. The two are replaced by a lower slanted bar from right (higher) to left (lower).

Reply
Oct 5, 2021 07:04:17   #
selmslie Loc: Fernandina Beach, FL, USA
 
DirtFarmer wrote:
... Normally I would expect the mullions to be in a similar configuration in adjacent windows.

It appears to me that this is AI adding things that aren't there.

You have seen through what is happening here.

If you look a little further to Image Super-Resolution via Iterative Refinement you will see that all of the examples require a high resolution "Reference" or target for the output of the models. Without that reference there would be no way to create a result from a random collection of noise. There would be no way to distinguish hair on someone's face from a charcoal smear.

If this were an image enhancement process it would have produced a new image larger than the reference. It didn’t.

Reply
Oct 5, 2021 09:50:42   #
anotherview Loc: California
 
I didn't finish reading the article, but found its approach useful yet in practice limited. It did not match the sharpness of a properly focused lens and the application of the camera shake reduction filter and high pass filter together in the full Photoshop. I went by my eyesight and personal perception.
Rongnongno wrote:
I have been following articles published by google and their imagery progress is rather incredible.

Click to see the article. Check the movie withing the article...

I would like to see this implemented in a software. Does any of you know if there is one using this technology?

Reply
Oct 5, 2021 09:51:06   #
JBRIII
 
There was an article last year? on a process developed, NASA?, to combine low res. satelite images taken at different times and thus angles relative to the ground to yield high res. images. Quite impressive if you had a supercomputer and several months. Not an automatic process. Since todays computers are often = to the supercomputers of yesteryear. There are often algorithms that exist, but aren't practical on the computers available even in the average research lab.

Reply
 
 
Oct 5, 2021 14:18:29   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
Two points:

(1) I do not believe that all photos require that they represent reality absolutely. Certainly there are some photos, that by their application require that they represent reality as closely as possible. Photos of people require enough reality that the people look like the person being photographed, but cosmetic removal of defects is OK to the extent that it does not alter the image enough to make it unrecognizable.

(2) I do not believe that AI will ever be able to supply details to an image in which the details are lacking without external information being supplied. That is not to say that AI cannot produce valuable photos. As long as the result looks good and is not used to affirm some detail that was not in the original photo, AI will be a valuable tool in photographic processing.

I look forward to seeing the featured research developed into a tool usable by photographers, but with the understanding of the above points. AI is here and is in use currently in the form of Topaz DeNoiseAI and Topaz SharpenAI, not to mention Photoshop's Content Aware Fill. I use those tools and if the featured research algorithms were available to the hoi polloi (me) I would probably find a use for it.

Reply
Oct 5, 2021 15:49:57   #
selmslie Loc: Fernandina Beach, FL, USA
 
DirtFarmer wrote:
... (2) I do not believe that AI will ever be able to supply details to an image in which the details are lacking without external information being supplied. ...

Exactly!

For the examples posted under the link I provided:



The SR3 (ours) version comes close to the Reference but does not exceed its sharpness or level of detail.

The only way that AI can improve on the reference would be to recognize that the image includes human or cats eyes and, using information from a separate even higher resolution reference, alter that part of the image to increase its level of detail. The same approach would be needed for the human hair or the animal's fur.

Now imagine a database to hold all of those higher resolution details. Not only would it need to contain a prodigious number of samples from all kinds of natural subjects, AI would need to match the necessary reference to a specific part of the scene.

What would happen, for example, if the scene contained a lace fabric?

And what about parts of the scene that are beyond the DOF range of the original image? If you sharpened them and added detail it would destroy the bokeh.

This kind of AI doesn't look like a practical solution.

Reply
Oct 5, 2021 17:02:23   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
selmslie wrote:
...What would happen, for example, if the scene contained a lace fabric?....


I believe you can see what the answer to that question might be by comparing the blown-up sections of the SR3 (ours) with the reference.

First example, the face: I see structural differences in the hair on the left side of the image (right side of the face).
Second example, the leopard: I see pattern differences in the spots just above the leopard's nose (and elsewhere).

I believe this confirms my contention that the AI adds things that are not there.

If the structure of the hair were important (I wouldn't expect it to be in a portrait) then the improvement would be a failure. But the result is a good-looking image.

I can expect that the pattern of spots on the leopard could be used to identify an individual leopard. If that were the purpose of the image, the improvement is a failure because it changes the look of the individual.

So a piece of lace in the reconstructed image could easily have a different pattern than the reference image.

Reply
Oct 5, 2021 17:13:29   #
selmslie Loc: Fernandina Beach, FL, USA
 
DirtFarmer wrote:
I believe you can see what the answer to that question might be by comparing the blown-up sections of the SR3 (ours) with the reference.

First example, the face: I see structural differences in the hair on the left side of the image (right side of the face).
Second example, the leopard: I see pattern differences in the spots just above the leopard's nose (and elsewhere).

I believe this confirms my contention that the AI adds things that are not there.

If the structure of the hair were important (I wouldn't expect it to be in a portrait) then the improvement would be a failure. But the result is a good-looking image.

I can expect that the pattern of spots on the leopard could be used to identify an individual leopard. In that were the purpose of the image, the improvement is a failure.

So a piece of lace in the reconstructed image could easily have a different pattern than the reference image.
I believe you can see what the answer to that ques... (show quote)

Altogether the entire concept appears to be academic.

It kept some very bright people employed but it’s just not going to make it in the real world.

Reply
 
 
Oct 5, 2021 17:16:34   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
selmslie wrote:
Altogether the entire concept appears to be academic.

It kept some very bright people employed but it’s just not going to make it in the real world.


Not sure about that. Photoshop's sky replacement has a couple bugs in it, as does Content Aware Fill, but they are very popular.
As with anything, you have to be aware of the limitations.

I am reminded of the time I got my first backhoe. I found that with any new tool you will find uses for it that it was never designed for.

Reply
Oct 5, 2021 17:21:56   #
selmslie Loc: Fernandina Beach, FL, USA
 
DirtFarmer wrote:
Not sure about that. Photoshop's sky replacement has a couple bugs in it, as does Content Aware Fill, but they are very popular.
As with anything, you have to be aware of the limitations.

I am reminded of the time I got my first backhoe. I found that with any new tool you will find uses for it that it was never designed for.

Capture One also does a fine job with spot removal and clarity but the point of this exercise was to provide a more general solution. The examples provided don’t do that very well.

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Post-Processing Digital Images
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.