When I was a young photojournalist I loved being on my own and I guess sewed my photography oats at that time. Now I can only think of travel as a married couple. My wife knows my interests and even points of interesting subjects. I travel only with a pocket camera that can handle all I need for photography. I can put the camera in a belt pouch allowing me to hold my wife's hand while out and about.
I feel I do not miss much, I am not the type of person who can speed days with a group of photographers, I am a guy who has always liked to shoot alone. (but in Florida each morning I shoot with a group of photographers but that is only for about 2 hours each morning, not the whole day and not for a specific purpose, we are together but not together, we are all shooting on our own)
I could not consider traveling anywhere without my wife at my side. I do not think I miss much.

I would have to concur, that I would find it encouraging if I could travel to places that my spouse would not in the company of other photographers who enjoy the creative photography aspect. However that will not happen, as I have never, and would never take a "vacation" without my spouse. When I was younger, I traveled the country for my businesses alone, or with associates, but never took the time to "smell the roses" when in an interesting city, as it was always business first, as it was my money I was spending. But the concept above does sound refreshing for day trips and I agree with the OP.

Muse away! I am in a similar situation with long range fishing. Stay with me. My friend and I go on long range boats to catch big fish. Over the years we talked fishing, lived fishing, breath fishing. At first, we were hot to get big fish...200 lbs plus...or at least be catching all the time. Over the years we have talked about every subject there is...marriage, kids, life. And then the big C struck him...cancer. Recently, we went on a trip and the fishing was no so good. But even when the boat was catching fish, there were times he and I simply sat and talked or sat in silence. We realized the fishing (or photography) is simply a way to experience life and to be. To be on the open sea, or in a meadow taking photographs, is a way to find peace.

I do not care for dedicated photography travel...which I have done. Ive done a few seminars and enjoyed and learned, but really had no use for the photographs I took...because they were not meaningful memories to me. When my wife and I travel, I shoot on the move and do not take a lot of time to do it. I try to have interaction with people and take their photos. It is always interesting during my photo clubs critiques of photos that my photos often need more shoots, better background, a different moment in time...which I agree with...but I do not stay in one place, holding my wife, or the group back while I try to get the exact right photo...its a bore to me and to them. So, my photos are for my memories and quick stabs at art.

But to photograph with a special friend? think of it as a fishing trip...its not the fish...its the time together...the rest is secondary.

I don't agree with you. When we travel my wife makes me take a lot of pictures of her in front of famous/far away landmark. She post the pictures on facebook just to make her friends envy. I don't think people enjoy seeing pictures from your exotic vacations.
Let put it this way, although you may say the locations are exotic but chances are they have been photographed before and one wants to see what they look like they can see them on the web or books. Seeing your pictures of those location only prove that you were there.

I believe that there can be a place for both. Many years ago, I embarked on a major road trip with three other guys to photograph significant points on the major western railroads. We were focused on parts of all four of them at the time...the Santa Fe (our primary interest), the Denver & Rio Grande Western, the Union Pacific, and the Southern Pacific.

We traveled over 3000 miles over the span of 10 days, and much of what we committed to film can never be seen again. We went off the main path and in some cases entirely off-road to capture some of our images, like the Canyon Diablo bridge on the Santa Fe in eastern Arizona.

We were a diverse group (engineer, store owner, direct marketer, and church pastor), and we had a great time, despite a couple of times that we drove way too fast to get in position ahead of a train. We were in nine different states during our trip. We drove all night twice. There's no way that any of us could have done that trip with our families. But there's also no way I would do every trip that way.

Our focus was photography. We did not do anything that was not in support of meeting our photographic goals. I'd love to do one or two more of those trips. My wife and I are planning an autumn trip to the northeastern US. We will be visiting friends also. There will be lots of photography, but it won't be like that railroad photography adventure.