Two points:
First, if everything is affected, then so are your measuring systems, i.e., no way to check.
Second and more important, it is the fabric of space itself that is suppose to be expanding. But mass affects the local rate, so first, galactic clusters will hold together, but other clusters will vanish over the edge so to speak ( so far away and expanding so fast, their light will never reach us ). Then the clusters will separate leaving isolated galaxies. This will continue until we would see only our solar system, the it disintegrates, then atoms are ripped apart. Call the big rip. All ends as a cold, empty ever expanding space.