Assuming the 24-105 is a decent lens, I would recommend it over any fixed or prime lens. The zoom will allow you to do your cropping in camera, resulting in better prints than a highly cropped image from a fixed lens.
andrew.haysom wrote:
I have both of the lenses you mention, but my favourite portrait lens is my 100mm f2.8 L macro lens. I agree with the previous posters that the 50mm is too short. The 24-105 came as the "kit" lens with my 6D. Definitely a class above the usual kit lenses, but does suffer a bit with distortion and CA. I find most images need a bit of post production with this lens.
So, if you have any interest in macro, add the 100mm macro to your list to consider.