From what I see in your sample images, you don't have an autofocus problem with your lens. But I can't be sure because your target setup and techniques are wrong, for an autofocus test. Previous responses are right, but don't give much detail on how to correct what you are doing.
First, set up your target at a distance 50X the lens focal length. A 50mm lens x 50 = 2500mm or about 8 feet. So at 200mm your target should be about 32 feet from the camera and lens. At 100mm your target should be about 16 feet from the camera. (Note: this is not always possible, with very long focal lengths... but most recommend a miniumum of 25X focal length with super teles.)
Target needs to be in good light and have strong contrast to optimize AF's ability to lock onto it.
Next, the scale on the right should be set at a 45 degree angle relative to the plane of focus. The "0" should be the axis point and should align with the larger "target" area on the left. That target area should be set up perfectly parallel to the plane of focus. The scale set at an angle will show depth of field and whether lens is front- or back-focusing, and if so by how much.
Now put your camera on a tripod and lock it down aimed at the target. Set the lens to its largest aperture (f2.8 or f4, depending upon model), remove any filter and install the lens hood. (Filters can mess with focus and overall image sharpness, giving the impression of missed focus. Lens hood can reduce oblique light striking the lens and help insure fast, accurate focus.)
It really doesn't matter what ISO you use. However, if using a low one that's causing slower shutter speeds, you also may want to do mirror lockup on the camera and use either the self-timer delay or a remote release to trip the shutter, so that you aren't touching the camera.
Next, enable AF on the lens and set the camera to One Shot mode, then select Single Point and choose the center AF point. Put that point on the target and activate AF (half-press the shutter release or use AF-On button if using Back Button Focusing method). Take a shot with this focus.
Now switch the camera to Live View and, while looking at the distance scale on the lens, redo autofocus. If the distance scale moves at all, then autofocus needs to be adjusted (either by calibrating the lens itself or by adjusting it using Micro Focus Adjust feature on the camera, if yours has it.) Take another shot after Live View focusing.
Now carefully turn off AF on the lens and manually focus, using Live View. You can zoom in with Live View on most Canon cameras, to check focus accuracy (this doesn't zoom the image itself... just allows for closer inspection when manual focusing). When you are satisfied you have achieved the best manual focus possible, take another shot.
The three shots taken should be compared to each other. Ideally, they should all three appear to have the exact same focus. If so, you don't have any problem with autofocus. If the lens changed focus between the first and second/third shots, then the viewfinder based, phase detection AF needs to be calibrated (either the lens itself or with MFA in the camera). This is because the contrast detecting AF of the Live View mode tends to be the most accurate, best to use for really critical focus, although on most Canon models it's too slow to be practical for any sort of moving subject (Note: Live View Dual Pixel Focus on 70D and now 7DII is much faster and may be more widely usable, though I can't say from personal experience, haven't used it yet.)
Optionally you might do a fourth focus test and take another autofocused shot with the camera set to AI Servo instead of One Shot. But this really is just for comparisons sake... to see differences between them (which will be slight... AI Servo isn't quite as accurate as One Shot).
Be very careful not to touch the focus ring or zoom ring while doing the autofocus shots (of course you have to touch the focus ring while doing the manual focus shot).
All the Canon 70-200s are classified as parfocal lenses, meaning that they should maintain focus when zoomed. However, although close, IMO they really aren't. Like most modern zooms they are varifocal (only slightly, in the case of the 70-200s). This means any change in focal length requires re-focusing afterward, for maximum accuracy (using AI Servo as default mode of focusing makes this automatic... using One Shot requires consciously re-focusing... see below.) So be careful to not change focal length of the zoom, between your focus test steps. You may want to test the zoom at several different focal lengths, just be sure to leave the lens set to that focal length for all shots during the three or four done for the test.
Some advise turning off IS on the lens when doing autofocus tests. This really isn't necessary on the 70-200s, since all IS versions of it that Canon has made utilize the type of IS that turns itself off automatically when there's no movement to compensate (i.e., when the lens is locked down on a tripod). It's up to you, whether or not to turn off IS manually. Or, you might try it both with and without it on, to also check if IS if working properly. Some other Canon lenses (i.e., 300/4 IS, 28-135 IS... and reportedly 100-400L and 24-105 IS) do need to have IS turned off when locked down on a tripod... or IS will go into sort of a feedback loop where it creates movement, when there is none. You actually can see this happening in the viewfinder, though. So it's easily avoided if one forgets to turn off IS or it starts occurring on other lenses.
Also note that for autofocus tests you should use the lens wide open, as this renders the shallowest possible depth of field and that in turn will best illustrate any missed focus issues. However, many lenses are not at their very sharpest when wide open... The 70-200/2.8 non-IS and the first version of 70-200/2.8 IS among them. On the other hand, both the Canon 70-200/4s and the latest 70-200/2.8 Mark II are pretty darned sharp at their max apertures. Keep this in mind and don't misinterpret some minor loss of sharpness due to the big aperture being used as a missed focus issue. Apparent image sharpness also can be adversely effected by things unrelated to the lens itself or focus accuracy, such as a filter on the lens, atmospheric conditions, or a camera's image sensor sorely in need of cleaning.
Also keep in mind that all digital images need some sharpening in post-processing if shooting RAW, and/or in-camera if shooting JPEGs. "Straight out of the camera" concerns me.... does that mean "no sharpening applied"? If so, then I wouldn't expect an image to be critically sharp and that it would have some overall softness, much like your images seem to have (plus, there really is no such thing as "straight out of the camera"... all images have some default processing applied, whether it's in camera or afterward in post-processing).
With a zoom, I'd test it at the extremes of it's focal length range. However, only recent Canon camera models allow for proper Micro Focus Adjustments with zooms, if they are off by different amounts at the extremes. The latest 40-lens version of MFA (70D, 7DII, 6D, 5DIII & 1DX allows for two adjustments per lens, with zooms. The earlier 20-lens version of MFA (50D, 7D, 5DII, for example) only allows for one adjustment even with zooms, so it may be necessary to compromise with an averaged adjustment or to only adjust for the extreme that you use the most.
Some things you need to keep in mind when shooting for real, rather than for tests...
1. Stopping down a little increases depth of field and gives more leeway for minor focus errors. It also optimizes lens sharpness in many cases. Unless going for heavily blurred backgrounds, shallow depth of field effects or really struggling for enough light, why not stop down a little giving AF a break and using the lens at it's best?
2. Try using only the center AF point as much as possible. This way you are putting it right where you want the lens to focus. (If letting the camera chose the AF point, it will usually focus on whatever is closest and covered by one of the AF points in the array... such as the nearest wing tip of a bird flying past, rather than the eye of the bird. This isn't ideal for focus with unforgiving, large aperture lenses.)
3. Be certain you are using AI Servo with moving subjects. The audible "beep" of Focus Confirmation is a dead give-away that One Shot is in use... and I hear other photographers' cameras beeping away a lot, when I'm shooting sports/action alongside them... Sorry, but most of their shots will not be in focus due to the wrong focus mode, though they're likely to blame the camera and lens instead of take responsibility for their mistake.
AI Servo is continuous focus... the camera keeps adjusting focus on moving subjects as long as you keep the shutter release button half-pressed or your thumb pressing the AF-On button. AI Servo does not (actually cannot) give Focus Confirmation, so there is no "beep" when using it. AI Servo also isn't quite as accurate as One Shot.
However, One Shot achieves focus, then stops and gives Focus Confirmation. If the subject (and/or the photographer) is moving, the focus distance is most likely no longer correct before the shutter can be released. So One Shot is great for stationary subjects, but not really usable with moving ones (except for pre-focus technique, where you know where the subject is going to go and pre-focus on a certain point, then wait for the subject to arrive there).
Many Canon cameras also have AI Focus mode... which really isn't a third focusing mode at all. Set to AI Focus, the camera is supposed to decide for you whether or not the subject is moving, then will switch to use the appropriate mode, either One Shot for stationary subjects or AI Servo for movement. However, there's a short delay while the camera makes this decision, plus I've found it doesn't always choose correctly... or the subject begins moving after the camera has already made it's choice. Anyway, I avoid using AI Focus at all, on my Canon cameras. Recommend you do the same. Make your own choices, between One Shot and AI Servo. (It might be a clue that the most pro-oriented Canon models don't even have AI Focus mode... they only have One Shot and AI Servo modes.)
4. Back Button Focusing is a popular shooting technique, especially with birds, wildlife in general, and with sports/action shooting. You might want to try it, if not already using it. Basically BBF just separates the autofocus function from the shutter release button, putting the photographer more in control of focus. It also makes possible focus-and-recompose while in AI Servo mode (which should be avoided when using shutter release control of AF). BBF allows AI Servo to be used as one's default AF method... working well for either moving or stationary subjects. Read more about it here:
http://www.learn.usa.canon.com/resources/articles/2011/backbutton_af_article.shtml5. All Canon DSLRs offer at least two focus patterns: All Points/Auto Selection and Single Point/Manually Selected. As mentioned above, for critical focus it's usually best to use Single Point. Some models have additional focus patterns such as Spot Focus (High Precision Single Point), Zone Focus, Expansion Points and more. If your camera has these, it's important to study how they work to know when to apply them, and perhaps more importantly, when to avoid them!
Also, most Canon cameras have at least one enhanced AF point... at the center. It's the most sensitive and responsive. Some models have other high performance points elsewhere in the array, though they often aren't quite as good as the center one (or several points at the center, such as with the 5DIII and 1DX). Find out what your camera has, to know how to get the very best out of it. In addition to tech info, you might run some focus tests with the different points, just to get a feel for yourself, the differences between them.
Hope this helps! In conclusion, just looking at your sample images I can't see any focus problems... But looking at the largest versions I do see some overall softness that may or may not be due to missed focus issues. Can't really be sure of the cause, based upon how you set up your tests. So redo your tests to see if there really is a problem. (But I kinda doubt it... I own and use two Canon 70-200s, one of which is now nearly 15 years old and has seen a lot of use, and have used others on occasion... They have all been fast and accurate focusing lenses that I've never seen a focus issue with.)
Edit: Bill's shots shown above a great example of the value of MFA! But anyone who shoots with a 600/4
handheld is either nuts or can give Arnold Swarzenegger (sp?) a run for his money! ;)