If you wonder about the origin of the terms:
Portrait - paintings/portraits of people are almost always a rectangle or oval with the long axis in the vertical. People are taller than they are wide - mostly You will see landscape oriented paintings of groups of people or people reclining.
so vertically framed photos are "portrait"
Landscape - landscapes are mostly painted to show the width and sweep of the land so they are horizontal but you will see some things like waterfalls done in the vertical
so horizontally framed photos are "landscape"

A "mode" usually refers to a dial choice (a pre-set, if you will) on your camera and its effects will be described in your user manual. Often "portrait mode" will give a shallow depth of field - blurring the background - and do a bit softer focus, as well as more gentle color saturation. These are qualities one might desire when taking a traditional portrait (head and shoulders) photo of a person.

The physical turning of your camera to vertical/portrait orientation is used such as you mention - for waterfalls, trees and anything else with a tall aspect: buildings, flowers on long stems, and much more. I shoot a fair number of landscapes in portrait orientation - aided by a rotating LCD screen

I think many newbies don't consider portrait orientation all that often, so it's great you brought it up!