During one of my first portfolio reviews, the reviewer made an observation. "So many of your photos are in a horizontal format. Try including more verticals," he said to me.
I hadn't thought about it too much up to that point but he was right; I did shoot a lot of horizontal format photos. It is an easy mistake I suppose since our cameras are lined up that way. And at the time my camera only had the one shutter button. (I now have newer camera bodies that include a shutter button for the vertical orientation.)
Since then I make a point of spinning my camera around getting vertical shots as frequently as possible for each scene I am shooting. In particular situations, verticals actually work better than horizontals. For example, if an animal has approached fairly close, a horizontal format may no longer provide enough vertical space to include a full head or the appropriate amount of the animal's body.
Another piece to consider for shooting vertical and horizontal photos is the use of the photo. Will the photo be submitted as a potential magazine cover requiring a vertical composition, or an interior double-page spread requiring a horizontal format? Is the frame you plan to place prepped for horizontal or vertical?
And sometimes the mood of a photo can change with the orientation, such as with these two images of a bison cow with her calf. In the vertical image, the focus is purely on the cow's activity of cleaning her calf. In the horizontal image, the photo has more of a feel of where the animals are located and what is around them.
In the case of landscapes, if you are using a polarizer or neutral density filter, remember to spin that too to get the correct orientation.
Have fun trying out new formats. Remember, it is only a few more pixels to try it. If you don't like the outcome, the file can be easily deleted.