Oblivion.
That's how I started.
Looks good, plays good and presents a lot of TES lore. Shivering Isles expansion has the best TES setting and characters so far, at least in my opinion. Be sure to get GOTY edition, as it contains the original game and both expansions for a lower price. Original game's setting is a bit generic, but that's because Cyrodiil (the main province, in which TES4 takes place) is a normal human land. But as I said, expansions will take you to some strangely wonderful places.
Two expansions, already mentioned Shivering Isles and Knights of the Nine plus some DLC (included in GOTY version).
Oblivion introShivering Isles expansionGOTY edition trailerThen Skyrim which plays even better, has the best combat system, wonderful immersive environments and takes place 200 years after the Oblivion crisis. Nords, snow, ancient tombs, mountain peaks, huge caverns and dragons. Has a small house DLC Heartfire, Dawnguard expansion (vampires) and Dragonborn expansion (available only for Xbox right now)
Skyrim trailerDawnguard expansionDragonborn expansionMorrowind last. It has an unmatched setting, originality and story (happens around 7 years before Oblivion, irc). But it's slow and outdated and you won't get that much fun these days if you're not familiar with TES lore. If you are, it's a blast. A huge number of TES fans will say that this game is the best part of the series.
Also has GOTY edition which contains both expansions, Tribunal and Bloodmoon.
Morrowind trailerIt's also worth noting that all of these games have a huge modding community.
And Oblivion, in particular, has a huge total conversion named Nehrim, which takes place in a completely different universe.
Nehrim - At Fate's Edge