To use a metaphor, what's the difference between a barroom brawl and a full-scale war? The brawl is faster and just as decisive at the end, but if you rush the front lines of an opposing army swinging a barstool, you'll probably not get very far. You need to rely on your commanders and your fellow soldiers to get you close enough that your barstool will actually connect.
In other words, a 9x9 game is tactical whereas a 19x19 game is strategic. Consider the following Go proverb:
The third line is for territory, the fourth line is for influence
This is the fundamental difference between a high move (fourth line or higher) and a low move (third line or lower). Territory is valuable for obvious reasons: that's where you get your score from. Influence is also valuable, but for less obvious reasons: you use influence to fight and either secure more territory, or destroy your opponent's territory.
Now, on a 9x9 board, the ratio between influence and territory is heavily biased towards the territory side; almost the entire board is on the third line or lower, and the only space above the fourth line is the center point. This makes a game on the smaller board faster, but focuses very much on close-range tactics rather than wide-scale strategy. Usually it's one quick but brutal fight which ultimately determines the winner of the game.
On a 19x19 board, however, roughly half the board is "territory" and half the board is "influence". Rather than one short but violent fight, the game focuses just as much on deploying your troops and fortifying your defenses. You'll often end up with a series of skirmishes, none of which necessarily decide the game on their own. You may lose a battle, or even many battles, yet you can still win the war.
On the larger board, playing for influence is can be just as important as playing for territory; very often a move that secures territory for yourself will allow the opponent to gain influence, and vice versa. Whether you decide to play a territory-heavy or an influence-oriented style, understanding this balance becomes essential to victory.
This subtlety is almost entirely missing on a 9x9 board.
Now whether you prefer to while away your idle hours with the occasional drunken fistfight, or you'd rather start up World War III, that's very much up to you. At my local go club, it is not uncommon to see people playing nothing but 9x9 games all day long, and other people breaking out the 19x19 board and nothing else. Often players will switch between the two boards from game to game. Both board sizes allow for enjoyable and viable games, but even with the exact same rules the games will be completely different.