It's safe to say that, unless you have a very specific strategy that can put deck clogging victory cards to use, you'll never want to buy them early in the game.
The most crucial aspect is your opponents deck.
If you are tuning your deck into something that might be able to buy a Province every turn, then you can hold off till your opponent buys the first Province (assuming their deck is not as good as yours).
If your opponent is ahead of you, or you can see that they will eventually have a deck you cannot compete with - time to get creative and start buying earlier, hopefully in ways that will allow you to acquire victory points steadily and not slow you down late (at which point your opponents deck will probably be even more powerful).