I was so sure during today's game that the torch could destroy an altar but when I looked through the rules I couldn't find it anywhere. Have I only dreamed it or is it there somewhere in the rules?
There is no such rule that I've been able to find. I've checked both the rules and the scenario setups (both player and Keeper).
Allowing an altar to be destroyed would make the game significantly harder for the Keeper.
To my knowledge, the torch doesn't destroy any other objects, it just sets fires which are a significant environmental issue for the players to deal wih.