Yes you can.
You can reform if:
- You win the round of combat
- Or if you lose combat, hold and pass an unmodified leadership test.
I believe there is a limit to how many columns you can adjust per turn (from memory I think it's 5) however there should be no problem going from 10 columns to 5.
Assuming your models are on 20mm bases this will give you 4 models from each rank in each rank attacking and a +1 rank bonus (until the monster kills another model at least!).
Where the rules begin to get a little vague is when you're "moving" your unit in the combat to get more attacks. It's best to agree with your opponent before you start moving models.
The rules are clear that can't reform out of combat - they're a little vague on reforming to get more attacks in (although bear in mind that this is what you're effectively doing when you add more ranks). In my opinion the simplest solution is to treat the centre of combat as the centre of the unit and reform around that.