You should really ask your questions one at a time, in separate questions.
Do mortgages go to the bank after three rounds?
No. The rules say nothing about this. When your friends bring up a rule you have never heard of, ask them to point to the rule in the rules booklet. If you lost yours, print a new one from here.
How much does it cost to buy back mortgages?
The amount of the mortgage plus 10% interest. A $70 mortgages costs $77 to lift. From the rules:
In order to lift the mortgage, the owner must pay the Bank the
amount of the mortgage plus 10% interest.
Can I take actions when it is not my turn?
The rules are unclear on this. Some sources say you can conduct your business whenever you want:
Other sources say you can only conduct most business on your turn:
One action most that people agree on is that you cannot buy houses immediately after a dice roll ("Hah! You rolled a 6! I'm going to upgrade that spot before you land on it!"). Also, some business must occur when it is not your turn. For example, you may be required to mortgage a property or sell houses in order to pay a debt. We will categorize this as "necessary business".
That leaves unnecessary business. Unnecessary business would include pretty much any action that you take when you are not in danger of going bankrupt. For example, buying a house, unmortgaging a property, or trading. You and the other players should make it clear when these types of actions are allowed before the game starts.