If I rent someone for a large amount of money, but they have no money, or properties, and all they have is a full set, can I take the set as payment?
There is nothing that prevents properties in a monopoly from being used to pay for rent, but the decision belongs to the paying player.
You can pay with cards from your bank, properties, or a combination of both. You choose how to play - not the player you're paying!
The Sly Deal and Forced Deal cards have exceptions for properties in a set; rent does not have this exception. So, if a player must pay you rent and all they have are properties in sets, they must pay you using property from those sets.