I know there is a card listed in the FAQ that does exactly this but in the graveyard, I don't remember one from the library, but the ruling would be transferable.
From the Arkham Horror FAQ
Q: A Graveyard encounter summons "all Undead monsters in Arkham, the Outskirts, and the Sky" to the Graveyard. If this would bring the number of monsters in Arkham above the monster limit, which ones are sent to the Outskirts?
A: Any monsters that were in the Outskirts when this encounter was drawn are the first to be returned to the Outskirts if the monster limit is exceeded. In the event that not all monsters that came from the Outskirts need to be returned there, the first player decides which monster tokens are placed back in the Outskirts. If more monsters still need to be moved to the Outskirts, move the monsters that came from other towns (Dunwich, Kingsport, or Innsmouth) to the Outskirts until the number of monsters in Arkham is again at the monster limit. In the event that not all monsters that came from another town need to be returned there, the first player decides which of those monster tokens are moved to the Outskirts.