On the card it states it is not part of the supply.
Is there any place in the rules that states that you can or can not?
Board & Card Games Stack Exchange is a question and answer site for people who like playing board games, designing board games or modifying the rules of existing board games. It only takes a minute to sign up.Sign up to join this community
Yes, you can.
There is nothing about being part of the supply that affects trashing; they are still "cards" and thus are affected by things that talk about cards.
The rules for trashing from the base game rulebook don't say anything that would restrict it to supply cards:
"Trash a card" - Put a card into the Trash pile.
This is a way of getting rid of a card; a trashed card is no longer one of your cards.
"Gaining" is different because of the rules for gaining itself; the rules for "gain a card" include that the card must be from the supply unless specifically said otherwise.
"Gain a card" - Take a card from the Supply and put it into your discard pile.
Sometimes a card will let you gain a card from a place other than the Supply, but by default gained cards come from the Supply.
Now, your specific example of Improve and Horses is tricky because normally you wouldn't have a Horse in play at the start of cleanup; since it returns itself to its pile when you play it. But with things like Ways from Menagerie, there are ways to do it.