Yes. The key is to get common and uncommon cards, avoiding rare and mythic rare cards (Note: A year after answering this question, I wrote a lengthy comprehensive guide on how to keep cost low while still having fun with MtG. I still stand behind the following answer, but I did find out other ways to keep costs low).
Here's how:
If you don't already know the game, your buying will be much more efficient and effective if you learn it first. You can learn the rules and play some sample games on an iOS or Android device with the official app (at the time of this writing, it's Duels of the Planeswalkers). You don't have to spend any money to play a few sample duels but if you want to play all of the different duels you'll have to pay to unlock additional duels ($15 on iOS). You should begin to get a feel for different possible deck strategies.
Alternatively, if you have a friend with several (or more) decks, have your friend teach you the game using his or her decks. Be sure to try out several different styles of decks.
Read about deck types.
Armed with knowledge of the basic deck types, you're ready to go to a local store that processes a lot of cards. A good resource for finding a store near you is the official store locator.
Stores will vary in their policies so you'll have to find one that sells bulk common cards at very low cost. The store I go to lets me sift through many thousands of unsorted common and uncommon cards and pile them into a box that fits about 500 cards, at a cost of $5. It's time consuming, but I get to choose the cards.
This low card cost is possible because some stores generate so many common and uncommon cards as byproducts of events or buying collections that they can't possibly sell them all. They pick out the rares and unusually valuable uncommons/commons. The remaining commons/uncommons will eventually get recycled, so some store owners are happy to sell them for something.
The first time you do this, you will make mistakes, and you'll quickly realize that you don't have enough of certain kinds of cards and too many junky cards. That's okay. You've only spent $5. So you go back to the store a few more times. By your 4th trip you should be a lot better at knowing which cards are interesting and which to avoid. It helps to bring a list of what kinds of cards you want, including specific cards.
By the time you've accumulated around 2500-3000 cards (at least a few hundred of these will be lands and dual lands), you'll find that it's possible to construct a wide variety of decks and pursue a wide range of deck strategies. You have the potential to create hundreds or maybe even thousands of different, effective decks. However, you'll only be able to have 12-15 decks going at one time due to limited number of lands, and different decks competing for certain cards that tend to get used in many decks.
All this can be done for around $25-$30.
It is also possible to buy "fat packs" on sale for $30 or so, and then you can sell off the rares and mythic rares for $15-$45, depending on the market value of what you were lucky (or unlucky) enough to get. But doing it this way subjects you to a lot of randomness and you may get stuck with lots of commons you don't want.
If you decide to participate in tournaments (which you can do with the pauper format or the peasant format), you will likely want to supplement with a few commons you lack. So you may end up spending another $5 or $10 to supplement your several thousand card collection.
All told, it is possible to get quite a variety for less than $50, which will provide plenty of fun and variety in casual play. For a few more dollars, you can fine tune some of your decks to compete well in pauper or peasant formats.
EDIT: About half a year after writing this, I started to get serious about competing with Pauper. While still much less expensive than other formats, there is little chance of staying under $50 if you want many competitive pauper decks. Depending on how many competitive decks you want, your total cost could range from $300 to $1200 to have a wide variety of top pauper decks all built and ready to play.
Expenses for competitive pauper are higher than casual play because there is high demand for many "staple" commons that cost $2 or more, such as Lightning Bolt, Rancor, Gitaxian Probe, etc. that you are very unlikely to find in a penny commons bin. If you want to stay under $500 total you'll have to avoid cards costing well over $5 such as Sinkhole, Three Visits, Chain Lightning, and Oubliette. Most competitive Pauper decks do not need individual cards costing over $5.