Getting Started for Dirt Cheap—12, 20, or more $$

Notes:

The rulesets I recommend are cheap compatible with the majority of OSR stuff, and safe picks for any group. Some more exotic or wacky, but free, rulesets are given at the end of the post. Check them out, one might grab your eye.

Taxes, I did not factor in taxes. Usually a couple extra bucks.

Shipping, if included, is for WA, USA.

Amazon, if you don't have prime (free shipping), ask a friend.

Library Printing (LP), usually ~$0.10 a single-sided page. They may also have staplers and the big guillotine things, sometimes card-holders get a free printing allowance.

Packing Tape, put strips over paper, now you can use dry-erase markers.

One page dungeons, print some.

Magister of Miserliness ~$12.00

• Dice 
• $3.89, Single set on Amazon, ~$5.00 at dollar general.
• Ruleset, one from: 
• $1.40 LP, Basic Fantasy Role-Playing Game Beginner's Essentials
• $1.12 LP, Cairn Booklet
• Adventure, one from:
• $3.95, Morgansfort: The Western Lands Campaign (BFRPG)
• $3.75, The Chaotic Caves (BFRPG)
• $2.20 LP, Tomb of the Serpent Kings
• Misc, 
• $1.00, Lined notebook available at dollar general or college stores.
• Find or steal a pencil.
• Extra
• For ~$2.00 - 3.00 over budget, you can get a roll of wrapping paper that has a 1 in. grid on the back (dollar store / walmart). People often have extra wrapping paper you can ask for (and you'll also know if it has a grid).

Considerations

• The BFRPG essentials only covers 1st level. The core rules are $5.50 on amazon.
• There's no playmat for trve Champions of Cheapness, use your imagination or battle-sketches.

Champion of Cheapness ~$20.00

Entering the hobby at this tier of frugality, options are a little more varied:

• Dice
• ~$10.00, Get multiple sets of dice. They come in 5-7 sets of 7 dice with a bag for each set.
• Ruleset, instead of above rulesets, order one of the following:
• $4.12 LP, Stick with Cairn and print out 30 more pages at the library.
• $4.29 White Box: Fantastic Medieval Adventure Game
• $5.50 BFRPG Softcover
• $6.50 LP, Old-School Essentials: Basic Rules
• $6.30 LP, BLUEHOLME™ Prentice Rules
• $6.40 LP, 1981 Moldvay Basic Set
• Adventures
• I'd still recommend one of the BFRPG adventures.
• $6.18 (w/ cost of PDF) LP , The Black Wyrm of Brandonsford 17 pages, great module but a little light.
• Misc.
• Macgyver a playmat with packing tape and wrapping paper (or drawn grid).
• Print out a zine or two for your chosen system.
• Get a 3-ring binder, try to round up some graph and lined paper.

Parsimonious Peasant ~$35.00 - $40.00

At this point, you're hardly even poor. If you liked one of the free systems, just stick with it, try some neat stuff you find on blogs or buy BFRPG softcover books, they're like $4 ea.
Or purchase as a Champion of Cheapness substituting either a ruleset or adventure from below:

• Rulesets
• ~$12.80 LP, 1981 Moldvay Basic Set & 1981 Cook Expert Set
• The real B/X deal. Play up to 14th level.
• ~$21.96 (w/ shipping), OSRIC 2.2 (BW SC)
• The O.G. OSR game. A retro-clone of AD&D and compatible with a huge selection of modules. Might be too crunchy for some.
• ~$20.00 (w/ shipping) Swords & Wizardry Core Rules (Softcover)
• A 0e clone like FMAG. This one is 144 pages. PDF is free.
• ~$25.00 (w/ shipping) Whitehack Fourth Edition
• A free-form rpg that's mostly compatible with everything. Review for the previous edition.
• ~$26.00 (w/ shipping) Labyrinth Lord: Revised
• Read the free no-art pdf, decide if you want it. It's a great B/X retroclone.
• Adventures
• ~$22.00 Anomalous Subsurface Environment
• It's really great, build a campaign around it.
• $14.70 (w/ cost of PDF) LP Tomb Robbers of the Crystal Frontier

Wealthy Wretch ~$55.00 - $60.00

Alright moneybags. At this tier, you can grab one of the ~$20 rules sets and an adventure from above.

• Also
• $?.?? DCC First-Time Fan Kit It was $35 when I bought it. If it drops back down grab it and $10.00 worth of dice. Includes all you need to play DCC.

The Promised List

• $2.40 LP, Black Pudding #4 is basically a complete game in a zine. Print out a couple more issues of Black Pudding, and you'll have a blast.
• $2.40 LP, Vaults of Vaarn is a science-fantasy desert RPG in a zine, with two more expansions. Reminds me of Caves of Qud and Nausicaä of the Valley of the Wind.
• $2.80 LP, CRACK! is a bare-bones game compatible with B/X with some small supplements. Can hack it into something pretty darn interesting.