I rock climb. A lot. Over 100 days a year for the past several years. This is all outside either on rock or ice, not in the gym, all over North America. You want to start off with some solid basics. Buy the following:
Black Diamond Momentum AL Harness. You will find this for under $50. It is a great harness. I have bought several of these, and it is still my go-to harness.
Black Diamond ATC Guide. Buy the Guide over the regular model, in case you ever get into trad climbing.
A locking carabiner. The larger the better for a belay biner.
Pair of shoes. Do not order these online. You need to go in and try them on. If you're really cheap, go try them on and then order them online. Every brand fits differently. 5.10 and La Sportiva are the two big brands. Most people have either a 5.10 foot or a La Sportiva foot. I wear 5.10 personally. You can go with Mad Rock or Evolv and get very cheap shoes. Do not buy something super aggressive that will hurt your foot. Just go to a store and buy something that they suggest.
Chalk Bag - this does not matter at all. Just get any one. They are all the same.
This is all you need to get going. Rock climbing is probably the hardest sport to get into. People don't want to take out beginners because they can't climb well enough to make it fun, and the trust element isn't there. In general I do enjoy taking out beginners and showing them the basics. I saw you're in SD. If you were in NY we could make something happen.
It is finally ice season now, or at least almost. I went out yesterday and played around on some early season ice. Very thin and scary, but a sign of good things to come. Definitely go to a gym. You need to be social and get to know people. They are your portal into real climbing, unless you have thousands of dollars to hire a guide again and again. Hell, if you have thousands of dollars fly me out to SD and we'll go climb the Needles and Spearfish Canyon.
Also, BE GENEROUS. When someone finally does take you out, you will be using thousands of dollars of their gear if you are trad climbing. You will be putting wear on their rope. They will be guiding you the first few times. Pay the gas money. Buy the beer. Pick up the dinner tab. Bring a nice snack to share. Basically, be their bitch.
To give you some idea of the money commitment down the road, a basic sport climbing setup will run you about $250. A trad setup will run upward of 2K. Climbing is a perpetual money pit, but the gear lasts forever (other than the rope and shoes). I have amassed a shit ton of gear. 40+ cams (all C4s and aliens), sooo many ropes (most retired now sadly), 40+ draws, webbing, GriGri 1, GriGri 2, 2 mini-traxions, Cinch, pulleys, hundreds of carabiners, jugs, croll, three sets of stoppers, set of offset stoppers, double set of RPs, gear sling, 17 ice screws (all BD express), Petzl Nomic ice tools, La Sportiva Nepal ice boots, Grivel G14 crampons, ice clippers, 5 harnesses, countless backpacks, a tag line, 6 pairs of shoes, many different jackets and other outerwear, Black Diamond Spectre, ice clippers, two chalkbags, several nut tools, three ATCs, two cordalettes, two helmets, and I'm sure I'm forgetting a lot. I can finally walk into the gear store and not want anything on the shelf...mostly. Of course a veritable library of guidebooks for all the different areas. It' a lifestyle, and a damned good one.
