Dont make the mistake of buying everything new when you are just starting out, have seen so many people do this only to find out kart racing wasn't really for them and they ended up losing a ton of money when they sold out everything. Find yourself a good used chassis, the newer the better, along with some good used equipment and give it a try to see how you like it. Then if you do like it, consider getting better equipment once you get some seat time and get the hang of it. One thing that would really suck would be to buy a new chassis and get caught up in a bad wreck first time out there and bend the chassis, then you are really up the creek without a paddle. Do yourself a favor and look around the classifieds on here and on facebook, you can usually find someone selling a race ready kart or 2 pretty reasonably cheap and see how you like it first before diving in. Brands to look for if your gonna be racing on dirt would be Millenium, Phantom, or Charger, those are the main 3 you see and have the most support out there for them.
One more thing to keep in mind, dont get discouraged If you go out there for the first time and get lapped or get beat pretty bad, you will be racing with people with more experience and you are new, so I wouldn't expect to go out and win races until you at least get the hang of it and get seat time. I always tell people that its best to not even worry about trying to be competitive when you first start out because very few people have that kind of natural talent. Also, try to stay out of the classes that have high kart counts and get into the class with the fewest karts until you get the hang of it, that way you aren't in the way and wont put yourself at risk of getting seriously hurt. I see new guys all the time show up and get into the biggest class out there, get beat pretty badly or get wrecked and come off track frustrated, mad, discouraged and all sorts of things.