Tried it a long time ago... biggest problem was that I really needed a good three inches of ground clearance (at least), while racing karts have typically less than one inch. Just mounting big wheels and tires to get that much ground clearance causes problems with kart component clearances... realize that your chassis lifts only half as much as the difference in diameter of the old wheel/tire compared to the new wheel/tire, so to gain at least two more inches of ground clearance you need a new wheel/tire that is at least four inches bigger (in diameter) than the wheel/tire sizes of the racing kart.
A lot of those wheel/tire assemblies that might be big enough are not made to fit on racing kart axles, so you need to figure out adapters and will probably have to fabricate new front spindles. If you're willing to fabricate there are a lot of modifications that you can do to gain ground clearance, but the time and materials involved may make buying a yard kart cheaper and easier.
Also had trouble with the rear gear digging into the dirt, and aluminum gear halves bending and breaking. Can combat that somewhat by changing to the smallest rear gear you can find, but then you really lug down your engine.
We found it easier overall to level off a section of the back yard and make our own track. That allowed others to bring their karts and play too.