I am at my cabin now (getting ready to go ride tomorrow, yea!) and pulled out my 2000 Polaris service manual. Specs for the 500 VES motor, with TM 38 carb's are as follows:
Main jet - 420, with #3 clip postion. Good for -20 to +10 and 0-900 feet altitude
Pilot jet - 40
Air screw - 1.5 turns out from seated
Pilot Air jet - .9 (this is the one that screws in the throat on the airbox side)
Looking at the jetting chart, the main jet drops to a 380 for 900-1800 feet and -20 to +10. Depending on where you are altitude-wise, you could easily drop to a 410 or 400 main jet - here in MI my altitude ranges from 700 feet and up and a 410 works fine in a 600. But I don't think that is the problem here.
I'm guessing a little, but with your air screw at 3.5 turns out, your idle mixture is too lean. The troubleshooting section says that backfires are due to a lean mixture. My wife's 2005 600 backfires on occasion just off idle.
Does the problem go away if you put the choke on? If so - mixture needs to be richer.
Regarding the clutching, just to check:
Engagement is 4000 RPM +/- 300
Operating RPM is 8500 +/- 200
If your motor isn't operating in these ranges consistently, then you may have a clutching issue. But I'm betting on the idle circuit. I would try 2 turns out from seated and see if it's any better. 3.5 turns is really out there.....
Good luck!