Adding bumpers would increase the weight of the jeep and lower the ride therefore increasing caster. +-5 degrees of level on the front sway bar is important (due to potientially flipping it forward under full droop) but on the rear not so much, more of a clearance issue back there. If you have swapped out control arms you may need to get an alignment to make sure your thrust angle is correct, they can check your toe in as well as it is very hard to do it on your own with 1/16"-1/8". Just tightening the jam nut can throw it off spec.

 

Check your caster by removing your grease fittings on your upper ball joints. Park on a level surface and put your angle finder on top of the BJ's. It shoub be reading 4.2* -/+ to the rear. The pitman arm but on the bottom of your pitman arm. Mine was loose and caused my jk to wander. Your caster shouldn't have changed by installing a new bumper, that is affected by lifting/ lowering your vehicle

 

There are no grease fittings on factory axle ball joints so unless you upgraged to aftermarket, don't spend a bunch of time looking for them.
Caster will change due to the changes in height, adding armor and other weight will lower your jeep, so yes, adding bumpers can change your caster due to lower suspension height.

 

Any luck with fixing this yet? I'm having the same issue. I've done an alignment replaced most steering components.... Nothing.

 

Mine had a tendency to wander after I installed my after aev 2.5" lift. Their geometry brackets made huge difference. Also my tire pressure was to high and adding to the problem. Since installing my fox ats stabilizer it drives like like a dream. If you don't run the brackets at a minimum you should have adjustable front lower control arms to put the suspension where it wants to be.