In the 50's I worked for a man who had an E. E. degree from the University of Missouri.  This is the way he taught me to do it.

This method will work on idlers of one diameter, not two on one shaft as in the RCA 45 changer.  Please note that the diameter of the idler has no effect on the speed of the turntable.

You must first have access to a lathe, even a small one will do.  First object is to pick a screw the size of the hole in the idler and a nut to match.  Install the screw and draw the nut tight on the opposite side of the idler, chuck the threaded part in the lathe as close to the chuck as you can get it.

The simple way is to use a tool post grinder turning the opposite direction from the headstock.  Another way would be to attach a hand grinder with a small grinding stone to the tool post in a sturdy way.  You don't want any movement.

If the rubber is not hard or dry you can then traverse the small grinding wheel across the idler, taking a small bit of the rubber off each time.  After a few trips across the idler the surface will be a matte finish and will work nicely.

If the rubber is hard or dry we must go another step.  The original idler rubber must be removed until an "O" ring (obtained from your local hardware store)  will just fit.  The "O" ring is glued into place and allowed to set.  When it has set the idler is again chucked in the lathe and the rubber removed until half of it remains.  This will give you the greatest surface area.

Clean the turntable inside rim and the motor shaft with acetone and reinstall the idler.  You are finished.