would I need to take the drum off?
Probably, thats how its done on the standard size GE dryers to remove the drum bearing. Not a big deal once you have it apart.
Slides and the nylon drum can be replaced, both are common wear items. If a slide goes metal on metal it can damage the drum...