02 Hyun Sante Fe Cam Chain Guide

Hey all!
So here is the quick and dirty. I have an 02 Santa Fe, v6 2.7 L (FWD only) and one day when turning over my engine, i hear some catastrophic cacophony from the engine. Low idle, weird noise, almost stalls. Quickly shut her off and get to investigating. Start with the valve cover because its time for a new gasket anyway.

On the back (right bank) bank I find a rectangular piece of dried out rubber lodged by the cam chain. Some investigation reveals this is the rubber that sits on the lower cam chain guide.
I guess that explains the racket, but I realize to get at the lower cam chain guide, Ill have to remove the cam shafts and I just recently did my timing belt and not looking to recommit to that again.

I check the front bank and find the upper cam chain guide no longer has any rubber on it at all.

My question is this: Do i need to replace the guides? Is that 2mm of rubber on the chain guides essential to a smooth running engine?

Thanks for all your input!