I have totally removed the cat and baffles with no negative performance issues. I will say that if you have totally removed the baffles and are planning on getting an aftermarket exhaust in the near furture I wouldn't worry about the Cat. If I was charging myself labor it would've cost me twice as much to remove the Cat than to just buy an aftermarket exhaust
Seriously it's a pain in the ass to remove it all, or at least it was for me and took longer than I planned for. Not trying to discourage you but just make sure your preppared for the task at hand and it helps to have at least a six pack on stand by ;D