First off, who gives a flying fuck what fans of other teams think? The second the Krafts start making football decisions based on that is the second I hope they sell the team to someone who isn't that stupid.
In terms of BB, I'll admit there are things about how they have played that concern me, the sloppiness, lack of attention to detail, mental mistakes, it doesn't look much like a BB team. And of course he will lose it someday, and it is reasonable to think that decline might have already started. But given the fact that they have a putrid offense that in no way resembles an actual NFL offense, I'm not sure anyone else could get more out of the roster than he has. He obviously deserves the blame for assembling the offense and making the coaching decisions, but I do think he deserves a chance to fix that.
Given that IMO he has earned the right to stay too long rather than be forced into retirement too early I think talk of firing him is premature, but if next season they are .500 or worse again then I think he should be deservedly on the hot seat at that point.