I don't disagree that people keep getting hired away, but the backfill in recent years leaves something to be desired, and also the size of the staff is not that big. Compare the Patriots to the 49ers:
https://www.patriots.com/team/coaches-roster/
https://www.49ers.com/team/coaches-roster/
Outside of BB the Patriots have 17 coaches. Outside of KS, the 49ers have 25 coaches. Of those 17, 2 are his kids and 2 are former players who never played for anyone else (Troy Brown, Jerod Mayo). They may all be terrifically smart and talented, but if you had to guess which team was trying to be more innovative and source ideas from many different places, which team would you pick? The Chiefs have about 26 coaches on their staff. The Eagles 23. And you'll see these teams routinely pick up ex-coaches with a lot of experience with other teams (Steve Spagnuolo, Anthony Lynn) and put them into supporting roles. Is that the only way to do it? No, the steelers have 17 coaches, too, but if we're talking about the teams that are literally changing the league with roster construction and offensive concepts, those teams would be at the top of the list.
This isn't an historic issue with BB - it's just the last few years, it feels like they're running out of a bench and that's caused a brain drain across the organization. Again, I'm not saying it's arrogant or even that these coaches are incapable of doing their jobs, I just think any org would suffer if a lot of people leave.