RHF gives a great summary here, and I'd like to build on it a bit. I can't speak to the history before the late '90s, and will leave that to others to explore. For the purposes of this analysis I'm going to break things down into three time periods: The Steroid Era (late 90's to late '00s), the Open Draft era (late '00s to 2013) and the current era (2013-present).
I do agree with the premise that the Red Sox have fewer blue chip players right now than any time in recent history, but I think there is a clear explanation as to why this is that has little to do with organizational incompetence.
To start, let's look at each of the eras and the ways in which the Red Sox maintained an advantage:
- Steroid Era: This time period was characterized by fewer limits to spending and a significantly smoother aging curve due to substance use. I also wouldn't be surprised if players took longer to develop in this era due to a lack of some of the advanced training systems and analytics that we have today. Players typically peaked in their late 20's and sustained that peak longer into their 30s. These factors combined made free agency a MUCH more viable path to talent acquisition for a high budget team. The Yankees are best known for leveraging this approach, but for much of this time period the Sox weren't too far behind. The Red Sox developed some marquee players during this time period (Nomar, Tek) but traded for others and then locked them down with market rate contracts (Pedro, Schilling, Manny, later Beckett). Additionally, the Sox got lucky with Ortiz. Talent evaluation surely played a role, yes, but having a scrap heap acquisition turn into a franchise level superstar was a very unlikely outcome.
- Open Draft Era: By the late '00s, the aging curve was getting steeper and fewer players were extending their primes deep into their 30's. Player were also starting to develop a bit more quickly. Budgets were starting to feel the pinch of externally imposed restrictions. As a result, the game shifted younger and free agency started to become less attractive in comparison to developing young talent. During this period the Sox still tried to sign some marquee players but those contracts blew up in the Sox's face (Crawford, AGon). Luckily the organization was able to use their cash to spend more on the draft, which led to quite a lot of talent coming up through the system. While the success of the early 2000's was acquired in trades and market rate contracts, the success of the late '00s and early '10s came more on the shoulders of players like Ellsbury, Lester, and Pedroia who came up through the system. There was definitely a bit of luck here, too. I doubt many saw Pedrioa as having star potential early on, and he far exceeded expectations.
- Current Era: Starting with 2013, the slot system closed off another way for high budget teams to gain an advantage. Instead, focus shifted more toward the international free agent market. Trends that have made free agency less attractive have continued, with players peaking younger and declining younger. The Sox continued to graduate star players from their farm (the killer B's, Devers, etc.). The league also got more competitive as the balance of power between low spending and high spending teams narrowed thanks to a great many factors (plus the increase in available playoff berths). The narrowing set of levers to push to get an advantage led the Sox to push harder at the mid to late 2010's contention window. Dombrowski leveraged the farm to acquire MLB level talent, which has resulted in the system graduating very little in the late 10's and early 20's. Furthermore, the Sox lost a year of IFA spending AND suffered the tragic death of their top international prospect in Daniel Flores who could easily be a superstar player today if things had gone differently. It's also worth mentioning that Betts was a bit of a stroke of luck in the vein of Pedroia and Ortiz.
So in summary, as RHF said, the Sox's ability to leverage financial advantage has narrowed over time, which has made it more and more difficult to guarantee a steady supply of marquee talent. Furthermore, some recent events have resulted in one of the main areas where a team can realistically get an advantage (IFA market) being less productive for the Sox recently. The Sox were also lucky in getting several star/superstar players out of nowhere (Ortiz, Pedroia, Betts).
Right now the Sox are lacking in star power. If they had splurged in free agency they might have fixed this temporarily, but it likely would likely have just kicked the can down the road. Hopefully the relatively strong farm system will fix this problem by graduating enough star power to buoy the team in the medium term future. In the meantime we may just have to deal with a temporary focus on value players fueling up for the next push.
