I agree. Every match Federer ever played against Djokovic/Nadal he was 4 years older and slower.The H2H record for Federer against both of these guys isn't quite a fair comparison in my opinion - primarily because their peaks just didnt really match up. From 04-09 when Fed was at his absolute prime Nadal and Djokovic simply weren't getting to finals all that consistently. He was 1-1 against them in that stage in hard court slam finals (win over Djokovic at 2007 US Open, loss to Nadal at 2009 Australian). For example he is 11-9 against Nadal on hard court...something tells me if he had faced Nadal more at those hard court slams earlier in Nadal's careers while at the peak of his own , the H2H would look a little better. In a way he is almost penalized reputation wise for being the 2nd best clay courter of that period by routinely making the finals and having to face Nadal. Conversely, Nadal didn't make a hard court slam final until winning 2009 Australia.
All that said, splitting hairs about just how dominant these trio has been for my entire 'adult' life (I graduated college in 2004), is fun but somewhat fruitless in my opinion. At the end of the day you can make valid arguments for any of them being the GOAT (as opposed to say, Brady vs Montana...)