I did a trial membership with Amazon Prime and didn't renew it. I seldom need fast shipping, and the entertainment content I wanted to stream all too often had additional charges to view. For pricing I look at sites such as Google Shopping {froogle.com still lives} and then look at Amazon's price(s). For me at least over the last 24 months or so the Amazon prices are looking less and less competitive. Some of that may be other vendors price matching a short-term promotion at Amazon and then being slower to raise their prices back up. Some of that may be price-matching along with offering additional discounts (places like OfficeDepot with coupons and Sears in its slog to bankruptcy with some sizable rewards credit offers). Some of that may be Amazon making it so easy to order/reorder via Prime that you stop really looking at the prices.
Add ons like Honey are good for price match comparisons; apps like Earny also good for when stuff goes down and they monitor it and you can get the difference back.
If one is willing to take the time you search multiple sites, then yes, you'll always be able to find stuff that's cheaper on different sites - I go through this with my restaurant accounts, that tell me they can get product X for $.04 cheaper per piece at Sam's Club or item Y for $.10 less through Web Restaurant or Target online. Yes, if you want to spend the time, knock yourself out, I can't be lowest on everything.
Personally, I value my time more than that. If it's a large purchase, sure, I will price shop for a TV or something like that. If it's a $4 difference on the price of a pair of sneakers, I'd rather just click twice than spend 10 minutes inputting everything into a new account, dealing with more spam email, paying for shipping, etc.
I also place an extreme value on the free two day and the ease of reordering, or even subscriptions (and their discounts) for everyday items. I could probably save money by, I dunno, going to BB&B to buy the Simplehuman garbage bags we use, cash in a 20% coupon and save a few bucks; or i could lump five other things we need into the same box from Amazon and pay a little more. I'm fine with the latter if it makes it easer and I don't need to deal with the unwashed masses.