Joshua has suggested he is "coming towards the end of my career" and hinted that he will have a maximum of "five years left" in the sport.
But Whyte views this admission as a sign that Joshua no longer has the same burning desire for success.
"Maybe he's comfortable, maybe his drive and desire is going down," Whyte told Sky Sports.
"He seems to be fighting a lot more defensively than he used to do. Maybe he's thinking, 'You know what, I haven't got long left, it's a hard game.'
"When you achieve everything you've set out to achieve and you've got lots of money, sometimes you think, 'You know what, let me just retire and go and do something else.'
"He doesn't strike me as someone who loves fighting. He loves the business and he loves the opportunities.
"Obviously he's a good athlete and a good boxer, but he doesn't strike me as someone who loves to fight. I love fighting. I enjoy fighting for the sake of fighting. That's why I'll fight anyone and I'm always ready to go."
The Brixton man is preparing for a rescheduled rematch against Alexander Povetkin, with the winner propelling themselves back into world title contention.
Joshua is currently in talks over a two-fight deal with WBC champion Tyson Fury, although Whyte still expects to feature in the future plans of the WBA, IBF and WBO title holder.
Both men hold one victory apiece, with Whyte triumphing in the amateurs on his debut, being both the first to knock Joshua down and beat him, before Joshua's professional victory in 2015.
Asked whether Joshua will want a third fight, Whyte said: "He definitely should do. Why shouldn't he? Currently, it's one all. I look forward to the third fight in the future.
"However, I can't say what he wants and what he doesn't want."