Instead, Warren thinks Whyte should take on another of his Queensbery Promotions stable in Daniel Dubois.
The 22-year-old, the holder of the British heavyweight, Commonwealth and WBO International heavyweight titles, will take on Erik Pfeifer and then Joe Joyce as he looks to make his mark in the 200lb-plus division.
“Dillian will get his chance, just not now, and whilst he is waiting for his opportunity, a massive fight, a big fight even if there was no crowd there and we could do it on pay-per-view, would be Daniel Dubois,” said Warren.
“That would be a great fight that people would want to see. Then the winner could fight Tyson.
“Dillian Whyte has been sitting there for I don’t know how long. It’s not my fault, it’s down to him, he made his choices.