"The world has lost a great champion," said the 69-year-old Ali, who fought Frazier three times.
"I will always remember Joe with respect and admiration. My sympathy goes out to his family and loved ones."
Lennox Lewis said of Frazier: "He was definitely one of boxing's greats. He was legendary."
The Briton, a former world heavyweight champion himself, added: "He has made his mark in boxing, everyone knows his history.
"Without him, other boxing heroes wouldn't be great either because they really tested his talent against him.
"In a room filled of great men, he is definitely one of them."
Frazier, who won Olympic gold in 1964, became world champion in 1970 after Ali had been stripped of his titles in 1967 for refusing to fight in the Vietnam War.
When Ali tried to reclaim his belts, Frazier beat him, winning their much-hyped clash in 1971.
It was Ali's first loss as a professional but a defeat he avenged.
Frazier and Ali met a further two times in the ring, Ali winning both contests, including the epic "Thrilla in Manila" in 1975.
Frazier also fought fellow American George Foreman, another of boxing's heavyweight greats. They met twice, Foreman winning on both occasions.
Foreman took to social network site Twitter to share his thoughts about his former adversary.
"Good night Joe Frazier," he wrote. "I love you dear friend."
Other boxing stars, past and present, were also quick to pay their respects on Twitter, too.
Eight-time world champion Manny Pacquiao wrote: "Boxing lost a great champion and the sport lost a great ambassador."
Five-time divisional world champion Floyd Mayweather added: "RIP Smokin Joe. My thoughts and prayers go out to to the Frazier family. We lost an all time great tonight."
There were also messages from former world champions Oscar de La Hoya and Shane Mosley.