Manny Pinol, a former governor of North Cotabato and current manager of boxers through his Braveheart Boxing stable, says the success of the eight-weight world champion is hindering the amateur Olympic program at home.
Despite boasting a population of over 100 million and a love of boxing, the Philippines will send just one fighter to the London Olympics later this year - the same number as the tiny Seychelles - leaving little hope of adding to the three bronze and two silvers won by their boxers in previous Games.
All this while Pacquiao (welterweight), Nonito Donaire (super-bantamweight), Sonny Boy Jaro (flyweight), and Donnie Nietes (light flyweight) hold world titles in the pro ranks.
"Manny Pacquiao's success is hurting our amateur boxing programme because every young boy now would like to become Manny Pacquiao," Pinol told Reuters in a recent interview in Singapore.
"They don't want to be amateur because there is simply no money in it, it is just a dream of a (Olympic) gold and it is actually more difficult to win than a world boxing title."
In the Philippines, the rags to riches story of Pacquiao, who came from a poverty stricken broken home to become the first man to win world titles in eight divisions, and then be elected a congressman, inspires a nation.
His fights in America have earned him millions and regularly bring the Philippines to a standstill as his compatriots stop work and flock to big screens across the country to support him.
His guaranteed purse for his June 9 bout against undefeated American Timothy Bradley is $25 million, according to media reports, with more expected to come from television money which is counted after the fight.
In 2009, the government statistics agency said 27 percent of Filipinos lived below the poverty line, leading many young fighters and pushy parents to sidetrack the unpaid amateur game, where skills are honed and lessons learnt, to chase the wealth that Pacquiao commands in Las Vegas.
"The latest edition of our team, he has just turned pro, (he) would have made a very good amateur fighter because he is tall and has a long reach," Pinol said. "But he wanted to turn pro and his parents wanted him to turn pro."
But the path to greatness is a long hard road with no shortcuts. While promising talents may possess the power punching of the 33-year-old "PacMan" not all can boast his boxing skills.
Pinol was in Singapore with one of his fighters, Lorenzo Villanueva, who was attempting to win the vacant IBO featherweight title in a bout with Indonesian Daud Yordan.
Villanueva, a silent, timid but immensely powerful puncher, knocked down his opponent within 20 seconds of the first round of his fight. However, his inexperience showed and he was knocked out in the second after swinging aimlessly for a final punch.
For the slight 26-year-old Villanueva, whose father was murdered when he was a boy and his mother left shortly after leaving him to raise his brothers and sisters, the dream of accumulating enough money to send his siblings to college has been delayed by defeat in Singapore.
"Lorenzo is a product of a grass roots boxing programme which I started in 1998 when I became governor," Pinol said. "I formed a team, built a ring, acquired some gloves, went from one village to another and asked young boys interested in boxing to go up to the ring for the measly price of 100 pesos for the winner and 50 pesos for the loser.
"This story of Lorenzo, the story of how he was attracted to boxing like many other boys in the Philippines born in poverty. This is a story of poor boys who hope to get out of poverty as quickly as they can and the quickest way is through boxing.
"It is not the only way but when you have the talent, have the power and are inspired by Manny Pacquiao you immediately turn to the sport because of the fabled story of Manny Pacquiao - (but) not everybody can be Manny Pacquiao."
Pinol acknowledged Villanueva's lack of technical skill, and that his impoverished background forced him into the professional ranks early, but he hopes to send two fighters from his camp to the Rio Games in 2016.
He wants his school, where three of his 40 fighters have challenged for world titles, to be developed around the country in a bid to hone the talents of the next generation in order to win that elusive gold medal.
"It is the only way to do it, Cuba did it and won a lot of (boxing) gold, Ukraine are doing it and winning a lot of gold, we have to look at these models if we want to win gold for the Philippines."