Limited-overs coach Ashley Giles refuted the allegations from the former England captain, who, in his guise as a broadcast pundit, claimed the Champions Trophy hosts were using underhand methods to achieve reverse-swing at an early stage of a game.
"As a team we're really surprised by his comments," Anderson told the Mail on Sunday.
"Even though there's nothing to them whatsoever they could stick for a while to us, which isn't wanted, needed or helpful. It's come as a very unwelcome distraction and, frankly, we could do without allegations such as these."
The allegations have dominated the build-up to England's must-win match against New Zealand in Cardiff.
Anderson added: "It's very disappointing and frustrating that anyone, but especially a former England captain, should say these things, especially when we're preparing for a massive game.
"The fact is people can think what they like. We know the truth, I can state categorically that no one in the England team has ever tampered with a ball and we won't allow comments made by someone like Bob Willis to worry us."
The pace bowler claimed the team's ability to achieve reverse swing was down to pure skill.
"Reverse swing has been an issue in cricket for a while," he said. "But just because one bowler or one team reverse a ball better than another on a given day doesn't mean to say he or they have been tampering. It just means they're more skilful.
"I can assure you that if any of the England bowlers achieve it then it is within the letter of the laws."