Qatar will host the 2022 World Cup and Amnesty asked FIFA to work with the Qatari authorities to stamp out abuses of workers who mainly come from South Asia.
"Our findings indicate an alarming level of exploitation in the construction sector in Qatar," said Amnesty Secretary General Salil Shetty.
"FIFA has a duty to send a strong public message that it will not tolerate human rights abuses on construction projects related to the World Cup.
"It is simply inexcusable in one of the richest countries in the world that so many migrant workers are being ruthlessly exploited, deprived of their pay and left struggling to survive.
"Construction companies and the Qatari authorities alike are failing migrant workers. Employers in Qatar have displayed an appalling disregard for the basic human rights of migrant workers.
Amnesty is the latest organisation to focus on treatment of migrant workers in Qatar, following similarly scathing reports by Britain's Guardian newspaper and the International Trade Union Confederation.
Amnesty said it had carried out interviews with about 210 migrant workers in the construction industry during two visits to the country in October 2012 and March 2013.
It also held meetings with 22 companies involved in construction projects and met government representatives on more than a dozen occasions.
The report said abuses included "non-payment of wages, harsh and dangerous working conditions and shocking standards of accommodation."
Earlier this month a U.N. official called on Qatar to abolish the kafala, or sponsorship system, under which employees cannot change jobs or leave the country without the permission of their sponsors.
Many sponsors - often labor supply companies or wealthy individual Qataris who provide workers to businesses for personal profit - confiscate the passports of guest workers for the duration of their contract.
Shetty said the authorities had not committed to replacing kafala.
"We have met with the prime minister, labour minister and other officials and they all have not denied that there is a problem but they haven't made any commitment and this is what we need," Shetty said.
No one was immediately available to comment from Qatar's Labour Ministry. However, last week the ministry had already taken a positive step by recruiting more labor inspectors, Shetty said.
Amnesty also said its researchers met dozens of construction workers who were prevented from leaving the country for many months by their employers.
In one case, Nepalese workers employed by a company delivering critical supplies to a construction project associated with FIFA's planned headquarters for the tournament said they were "treated like cattle", the report said, adding that they were working up to 12 hours per day and seven days a week even during Qatar's summer heat.
James Lynch, Amnesty's researcher on Gulf migrants' rights, told reporters at a news conference in Doha that a worker would have to pay 600 riyals ($165) in order to pursue a grievance through local courts, a fee many could not afford.
Researchers also found migrant workers living in squalid, overcrowded accommodation with no air conditioning, exposed to overflowing sewage and uncovered septic tanks.
"Researchers witnessed 11 men signing papers in front of government officials falsely confirming that they had received their wages, in order to get their passports back to leave Qatar," Amnesty said.
FIFA said respect for human rights was part of all of its activities. The body said in a statement: "It is FIFA's aim that the host countries of our flagship event ensure healthy, safe and dignified working conditions for all - nationals and foreigners, including construction workers - involved in the preparation of the event."