Seeded 16th, Ivanovic was a break down to the 19-year-old in the deciding set but the former French Open champion fought back to take the tie break 7-5 as Robson double faulted on match point to miss out on a place in the last eight.
"I'll definitely take the positives out of the match," Robson said.
"It's been a good week of tennis for me and my level has picked up a lot since Estoril last week," added Robson, who is currently without a coach after splitting with Zeljko Krajan just before the Madrid event.
"It's not all bad and I've still got Rome to look forward to next week. It was a great match from her and hopefully we'll have some more great matches in the future."
The British number one defeated a top four player for the first time on Monday after beating Agnieszka Radwanska 6-3 6-1.
Third round results
16-Ana Ivanovic (Serbia) beat Laura Robson (GBR) 5-7 6-2 7-6(5)
Round 3 6-Angelique Kerber (Germany) beat Svetlana Kuznetsova (Russia) 3-6 6-4 7-5
Second round results
Daniela Hantuchova (Slovakia) beat 8-Petra Kvitova (Czech Republic) 2-6 6-2 6-3
13-Maria Kirilenko (Russia) beat Kristina Mladenovic (France) 6-7(5) 6-1
Ekaterina Makarova (Russia) beat 3-Victoria Azarenka (Belarus) 1-6 6-2 6-3
Kaia Kanepi (Estonia) beat Carla Suarez Navarro (Spain) 6-3 6-1
Varvara Lepchenko (US) beat Julia Goerges (Germany)WO