The incident involving the centre 10 minutes from time hurt the hosts, who had rallied from 14-6 down to lead 18-17.
However, Matavesi's yellow card for deliberate offside sealed their fate and Hill said after the 24-18 defeat: "Josh thought the ball was out and was unlucky to get the yellow card. With 15 men I think we might just have held on."
At the interval Worcester looked unlikely winners but a much improved second half effort nearly paid off.
Hill continued: "I am really delighted with the way we fought back and our driving maul paved the way for two tries. I must give credit to our forwards coach, Nigel Redman, who kicked a few backsides after last week's poor performance (at London Irish)."
The Warriors also seemed unlucky not to be awarded a first half try when David Lemi crossed after a splendid cross-kick from Andy Goode but the try was ruled out by the touch judge for a knock-on by Lemi.
Hill added: "It looked a good try to me and everybody else in the ground. It knocked the stuffing out of us."
Exeter coach Rob Baxter was happy with the win but accepted that his side had allowed the Warriors back into the game.
He explained: "Any away wins in the Premiership are big wins. Worcester don't lose many at home but we controlled the first half as we played with a very high tempo.
"However I was disappointed with the first 20 minutes after the interval as we became very edgy and tried to eke out a win when we should have continued to play as we did in the first half."