Bees winger Sam Saunders sealed the points that took his side to second spot when he sent keeper Declan Rudd the wrong way from the spot.
He got up to convert the penalty after being sandwiched by former Brentford player John Mousinho and striker Stuart Beavon in a moment of madness in the visitors' box.
But an angry Grayson raged: "It was very disappointing to lose to a soft penalty like that particularly when there were one or two Brentford players going down quite easily. If that had been up the other end I doubt we'd have got it.
"We defended really well, had a good shape and dominated for large parts in the second half so we can be proud of the performance against a side that's near the top pushing for promotion."
He added: "The lads showed we are a match for anyone and if the season went on for another 15 games we'd be right up there."
- Sports & Recreation
- John Mousinho