The 32-year-old is leaving Brunton Park after 12 years and Shiels is close to a deal.
The Kilmarnock manager, who is also aiming to sign goalkeeper Craig Samson from Saturday's Clydesdale Bank Premier League opponents St Mirren, said: "We have Peter Murphy from Carlisle. I am confident we can cross the line with that one.
"He is an experienced left centre-back who has over 400 games for Carlisle behind him.
"He is a top player. He fits our needs and he has the quality as well."
If he completes his deal, Murphy will be able to help nurture 19-year-old centre-back Lee Ashcroft, who made his debut for Killie in last weekend's 3-2 win over Dundee.
The teenager followed that by signing a new three-year contract and then scored on his first start on Wednesday in the 3-1 defeat by Hibernian.
Kilmarnock have also announced that another 19-year-old, striker Ross Davidson, had followed Ashcroft in signing a three-year deal.