England play all three of their group games in Ukraine, where far-right hooligans were recently exposed by a Sky Sports News investigation.
With England manager Roy Hodgson admitting that racism is a "concern" in the country, Walcott's father, Don, and brother, Ashley, have decided against travelling to Ukraine.
Ashley Walcott tweeted: "Unfortunately my dad and I have taken the decision not to travel to the Ukraine because of the fear of possible racists attacks confrontations [sic].
"Some things aren't worth taking the risk, but begs the question why hold a competition of this magnitude in a place that cannot police itself."
After announcing his 23-man squad for the finals on Wednesday, Hodgson conceded that non-white fans could be at risk if they journey to Ukraine to follow England.
“The issue of racism, especially the Sky report into the hooliganism and violence in Ukraine, is an obvious concern," Hodgson said.
"Not least the supporters who go over there and risk maybe getting beaten up.”