Rugby makes its Olympic debut at Rio in the sevens format and places on the Games squad are likely to be fiercely contested among regular team members and hopefuls in the 15-man game.
O'Connor said he would give short shrift to any 'johnny-come-latelys' trying to cash in on their celebrity to earn a berth.
"We have been building a young side and the bulk of the team we'll take to Rio in 2016 will be guys who've been in the program and gone to Super 15 or are currently in the program now," O'Connor told reporters in the Gold Coast ahead of the series-opening tournament this weekend.
"We're now starting to get these guys two or three years into sevens. They're really building valuable experience and that's just essential for the Olympics.
"If you just fill a side up with national 15s players the year of the Olympics, that's a recipe for failure.
"It's not about getting star players for the sake of a headline."
O'Connor said "one or two" top-level players had expressed an interest in playing Olympic sevens.
Those with ambitions of playing the regular rugby World Cup in England in 2015 could still be considered, O'Connor said, if they had some previous experience of playing sevens.
"For a lot of them, it's down the track and they'd have to make a substantial commitment to the program. Not just a case of coming over in 2016," he added.
"The squad's coming together quite nicely. We're on track for the next couple of years to have a very successful sevens outfit."