That makes no sense. If we win all our division games, and Duke wins all their division games other than against us, but we lose to top 5 FSU and Clemson and Duke beats bottom feeders Wake and NC State, Duke wins the division. Replace Tech with Miami or Louisville's opposite and yu have the same scenario.
But it's not just these teams. What if NC State and Wake happened to be great teams in that year but FSU and Clemson are crap and Duke beats Tech instead, but loses to Wake/NC State and we make it through by beating crap FSU and Clemson. Counting cross division games makes no sense at all. The problem for Tech, Miami and Louisville's permanent opponent is that they are more likely to suffer from these scenarios than other teams. But even if permanent opponents didn't exist, there is no reason to decide the divisions champ using external games when we play a round robin schedule.