That was my initial point. The Harris Interactive and USA Today Polls are the main polls used. They do not take into account the SoS. The voters may, but the record usually oversees any other metric.
They don't explicitly take into account strength of schedule, but they definitely do to some extent.
Just look at the Harris Interactive Poll.
Georgia had 2 losses, but was ahead of 1 loss KSU. That's probably because of perceived strength of schedule. Teams like Boise State (10-2), NIU (12-1), Louisville (10-2), Northwestern (9-2), Utah State (10-2), and Kent State (11-2) were all behind teams with fewer losses, simply because of strength of schedule.
Sure, record is the most important, but it's the same way with the computer polls too. Nobody is getting ranked based on strength of schedule alone.
If the point is, this won't affect them because strength of schedule doesn't matter, that is incorrect. If the point is, dropping FCS schools and adding lower tier FBS schools won't change the SoS enough for voters to notice, that's probably right.