Rankings are meaningless early in the year. And not much better at the end of the year, since teams play entirely different schedules, in different conferences, and with irregular home/away matchups.
The NFL format does a better job of getting to statistical significance by the end of the year. But even that isn't perfect, since schedules get weighted to balance somewhat with past performance.
As it is, today's college rankings are just eye-test conjecture on top of a really small sample of views. But hey, when you can get a "Top 5" game on TV, it sells.