I mean, the two guys that came before him were indisputably worse for Michigan.
Lloyd Carr was better.
Gary Moeller was mediocre.
So, 3 of the last 4 Michigan coaches before Harbaugh were worse than him. There's a good chance that whoever follows him would follow that pattern. Being a big time program is no guarantee that they land a great coach -- even the bluest of bloods have hired their fair share of lemons. And Michigan doesn't have the appeal of an Alabama or an Ohio State.
So in one sense, better the devil you know. But in another, there's a good chance that dumping Harbaugh results in a worse coach for Michigan. And I'm all for that.