I am a recent grad, with tons of friends who went to MIAC schools who now live in the area and some are in Grad school at the U. As they are getting into Gopher sports more now, they are all jealous that I got to spend my undergrad years involved in big time sports and all that comes with it.
Most of my friends at the U could not get enough of it, tailgating, going to bars to watch away games, etc.
So yes there is a culture out there, but the U is so big, someone could easily go around and ask students on campus if they are into the football, basketball, hockey scene at the U, and have a bunch of people say that they weren't. Hopefully with more winning seasons in a row, we can work on gaining more of those marginal fans out there who are on the fence.
Our biggest problem (actually a strength) is that there is so much more to do at the U and in the Twin Cities than there is in Madison, Iowa City, etc. People here have the ability to be involved in other things, like theater, and the arts, various music scenes, various other campus groups, not to mention pro sports teams. Winning teams will help us recruit some of these people to come to gopher games as well.