Reading With Pride?

During our recent trip to a conference in the US, I noticed at least two people reading books. One was perusing Mike Resnick's Purgatory, while the other was devouring something by Nora Roberts. And oddly, when they noticed that I was looking at them, both seemed intent on hiding their books.
( a bit of background, but no plot . . . )

I read science fiction. Publicly and with pride. Maybe I should get pins or ribbons - or bookmarks! - made and hand them out, rewarding these people for reading in public. After all, other people talk, watch TV, listen to music (and sometimes croon along), knit, or engage in other hobbies in public.

Why shouldn't people read in public?