I started reading the books after watching the first season, and I was pleasantly surprised by how little it has affected my enjoyment of the series, if anything I enjoy it even more! I was sad to see a few secondary characters get left on the cutting room floor, but given the difficulties of fitting such an epic story into a television show, I felt they did very well. The most important thing is that the real guts of the story and its central characters are all there, and rendered with amazing fidelity.
Given the kind of stupefying mediocrity which seems to make up 90% of TV these days, I give Game of Thrones and HBO a lot of credit for putting the time, budget and genuine care into the show. They definitely could have made a dumbed down, toothless, bloodless, sexless version of Game of Thrones and put it on prime time, but they didn't and for that I salute them!