The tension between Hester and the Puritan people is Hawthorne's design of a great protagonist. But neither of these characters are real, so are we not to go by what is said or believe it?
I think that Hawthorne invites us into this world and throws the shawl of the story around our shoulders, bringing us to him, not wanting us to stay outside and judge what is (of course, not) going on, but to be inside the story and seeing the world form his eyes.
My favorite chapter is The Leech and his Patient, allowing us to see the inside of both Dimmesdale and Chillingworth. Both of these men, rotting with emotional baggage, spar round their home on theology versus physiology.
In this chapter, we see not only physical ailments on Dimmesdale, but that his spiritual torment has caused this affliction.