I Can't Seem to Escape this Guy
Seriously, Robert Frost seems to follow me EVERYWHERE! This is about "Desert Places."
"I have it in me so much nearer home
To scare myself with my own desert places" (lines 15-16)
Why go somewhere to feel lonely when you can do it at home? The only reason people get lonely is pretty much their own doing: avoiding people, acting such a way that people would dislike you, etc. I probably sound heartless right now, but I also realize that loneliness can also be caused by forces we can't control.
http://jerz.setonhill.edu/EL237/2009/09/frost_desert_places/
I think the speaker more or less claims that he was going to avoid this feeling/place. In line two, he says that he's "In a field [he] looked into going past." Though, I do agree. You can be lonely pretty much anywhere.