Horns and gray skin make them easy to notice, like it or not. They are also substantially easier to avoid, though.

That is true. Dave himself left this place for a number of months, I think, and he ended up here once again, although I believe others leave for much a shorter timespan and return with memories of events from their worlds.