My younger brother harvests oysters on the Damariscotta River in Maine and talks about a kid on his boat from “up Machias way,” a town on the coast some two hundred miles north of Portland. A good number of the people there have never made it those two hundred miles south. Some never leave town at all. Jason Brown’s second collection of short stories takes place in a Maine town like this, but the fictional Vaughn wouldn’t fall into anyone’s lighthouse-and-lobster idea of “Vacationland.” As for “the way life should be,” the folks in Vaughn—angry high-schoolers, lonely widows, jealous siblings, loggers, teachers, and miscellaneous ne’er-do-wells—don’t have much of a choice one way or the other.

Like old mill towns, Vaughn suffers the burden of a history more prosperous than the present. The inhabitants, as a result, express the mix of pride (“I belonged to a large family that had lived in the same town in Maine for over two hundred years”), defiance (“I could go right now if I wanted to. There’s nothing stopping me”), and resignation (“She would never let them forget she was from Portland”) that comes from existing amid the ghosts of prior fortune. Vaughn is a place where people stay and get stuck.

People do try to escape. The characters that leave are “out there somewhere, vanished but not gone.” And if you leave, the town pulls you back as much as it pushed you away. Even the kids with the best chances of getting out are thwarted. In “The Plains of Abraham,” a highlight of the collection, the promising son of the town’s high-school history teacher is attacked—a sexual assault that’s never referred to as such—by two boys in the woods. He’s broken by it, and instead of embracing a future, he disappears from Vaughn and wayfares around the country. In the tense and bone-cold “A Fair Chance,” a recovering alcoholic clears trees—Brown’s descriptions of wood and water are reverent—and thinks about the life he’ll have in Portland. But will he get there?

Brown’s first collection, Driving the Heart, is set partly in Portland; it’s no better than Vaughn in Brown’s hands, and might even be worse—more violence, drugs, confusion, sadness—rendering the hopes of the people of Vaughn all the more sorrowful. Even if they were to get to Portland, we know what they’ll find.

Those who stay circulate the town but cannot pull away. A...