The Shrub Identification Book by George W. D. Symonds, photographs by A. W. Merwin, to the rescue!
The first half of this book is split into black and white picture keys by thorns, leaves, flowers, fruit, twigs, and bark. Each picture key points to all aspects of the shrub found in the second half, the master pages.
What an incredibly easy system! I quickly narrowed down my mystery seed pods to a few possibilities and after looking closely and reading descriptions of key identifiers, the mystery was solved within minutes.
Sweet Pepperbush, Clethra alnifolia, is a very common shrub found from Maine and down the coastal states to Florida and across to Texas. Appearances can vary as there are a few varieties with pink flowers and another as a more compact bush. My local shrubs all have white flowers--I remember them from last summer.
This struggled identification taught me better observation--important identifiers were that each seed pod was individually attached to a main stalk, that there were five petals, and that there was a persistent and elongated style. I hope this photo helps to illustrate these identifiers.
Here's a Google Books link to this great tool.
For earlier posts regarding the Sweet Pepperbush, see the Index over to the left.