Cook bacon in a 3-quart saucepan over medium-high heat, turning occasionally, until evenly browned and crisp, about 10 minutes. Drain on a paper towel-lined plate. Discard bacon grease and wipe the pan thoroughly with a paper towel.

Heat olive oil in the same saucepan over medium heat. Sauté onion, carrots, and celery in hot oil until onion is soft but not brown, 3 to 4 minutes.

Stir in broth, potatoes, and cayenne pepper; bring to a boil, reduce heat, and simmer, covered, until potatoes are tender, 10 to 15 minutes.