Combine the cold butter and brown sugar in a 9-inch cast iron skillet, turn the heat to medium-low, and cook, stirring until the butter melts. Take off the heat and stir in the lemon juice and 2 teaspoons vanilla.
Peel two bananas and slice into ¼-inch coins. Carefully place the banana slices in the brown sugar mixture.
Combine the flour, baking powder, and salt in a large mixing bowl. In another mixing bowl, combine the sugar, milk, melted butter, and remaining 2 teaspoons vanilla. Mash the remaining banana well and stir into the batter. Batter will be thick. Spread the batter evenly over the bananas and bake for 30-40 minutes or until the cake is golden brown and a toothpick inserted into the center comes out clean. Let cool for 2-3 minutes, then run a dinner knife around the edge of the pan, place a serving platter over the skillet, and flip it over. If the cake does not come out of the pan, put it back in the oven for a minute or two to heat the caramel topping.