Preheat the oven to 350ºF (176ºC). Butter and lightly flour an 8-by-2-inch round cake pan or an 8-inch springform pan, tapping out any excess flour.
In a small bowl, toss the cornmeal, all-purpose flour, baking powder, and salt.
In a large bowl, beat the butter and granulated sugar with an electric mixer until pale yellow and creamy, about 5 minutes. Scrape down the sides of the bowl with a rubber spatula and add the egg yolks, 1 at a time, beating after each addition. Scrape down the sides of the bowl and add the whole eggs, 1 at a time, beating after each addition. Mix in the lemon zest and vanilla. Add the dry ingredients and blend until just combined.
Spread the batter in the prepared pan. Place the plum halves, skin side down, at even intervals on top of the batter. Sprinkle the brown sugar on top of the fruit and batter.
Bake until the cake is golden brown on top and a toothpick inserted in the center comes out clean, 30 to 45 minutes.