Selecting with brush affects inverse not selection

When I select an area with a brush in order to apply an adjustment layer to it the applied adjustment affects the inverse area not, as expected, the selection. Any help with this conundrum would be much appreciated. Thank you.