How to add/subtract pixels to/from selection?

I try to add a few single pixels to this selection and remove some.

I know it works with the rectangle select tool when you set it to 1px, but I search for a pencil like solution where I just have to click.

It sounds very simple, but I try it for 15 minutes now.