Netto is a cheap German supermarket which sells SIM cards operating on the O2 mobile network.
The Pay-as-you-go rates are not as cheap as Aldi's, the cards cost (in 2019) either €9.99 with a €10 credit already on them or €5 with €5 credit, but you can select your preferred mobile number from whatever range they have in the shop because the numbers are printed on the outside of the packaging.
Any credit lasts up to 12 months before you are required to top up, and the minimum top-up value is €4 (which gets you an additional 12 months). This makes it quite a good option if you only need inbound calls or text messages.
Update 2021: Things have changed.
Note also that the Germans are now pretty fussy about knowing who is using which SIM cards; when you buy a card, it doesn't work until you activate it, and the activation process requires proving your identity with a German identity card (Ausweis) or a passport.