Washrooms for public use in Europe have been a scarce commodity. One takes it for granted in North America that stores, including grocery stores and hardware stores, tend to let people use their washrooms freely.
Perhaps it is no surprise that alleys in Europe sometimes smell of urine. In Stockholm and Copenhagen to use the washroom was about 5 krona and in the Netherlands and train stations it was approximately €0.50. I understand that flushing water, toilet paper, electricity and cleaning services cost something but I also understand why people may choose to pee in the alleys instead.
Notably Paris city center had free washrooms in public places, such as the garden outside Notre Dame. Don’t expect toilet paper or soap as supplies may have been depleted by those before you. Tip: buy the small packets of 10 tissue and small bottle of hand sanitizer for these situations.
If you happen to be shopping at a store already, do ask if you may use their facilities. A grocery store clerk did take us to their staff room with a bathroom when we had asked.
Tip: also bring your water bottle with you if you are cycling! It is a great time to refill your water.