1. Shopping malls, department stores and large supermarkets
Some people love to shop for hours on end. Store owners appreciate this, and also don't want customers soiling the goods. For these reasons, if you're in a city it's likely that there will be a mall or department store nearby with handy signs to the toilets and they are often kept reasonably clean. Even some large bookstores will have the facilities you seek. If you're traveling by car, it should be easy to find commercial zones which usually have toilets as well. (And with a car you can also visit petrol stations.)
If you find yourself in a campus town, head over there quick stat as entrance is free and students need to relieve themselves too. Just walk in all confident like you belong. I have utilised the University of Siena's toilets twice in as many days.
You may have to pay the price of a coffee or tea, but opting for a café allows you a little sit down as well, and often the toilets are in better condition than public ones. If you inquire nicely, some cafés will let you use the toilets without buying anything, but it's really up to the waiter you ask.
Public buildings with public toilets, libraries are a great choice (and quiet too). The City Library in Wellington, New Zealand is a three minute walk from the waterfront and shopping central Lambton Quay.
5. McDonalds and other busy chain-like eateries
It seems that McDo pretty much always has at least one person waiting in line at the counter even at strange times of day. You can easily slip in undetected and use the facilities. It depends per store, but some chain eateries require a purchase as in Burger King and about half the Subway stores in New Zealand. It can't hurt to have a peek, as many towns (unless they are very small and in France) will have at least one chain fast food joint.
It doesn't cost to simply enter a cinema and most have toilets near the ticket desk. The Embassy theatre in Wellington have beautiful toilets on the first and second floor for anyone to use, complete with sofas for reclining and wood paneling.
Many museums have toilets near the entrance and are well signposted - just pick a museum that doesn't have a queue to enter but is big enough to actually have toilets in the first place. Note that this is so far down the list as it really depends on the museum - I'd estimate that about half of all museums I've visited have toilets before the ticket desk, and half after, which would require paying the entrance fee. The good thing is that you can often tell as soon as you enter a museum if the toilets are easily accessible. In Wellington, the ever popular Te Papa is free to enter, so there are no queues, and there are toilet facilities on several floors. In European museums, the toilets are usually next to the cloakroom.
8. Public Toilets
I've left this one for last, as you never know how gross a public toilet will be. Many do not have toilet paper or hand soap. Many will cost you a few coins. I'd much rather pay a little more for some tea at a café and be comfortable than squat over a lidless porcelain pisshole and drench myself in hand sanitiser once I've escaped. That being said, if I'm in a hurry I have been known to use them and some of them are actually clean and not disgusting at all - it's luck of the draw. Tourist offices often have public toilets or can direct you to some nearby.
Good luck! Anything I missed? Let me know in the comments below.