A large number of non-pilot passengers have told me they don’t like flying in clouds. They want to be able to see the world outside the window. If you are going to be in actual instrument conditions, say so ahead of time and give a brief explanation about what to expect. I’ve had some pilot friends indicate that their spouses were so uncomfortable with being in clouds that they always flew VFR with them, even if it meant delaying a departure for a couple of days. Their spouses are still willing to fly with them.
One of the best axioms of military aviation is, “There is no excuse to fly through a thunderstorm in peacetime.” Even the most aeronautically nave passenger knows that thunderstorms are horribly powerful monsters that can destroy houses and airplanes. You know better than to fly through a boomer when solo—there’s no reason to subject passengers to one.
Ice worries passengers in varying degrees. Some know how truly treacherous it is, but even those who don’t will be curious about why the windshield became opaque and why the wings look like a freezer that needs to be defrosted and why you are suddenly sweating even though it’s cool in the cabin.
Out of consideration to passengers, make weather decisions even more conservatively than when you are flying alone. Look at it this way: The airlines and corporate pilots literally go out of their way and do all they can to give a wide berth to weather because they want repeat business from their passengers. Running your private flight department in such a way as to cause passengers to want to come back for more is sure to make any good capitalist happy.