Yes, racing should be the drivers' decision; not the fans, not the stewards, not the promoters, not even the sanctioning body - none of them are in the cars with their lives on the line. But drivers vary greatly in what they can tolerate.
I recall an F1 race in Japan years ago where rain became so intense that most of the drivers pulled off the track, stating it was to dangerous to continue; but the promoters didn't want to cancel or postpone the race so it continued with the few cars remaining.
At a CART race in Michigan back in the nineties it was estimated to be over 120deg (F) in the cars but no one thought that too extreme. At the end of the race many drivers were literally lifted out of their cars by the pit crews. Yet Mario Andretti climbed out of his car, removed his helmet and gloves, and walked over for an interview, looking no worse than if he'd just been on a Sunday stroll.
Should that F1 race have been postponed til teh rain stopped? I think so. The Michigan race? No drivers were complaining.