Ultimately, I have the same problem here as I do with the “Rawlsian” utility function of “maximize the utility of the worse off person”: it allows - and indeed encourages - trading literally everyone else’s utility for that one person. Do you really want a CEO to trade half of this years earnings for a 0.01% boost in consumer satisfaction? I think not.

And so we should probably go to the traditional (academic) solution: actually think about your values and choose the utility function to match. This is hard, but it’s ultimately the right solution.