Only one controller can run at once. If it crashes, Kubernetes will start another pod.
For many users, a short loss of workflow service maybe acceptable - the new controller will just continue running workflows if it restarts. However, with high service guarantees, new pods may take too long to start running workflows. You should run two replicas, and one of which will be kept on hot-standby.
Run a minimum of two replicas, typically three, should be run, otherwise it maybe possible that API and webhook requests are dropped.
Consider using multi AZ-deployment using pod anti-affinity.