He did get to live his dream later on though. Chief Justice of the US Supreme Court.
Only individual to be the “guy in charge” of two different branches of the federal government during his life.