If your philosophy had been followed in 1861, the Union army would have consisted of Lincoln and congress. Maybe 1000 people.
They would have been wiped out at First Bull Run. The South would have won in a month.
The nation would have been fragmented and probably still would be. Would you like that?