Eighty percent of Jews did not go out from Egypt with Moses. Rather, they died during the Plague of Darkness. (See Rashi's commentary on Exodus 13:18.) They were people who really didn't want to go out of Egypt or had great doubts about leaving their "homeland" for a desert and the unknown.
It's amazing to consider the numbers involved. If we assume that approximately 3 million Jews left Egypt, this means that 12 million Jews died in Egypt prior to the Exodus!
I hope this has been helpful.
