Prophet Abraham - 14:36 My Lord, they have led astray many of the people. Those who follow me are from me. As for those who oppose me, surely You are Forgiving, Merciful.
The believers were always a minority:
The Scatterers/51:36 We did not find therein any Muslim except in a house.
فَمَا وَجَدْنَا ف۪يهَا غَيْرَ بَيْتٍ مِنَ الْمُسْلِم۪ينَۚ
The Scatterers/51:35 Then We drove out whoever therein was of the believers.
فَاَخْرَجْنَا مَنْ كَانَ ف۪يهَا مِنَ الْمُؤْمِن۪ينَۚ
The Scatterers/51:34 Marked by your Lord for the transgressors.
مُسَوَّمَةً عِنْدَ رَبِّكَ لِلْمُسْرِف۪ينَ
The Scatterers/51:33 To send upon them stones of clay...
لِنُرْسِلَ عَلَيْهِمْ حِجَارَةً مِنْ ط۪ينٍۙ
The Scatterers/51:32 "They said, "We have been sent to a guilty people."
قَالُٓوا اِنَّٓا اُرْسِلْـنَٓا اِلٰى قَوْمٍ مُجْرِم۪ينَۙ
The Scatterers/51:31 "(Abraham said): "O messengers, what is your mission?"
قَالَ فَمَا خَطْبُكُمْ اَيُّهَا الْمُرْسَلُونَ

Prophet Jonah/10:83 Only a few of his people believed in Moses because they feared the torture and oppression of Pharaoh and his men. Pharaoh was a tyrant in the earth who exceeded the limits.
فَمَٓا اٰمَنَ لِمُوسٰٓى اِلَّا ذُرِّيَّةٌ مِنْ قَوْمِه۪ عَلٰى خَوْفٍ مِنْ فِرْعَوْنَ وَمَلَا۬ئِهِمْ اَنْ يَفْتِنَهُمْۜ وَاِنَّ فِرْعَوْنَ لَعَالٍ فِي الْاَرْضِۚ وَاِنَّهُ لَمِنَ الْمُسْرِف۪ينَ

Prophet Hud/11:40 "Finally, when Our command came and the sky boiled and overflowed, We said to him: "Load on him a pair of every kind, his children and those who believe, except those who were condemned before."Those who believed with him were already a few."
حَتّٰٓى اِذَا جَٓاءَ اَمْرُنَا وَفَارَ التَّنُّورُۙ قُلْنَا احْمِلْ ف۪يهَا مِنْ كُلٍّ زَوْجَيْنِ اثْنَيْنِ وَاَهْلَكَ اِلَّا مَنْ سَبَقَ عَلَيْهِ الْقَوْلُ وَمَنْ اٰمَنَۜ وَمَٓا اٰمَنَ مَعَهُٓ اِلَّا قَل۪يلٌ