Download the TP (PSP) Install tool from a reliable source. This tool will help you create the ISO file and install it on your PSP.
Some videos showing GTA 5 on a PSP are actually using "Remote Play," which simply streams the game from a PC or PS4 to the PSP screen rather than running it natively on the handheld. Official GTA Games for PSP download gta 5 psp iso file by tp install