The fun factor thing is a huge consideration. It is so huge that many restorers are taking older Mustang hard tops and sawing the tops off to make them into collectible ragtops. Correctly done, few could tell the difference. The same is true for four-door cars - some people "convert" them into coupes with major body work, as a coupe is worth some coin, but a sedan languishes at auction.
As an "investment" though, an old car is not a very good deal. You might have fun with it, and it might be a nice hobby to have, but you likely won't make money at it. Most hobbies are this way.