I owned Green & Yellow from gen 1 (I mean, a talking Pikachu?? On GB?? Hell yeah!) and jumped to Crystal, Black, Y & AS. If you haven't played them at all, it might be neat to start at the beginning and see the evolution of the game which is just as interesting (if not more) than the story itself. After all, they have to be doing something right to be able to make that many games without coming off as redundant, right?
I would try R/B/G/Y first and see if you like it. If it seems too retro to you, then yes, go for one of the 3 3D versions (X/Y OR/AS S/M). But I think you'll get the full appreciation and references if you have some knowledge of the earlier games, if you're willing to put in the time.
I think the fun comes from the meta-game, ie Mutiplayer. It can be fun if you're into that sort of thing; it's one of those "easy to learn, hard to master" things.
TBH though I skipped SM. I basically stopped at RS remakes because I was getting tired of them charging for the pokebank. I get that cloud storage costs money but it should be free to just transfer pokemon between games.