22 is totally the way to get going in this sport.
Once you are going to the range more you will get to know people and most are more than happy to let you try different firearms if you just start asking questions just don't be a pest.
Next in line could be something like an SKS that are still fairly reasonably priced and are relatively cheap to feed and are a stepping stone onto bigger and better things.
The Mosin 91/30 is low cost but not a great starter rifle. It kicks like a mule and can quickly discourage a new shooter but with some experience they can be fun and reasonably inexpensive to shoot if you use surplus.