I have to agree with the .22 for starting off, especially for the rifle.

Just a couple of points: If you buy quality, you will have it and enjoy it for the rest of your life. If you buy used, you will be able to sell or trade it for around the same as you paid for it, as long as you don't buy a lemon (they are out there also). Before you buy, sniff around some of your gunny friends and ask for the opportunity to put a couple of rounds downrange; offer to cover the ammo costs.

For a pistol, .22 is a good choice. A good alternative is a .38/357 revolver, especially if you will one day reload. Before you go that direction, look at the price of ammo for 100 rounds of .22 versus 100 rounds of .38.