If you've got a string, and you expect it to always be an integer (say, if some web service is handing you an integer in string format), you'd use Int32.Parse().
If you're collecting input from a user, you'd generally use Int32.TryParse(), since it allows you more fine-grained control over the situation when the user enters in invalid input.
Convert.ToInt32() takes an object as its argument, and I believe it invokes Int32.TryParse() when it finds that the object taken as the argument is a string.
Convert.ToInt32() also does not throw ArgumentNullException when it's argument is null the way Int32.Parse() does. That also means that Convert.ToInt32() is probably a wee bit slower than Int32.Parse(), though in practice, unless you're doing a very large number of iterations in a loop, you'll never notice it.