I have some math below, but to start...I tried doing some business in a small town and it didn't work. I used to be a stockbroker so I tried networking with other professionals in the town (lawyers, accountants...etc.). But they already had their 'go-to' business parnters. It was like trying to weasle my way into a good ol' boys club where they thought I didn't belong. If I had started business on my own there and had been successful, enough of the town would actually have been upset for having "stolen" business that it wouldn't be pleasant living there. I high-tailed it outta there!
SUPPLY and DEMAND:
Let's say one photographer could handle 50 clients in one year (weddings and seniors). In your town, let's say there are 500 clients available for that year (the whole town isn't getting married or graduating in the same year...and if they do, everyone will be out of business in 2008...lol). This means you're trying to capture 10% of the market (a huge market share for any business).
In a larger city (the nearest one is 40 times bigger)...the 50 clients you could handle only represents 0.25% of the market...a very feasible capture.
PEOPLE's ABILITY TO PAY
Average incomes will be lower in the town, and your prices will have to reflect that. So you'll have lower prices (relative to what you can charge in the city), and fewer clients (assuming you're not able to capture 10% of the market). Whereas in the city, people can afford to pay you more and you'll have an easier time capturing 0.25% of the market (50 clients).
That's just my .02 cents!

Good luck with your decision.