The rule of thumb for viewing distances is somewhere between 1.5 x the diagonal size of screen and 2.5 x the diagonal size of screen.
So, a 32" tv would be 32 x 1.5 = 48 inches to 80 inches. So, 4ft to 6.7ft.
If you sit back 10ft, your ideal screen size could include 46" to 52"
At the end of the day, if you are happier sitting further back...there is nothing wrong with that. Better to be a little too far away than a little too close.