Horses need a lot of room to roam, and it's hardly worth owning such a magnificent animal if you're not able to ride it. However many people who own a horse find themselves tiring of riding their horses around in a ring at the inner city equestrian club and yearn to get out into the country for some real riding. Horses enjoy the extra space too, so the experience can be good for both of you. Just be careful to stay off private property if you're going to roam further afield.
Trail riding is a popular hobby for horse owners and in many cases municipal, county, provincial and national parks will have some provision for horseback riders on the groomed trails that wind their way through public land. You should, however, check with the local authority before assuming that an empty stretch of land with a trail along it is public land, because it may be private property. Some landowners like to cut and maintain their own trails.
Your municipality may have its land ownership and use information mapped on its official website using GIS or the Geographic Information Service, however if it doesn't, a telephone call to the land registry office with the GPS coordinates or approximate location (e.g. across the street from the gas station) should yield the information you need. If the land is private property, you must stay off it unless given permission to use it.
If the area looks particularly enticing to you, you needn't give up on it forever, however. You can always contact the landowner for permission to pass through it on your rides. Many farmers and horse owners will have no objection to another rider who has introduced themselves and their animals using their trails. In other cases, the land might be unoccupied, owned by a city dweller who's trying to sell their current city home. In that case you may be able to buy or rent right of way on the land. In both cases, however, you should not allow your horse to graze while on their property unless you've bought those rights.
Of course, if you really want to ride in the country under your own rules, you'll need to move out of the city and into a rural area where you can buy significant acreage of your own.