{"id":4864,"date":"2018-03-22T06:37:43","date_gmt":"2018-03-22T13:37:43","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=4864"},"modified":"2025-06-13T23:43:11","modified_gmt":"2025-06-14T06:43:11","slug":"geospatial-queries-using-python-search-cities","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/geospatial-queries-using-python-search-cities\/","title":{"rendered":"Geospatial queries: Using Python to search cities"},"content":{"rendered":"

\"16665891<\/h3>\n

Daniel Ancuta is a software engineer with several years of experience using different technologies. He\u2019s a big fan of \u201cThe Zen of Python,\u201d which he tries to apply not only in his code but in his private life as well. You can find him on Twitter: @daniel_ancuta<\/a><\/u><\/p>\n

Geospatial queries: Using Python to search cities<\/strong><\/h2>\n

Geolocation information is used every day in almost every aspect of our interaction with computers. Either it’s a website that wants to send us personalized notifications based on location, maps that show us the shortest possible route, or just tasks running in the background that checks the places we’ve visited.<\/p>\n

Today, I’d like to introduce you to geospatial queries<\/a><\/u>\u00a0that are used in Couchbase. Geospatial queries<\/a><\/u>\u00a0allow you to search documents based on their geographical location.<\/p>\n

Together, we will write a tool in Python that uses geospatial queries<\/a><\/u>\u00a0with Couchbase REST API<\/a><\/u>\u00a0and Couchbase Full Text Search<\/a><\/u>, which will help us in searching a database of cities.<\/p>\n

Prerequisites<\/strong><\/h2>\n

Dependencies<\/strong><\/h3>\n

In this article I used Couchbase Enterprise Edition<\/a> 5.1.0 build 5552\u00a0and Python<\/a> 3.6.4.<\/p>\n

To run snippets from this article you should install Couchbase\u00a02.3 (I am using 2.3.4) via pip.<\/p>\n

Couchbase<\/strong><\/h3>\n
    \n
  1. Create a cities\u00a0bucket<\/li>\n
  2. Create a cities\u00a0search with geo\u00a0field of type geopoint\u00a0type. You can read about it in the Inserting a Child Field<\/a><\/u>\u00a0part of the documentation.<\/li>\n<\/ol>\n

    It should look like the image below:<\/p>\n

    \"image<\/p>\n

    Populating Couchbase with data<\/strong><\/h2>\n

    First of all, we need to have data for our exercise. For that, we will use a database of cities from geonames.org<\/a><\/u>.<\/p>\n

    GeoNames contains two main databases: list of cities and list of postal codes.<\/p>\n

    All are grouped by country with corresponding information like name, coordinates, population, time zone, country code, and so on. Both are in CSV format.<\/p>\n

    For the purpose of this exercise, we will use the list of cities. I’ve used PL.zip<\/a><\/u>\u00a0but feel free to choose whichever you prefer from the list of cities<\/a><\/u>.<\/p>\n

    Data model<\/strong><\/h3>\n

    City\u00a0class will be our representation of a single city that we will use across the whole application. By encapsulating it in a model, we unify the API and don’t need to rely on third-party data sources (e.g., CSV file) which might change.<\/p>\n

    Most of our snippets are located (until said otherwise) in the core.py\u00a0file. So just remember to update it (especially when adding new imports) and not override the whole content.<\/p>\n

    # core.py\r\nclass City:\r\n   def __init__(self, geonameid, feature_class, name, population, lat, lon):\r\n       self.geonameid = geonameid\r\n       self.feature_class = feature_class\r\n       self.name = name\r\n       self.population = population\r\n       self.lat = lat\r\n       self.lon = lon\r\n\r\n   @classmethod\r\n   def from_csv_row(cls, row):\r\n       return cls(row[0], row[7], row[1], row[12], row[4], row[5])<\/pre>\n
     <\/p>\n
    CSV iterator to process cities<\/strong><\/h3>\n
    As we have a model class, it’s time to prepare an iterator that will help us to read the cities from the CSV file.<\/p>\n
    # core.py\r\nimport csv\r\nfrom collections import Iterator\r\n\r\nclass CitiesCsvIterator(Iterator):\r\n   def __init__(self, path):\r\n       self._path = path\r\n       self._fp = None\r\n       self._csv_reader = None\r\n\r\n   def __enter__(self):\r\n       self._fp = open(self._path, 'r')\r\n       self._csv_reader = csv.reader(self._fp, delimiter='\\t')\r\n\r\n       return self\r\n\r\n   def __exit__(self, exc_type, exc_val, exc_tb):\r\n       self._fp.close()\r\n\r\n   def __next__(self):\r\n       return City.from_csv_row(next(self._csv_reader))\r\n<\/pre>\n
    Insert cities to Couchbase bucket<\/strong><\/h3>\n
    We have unified the way to represent a city, and we have an iterator that would read those from csv file.<\/p>\n
    It’s time to put this data into our main data source, Couchbase<\/a><\/u>.<\/p>\n
    # core.py\r\nimport logging\r\nimport sys\r\nfrom couchbase.cluster import Cluster, PasswordAuthenticator\r\n\r\n\r\nlogger = logging.getLogger()\r\nlogger.setLevel(logging.DEBUG)\r\nlogger.addHandler(logging.StreamHandler(sys.stdout))\r\n\r\n\r\ndef get_bucket(username, password, connection_string='couchbase:\/\/localhost'):\r\n   cluster = Cluster(connection_string)\r\n   authenticator = PasswordAuthenticator(username, password)\r\n   cluster.authenticate(authenticator)\r\n\r\n   return cluster.open_bucket('cities')\r\n\r\n\r\nclass CitiesService:\r\n   def __init__(self, bucket):\r\n       self._bucket = bucket\r\n\r\n   def load_from_csv(self, path):\r\n       with CitiesCsvIterator(path) as cities_iterator:\r\n           for city in cities_iterator:\r\n               if city.feature_class not in ('PPL', 'PPLA', 'PPLA2', 'PPLA3',\r\n                                             'PPLA4', 'PPLC'):\r\n                   continue\r\n\r\n               logger.info(f'Inserting {city.geonameid}')\r\n               self._bucket.upsert(\r\n                   city.geonameid,\r\n                   {\r\n                       'name': city.name,\r\n                       'feature_class': city.feature_class,\r\n                       'population': city.population,\r\n                       'geo': {'lat': float(city.lat),\r\n                               'lon': float(city.lon)}\r\n                   }\r\n               )\r\n\r\n<\/pre>\n
     <\/p>\n
    To check if everything we wrote so far is working, let’s load CSV content into Couchbase.<\/p>\n
    # core.py\r\n\r\nbucket = get_bucket('admin', 'test123456')\r\ncities_service = CitiesService(bucket)\r\ncities_service.load_from_csv('~\/directory-with-cities\/PL\/PL.txt', bucket)<\/pre>\n
     <\/p>\n
    At this point you should have cities loaded into your Couchbase bucket. The time it takes depends on the country you have chosen.<\/p>\n
    Search cities<\/strong><\/h2>\n
    We have our bucket ready with data, so it’s time to come back to CitiesService\u00a0and prepare a few methods that would help us in searching cities.<\/p>\n
    But before we start, we need to modify the City\u00a0class slightly, by adding the following method:<\/p>\n
    # core.py\r\n\r\n@classmethod\r\ndef\u00a0from_couchbase_dict(cls, row):\r\nfields =\u00a0row['fields']\r\n\r\nreturn\u00a0cls(row['id'],\r\nfields['feature_class'],\r\nfields['name'],\r\nfields['population'],\r\nfields['geo'][1],\r\nfields['geo'][0])\r\n\r\n<\/pre>\n
    That’s a list of methods we will implement in CitiesService:<\/p>\n