Python and MongoDB: Connecting to NoSQL Databases – Real Python

By Real Python

While PyMongo is a great and powerful Python driver for interfacing with MongoDB, it’s probably a bit too low-level for many of your projects. With PyMongo, you’ll have to write a lot of code to consistently insert, retrieve, update, and delete documents.

One library that provides a higher abstraction on top of PyMongo is MongoEngine. MongoEngine is an object-document mapper (ODM), which is roughly equivalent to an SQL-based object-relational mapper (ORM). MongoEngine provides a class-based abstraction, so all the models you create are classes.

There are a handful of Python libraries to help you work with MongoDB. MongoEngine, however, is a popular one that provides a nice set of features, flexibility, and community support. MongoEngine is available on PyPI. You can install it using the following pip command:

$ pip install mongoengine==0.22.1

Once you’ve installed MongoEngine into your Python environment, you’re ready to start working with MongoDB databases using Python’s object-oriented features. The next step is to connect to your running MongoDB instance.

To establish a connection with your database, you need to use mongoengine.connect(). This function takes several arguments. However, in this tutorial, you’ll use only three of them. Within your Python interactive session, type the following code:

>>>
>>> from mongoengine import connect
>>> connect(db="rptutorials", host="localhost", port=27017)
MongoClient(host=['localhost:27017'], ..., read_preference=Primary())

Here, you first set the database name db to "rptutorials", which is the name of the database you want to work in. Then you provide a host and a port to connect to your current MongoDB instance. Since you’re using the default host and port, you can omit these two parameters and just use connect("rptutorials").

To create documents with MongoEngine, you first need to define what data you want the documents to have. In other words, you need to define a document schema. MongoEngine encourages you to define a document schema to help you reduce coding errors and to allow you to define utility or helper methods.

Similar to ORMs, ODMs like MongoEngine provide a base or model class for you to define a document schema. In ORMs, that class is equivalent to a table, and its instances are equivalent to rows. In MongoEngine, the class is equivalent to a collection, and its instances are equivalent to documents.

To create a model, you need to subclass Document and provide the required fields as class attributes. To continue with the blog example, here’s how you can create a model for your tutorials:

>>>
>>> from mongoengine import Document, ListField, StringField, URLField >>> class Tutorial(Document):
...  title = StringField(required=True, max_length=70)
...  author = StringField(required=True, max_length=20)
...  contributors = ListField(StringField(max_length=20))
...  url = URLField(required=True)

With this model, you tell MongoEngine that you expect a Tutorial document to have a .title, an .author, a list of .contributors, and a .url. The base class, Document, uses that information along with the field types to validate the input data for you.

Note: One of the more difficult tasks with database models is data validation. How do you make sure that the input data conforms to your format requirements? That’s one of the reasons for you to have a coherent and uniform document schema.

MongoDB is said to be a schemaless database, but that doesn’t mean it’s schema free. Having documents with a different schema within the same collection can lead to processing errors and inconsistent behavior.

For example, if you try to save a Tutorial object without a .title, then your model throws an exception and lets you know. You can take this even further and add more restrictions, such as the length of the .title, and so on.

There are a few general parameters that you can use to validate fields. Here are some of the more commonly used parameters:

  • db_field specifies a different field name.
  • required ensures that the field is provided.
  • default provides a default value for a given field if no value is given.
  • unique ensures that no other document in the collection has the same value for this field.

Each specific field type also has its own set of parameters. You can check the documentation for a complete guide to the available field types.

To save a document to your database, you need to call .save() on a document object. If the document already exists, then all the changes will be applied to the existing document. If the document doesn’t exist, then it’ll be created.

Here’s an example of creating and saving a tutorial into your sample tutorials database:

>>>
>>> tutorial1 = Tutorial(
...  title="Beautiful Soup: Build a Web Scraper With Python",
...  author="Martin",
...  contributors=["Aldren", "Geir Arne", "Jaya", "Joanna", "Mike"],
...  url="https://realpython.com/beautiful-soup-web-scraper-python/"
... ) >>> tutorial1.save() # Insert the new tutorial
<Tutorial: Tutorial object>

By default, .save() inserts the new document into a collection named after the model class, Tutorial, except using lowercase letters. In this case, the collection name is tutorial, which matches the collection you’ve been using to save your tutorials.

PyMongo performs data validation when you call .save(). This means that it checks the input data against the schema you declared in the Tutorial model class. If the input data violates the schema or any of its constraints, then you get an exception, and the data isn’t saved into the database.

For example, here’s what happens if you try to save a tutorial without providing a .title:

>>>
>>> tutorial2 = Tutorial()
>>> tutorial2.author = "Alex"
>>> tutorial2.contributors = ["Aldren", "Jon", "Joanna"]
>>> tutorial2.url = "https://realpython.com/convert-python-string-to-int/"
>>> tutorial2.save()
Traceback (most recent call last): ...
mongoengine.errors.ValidationError: ... (Field is required: ['title'])

In this example, first note that you can also build a Tutorial object by assigning values to its attributes. Second, since you don’t provide a .title for the new tutorial, .save() raises a ValidationError telling you that the .title field is required. Having automatic data validation is a great feature that will save you some headaches.

Each Document subclass has an .objects attribute that you can use to access the documents in the associated collection. For example, here’s how you can print the .title of all your current tutorials:

>>>
>>> for doc in Tutorial.objects:
...  print(doc.title)
...
Reading and Writing CSV Files in Python
How to Iterate Through a Dictionary in Python
Python 3's f-Strings: An Improved String Formatting Syntax (Guide)
Working With JSON Data in Python
Python's Requests Library (Guide)
Object-Oriented Programming (OOP) in Python 3
Beautiful Soup: Build a Web Scraper With Python

The for loop iterates over all your tutorials and prints their .title data to the screen. You can also use .objects to filter your documents. For example, say you want to retrieve the tutorials authored by Alex. In that case, you can do something like this:

>>>
>>> for doc in Tutorial.objects(author="Alex"):
...  print(doc.title)
...
Python's Requests Library (Guide)

MongoEngine is well suited to manage your MongoDB databases for just about any type of application. Its features make it ideal for creating efficient and scalable programs using a high-level approach. If you’re looking for more information about MongoEngine, be sure to check out its user guide.