Index Queries

DynamoDB supports two types of indexes: global secondary indexes, and local secondary indexes. Indexes can make accessing your data more efficient, and should be used when appropriate. See the documentation for more information.

Index Settings

The Meta class is required with at least the projection class attribute to specify the projection type. For Global secondary indexes, the read_capacity_units and write_capacity_units also need to be provided. By default, PynamoDB will use the class attribute name that you provide on the model as the index_name used when making requests to the DynamoDB API. You can override the default name by providing the index_name class attribute in the Meta class of the index.

Global Secondary Indexes

Indexes are defined as classes, just like models. Here is a simple index class:

from pynamodb.indexes import GlobalSecondaryIndex, AllProjection
from pynamodb.attributes import NumberAttribute


class ViewIndex(GlobalSecondaryIndex):
    """
    This class represents a global secondary index
    """
    class Meta:
        # index_name is optional, but can be provided to override the default name
        index_name = 'foo-index'
        read_capacity_units = 2
        write_capacity_units = 1
        # All attributes are projected
        projection = AllProjection()

    # This attribute is the hash key for the index
    # Note that this attribute must also exist
    # in the model
    view = NumberAttribute(default=0, hash_key=True)

Global indexes require you to specify the read and write capacity, as we have done in this example. Indexes are said to project attributes from the main table into the index. As such, there are three styles of projection in DynamoDB, and PynamoDB provides three corresponding projection classes.

• AllProjection: All attributes are projected.

• KeysOnlyProjection: Only the index and primary keys are projected.

• IncludeProjection(attributes): Only the specified attributes are projected.

We still need to attach the index to the model in order for us to use it. You define it as a class attribute on the model, as in this example:

from pynamodb.models import Model
from pynamodb.attributes import UnicodeAttribute


class TestModel(Model):
    """
    A test model that uses a global secondary index
    """
    class Meta:
        table_name = 'TestModel'
    forum = UnicodeAttribute(hash_key=True)
    thread = UnicodeAttribute(range_key=True)
    view_index = ViewIndex()
    view = NumberAttribute(default=0)

Local Secondary Indexes

Local secondary indexes are defined just like global ones, but they inherit from LocalSecondaryIndex instead:

from pynamodb.indexes import LocalSecondaryIndex, AllProjection
from pynamodb.attributes import NumberAttribute


class ViewIndex(LocalSecondaryIndex):
    """
    This class represents a local secondary index
    """
    class Meta:
        # All attributes are projected
        projection = AllProjection()
    forum = UnicodeAttribute(hash_key=True)
    view = NumberAttribute(range_key=True)

Every local secondary index must meet the following conditions:

• The partition key (hash key) is the same as that of its base table.

• The sort key (range key) consists of exactly one scalar attribute. The range key can be any attribute.

• The sort key (range key) of the base table is projected into the index, where it acts as a non-key attribute.

Querying an index

Index queries use the same syntax as model queries. Continuing our example, we can query the view_index global secondary index simply by calling query:

for item in TestModel.view_index.query(1):
    print("Item queried from index: {0}".format(item))

This example queries items from the table using the global secondary index, called view_index, using a hash key value of 1 for the index. This would return all TestModel items that have a view attribute of value 1.

Local secondary index queries have a similar syntax. They require a hash key, and can include conditions on the range key of the index. Here is an example that queries the index for values of view greater than zero:

for item in TestModel.view_index.query('foo', TestModel.view > 0):
    print("Item queried from index: {0}".format(item.view))

Pagination and last evaluated key

The query returns a ResultIterator object that transparently paginates through results. To stop iterating and allow the caller to continue later on, use the last_evaluated_key property of the iterator:

def iterate_over_page(last_evaluated_key = None):
    results = TestModel.view_index.query('foo', TestModel.view > 0,
                                         limit=10,
                                         last_evaluated_key=last_evaluated_key)
    for item in results:
       ...
    return results.last_evaluated_key

The last_evaluated_key is effectively the key attributes of the last iterated item; the next returned items will be the items following it. For index queries, the returned last_evaluated_key will contain both the table’s hash/range keys and the indexes hash/range keys. This is due to the fact that DynamoDB indexes have no uniqueness constraint, i.e. the same hash/range pair can map to multiple items. For the example above, the last_evaluated_key will look like:

{
    "forum": {"S": "..."},
    "thread": {"S": "..."},
    "view": {"N": "..."}
}