Listing Items

The List family of APIs are one of the most useful APIs in StatelyDB. While APIs like Put, Get, and Delete allow you to operate on individual Items by their full key path, List lets you fetch multiple Items from the same Group in one go (i.e. Items that share the same Group Key). This can be useful for fetching collections of Items such as a customer’s order history, or a list of messages in a conversation. It’s especially useful for implementing features like infinite scrolling lists, where you want to keep fetching more Items as the user scrolls down.

Beginning a Paginated List

You start a list by calling BeginList with a key path prefix and an optional limit to get an initial result set. StatelyDB will return all the Items whose key paths have the given prefix.

Your initial result set might not contain all the Items under that prefix, if you set a page size limit. This initial result set is your first “page” of data - imagine it as the first several screens full of emails in your email app.

For this example we’ll use the schema defined in Example: Movies Schema, which defines these key paths (among others):

Item Type	Key Path Template
Movie	/movie-:id
Character	/movie-:movieId/role-:role/name-:name

Note how both Movie and Character share the same /movie-:id prefix.

Go

package main

import (
  "context"
  "fmt"
  "os"
  "slices"
  "strconv"
  "time"

  "github.com/google/uuid"

  "github.com/StatelyCloud/go-sdk/stately"
  // This is the code you generated from schema
  "github.com/StatelyCloud/stately/go-sdk-sample/schema"
)

func sampleList(
  ctx context.Context,
  client stately.Client,
  movieID uuid.UUID,
) (*stately.ListToken, error) {
  iter, err := client.BeginList(
    ctx,
    // This key path is a prefix of BOTH Movie and Character.
    "/movie-"+stately.ToKeyID(movieID[:]),
    stately.ListOptions{Limit: 10},
  )
  if err != nil {
    return nil, err
  }

  for iter.Next() {
    item := iter.Value()
    switch v := item.(type) {
    case *schema.Movie:
      fmt.Printf("Movie Title: %s\n", v.GetTitle())
    case *schema.Character:
      fmt.Printf("Character Name: %s\n", v.GetName())
    }
  }
  // When we've exhausted the iterator, we'll get a token that we
  // can use to fetch the next page of items.
  return iter.Token()
}

Ruby

require 'bundler/setup'
require_relative 'schema/stately'
require 'byebug'


def sample_list(client, movie_id)
  # This key path is a prefix of BOTH Movie and Character.
  key_path = StatelyDB::KeyPath.with('movie', movie_id)
  begin_list_result, token = client.begin_list(key_path, limit: 10)

  begin_list_result.each do |item|
    case item
    when StatelyDB::Types::Movie
      puts "[Movie] title: #{item.title}"
    when StatelyDB::Types::Character
      puts "[Character] name: #{item.name}"
    end
  end

  return token
end

Python

from __future__ import annotations

from typing import TYPE_CHECKING

from statelydb import ListToken, SyncChangedItem, SyncDeletedItem, SyncReset, key_path

from .schema import Actor, Change, Character, Client, Movie


async def sample_list(client: Client, movie_id: UUID) -> ListToken:
    # This key path is a prefix of BOTH Movie and Character.
    prefix = key_path("movie-{id}", id=movie_id)
    list_resp = await client.begin_list(prefix, limit=10)

    async for item in list_resp:
        if isinstance(item, Movie):
            print(f"[Movie] title: {item.title}")
        elif isinstance(item, Character):
            print(f"[Character] name: {item.name}")

    # When we've exhausted the iterator, we'll get a token that we can
    # use to fetch the next page of items.
    return list_resp.token

TypeScript

import { createClient, DatabaseClient, Movie } from "./schema/index.js";
import { keyPath, ListToken } from "@stately-cloud/client";

async function sampleList(
  client: DatabaseClient,
  movieId: Uint8Array,
): Promise<ListToken> {
  // This key path is a prefix of BOTH Movie and Character.
  const prefix = keyPath`/movie-${movieId}`;
  let iter = client.beginList(prefix, {
    limit: 10,
  });
  for await (const item of iter) {
    if (client.isType(item, "Movie")) {
      console.log("Movie:", item.title);
    } else if (client.isType(item, "Character")) {
      console.log("Character:", item.name);
    }
  }
  return iter.token!;
}

CLI

# There are no key path helpers for shell, so you need to
# manually base64-encode the UUID's bytes
stately item list \
  --store-id <store-id-goes-here> \
  --item-path-prefix '/movie-2hC3sMFFSlelJlFf9hRD9g'

Using the List Token to Continue

The result from BeginList includes a token which you can save for later, or use right away. Only the tokenData part of the token needs to be saved. There is also a canContinue property which indicates whether there are more pages still available, and canSync which indicates whether SyncList is supported for this list.

You can pass the token to ContinueList, which lets you fetch more results for your result set, continuing from where you left off. For example, you might call ContinueList to get the next few screens of emails when the user scrolls down in their inbox. Or, you could call it in the background to eventually pull the entire result set into a local database. All you need is the token—the original arguments to BeginList are saved with it.

The token keeps track of the state of the result set, and ContinueList allows you to expand the window of results that you have retrieved. Every time you call ContinueList, you’ll get a new token back, and you can use that token for the next ContinueList or SyncList call, and so on. This is also known as pagination, and it allows you to quickly show results to your users without having to get all the data at once, while still having the ability to grab the next results consistently.

For many applications, you’ll only need to call BeginList once, to set up the initial token, and from then on they’ll call ContinueList (as a user scrolls through results) or SyncList (whenever the user opens or focuses the application, to check for new and updated items).

In this example we’ve passed the token from the first example back into ContinueList to keep getting more Items:

Go

package main

import (
  "context"
  "fmt"
  "os"
  "slices"
  "strconv"
  "time"

  "github.com/google/uuid"

  "github.com/StatelyCloud/go-sdk/stately"
  // This is the code you generated from schema
  "github.com/StatelyCloud/stately/go-sdk-sample/schema"
)

func sampleContinueList(
  ctx context.Context,
  client stately.Client,
  token *stately.ListToken,
) (*stately.ListToken, error) {
  iter, err := client.ContinueList(ctx, token.Data)
  if err != nil {
    return nil, err
  }
  for iter.Next() {
    item := iter.Value()
    switch v := item.(type) {
    case *schema.Character:
      fmt.Printf("Character Name: %s\n", v.GetName())
    case *schema.Movie:
      fmt.Printf("Movie Title: %s\n", v.GetTitle())
    }
  }
  // You could save the token to call ContinueList later.
  return iter.Token()
}

Ruby

require 'bundler/setup'
require_relative 'schema/stately'
require 'byebug'


def sample_continue_list(client, token)
  # Fetch the next page of items
  continue_list_result, token = client.continue_list(token)

  continue_list_result.each do |item|
    case item
    when StatelyDB::Types::Movie
      puts "[Movie] title: #{item.title}"
    when StatelyDB::Types::Character
      puts "[Character] name: #{item.name}"
    end
  end

  # You could save the token to call ContinueList later.
  return token
end

Python

from __future__ import annotations

from typing import TYPE_CHECKING

from statelydb import ListToken, SyncChangedItem, SyncDeletedItem, SyncReset, key_path

from .schema import Actor, Change, Character, Client, Movie


async def sample_continue_list(client: Client, token: str) -> ListToken:
    # Fetch the next page of items
    continue_list_result = await client.continue_list(token)

    # Print out the paths of the next batch of listed items
    async for item in continue_list_result:
        if isinstance(item, Movie):
            print(f"[Movie] title: {item.title}")
        elif isinstance(item, Character):
            print(f"[Character] name: {item.name}")

    # You could save the token to call ContinueList later.
    return continue_list_result.token

TypeScript

import { createClient, DatabaseClient, Movie } from "./schema/index.js";
import { keyPath, ListToken } from "@stately-cloud/client";

async function sampleContinueList(
  client: DatabaseClient,
  token: ListToken,
): Promise<ListToken> {
  // You can call `collect` on the iterator to pull
  // all the items into an Array.
  const { items, token: newToken } = await client
    .continueList(token)
    .collect();

  for (const item of items) {
    if (client.isType(item, "Movie")) {
      console.log("Movie:", item.title);
    } else if (client.isType(item, "Actor")) {
      console.log("Actor:", item.name);
    } else if (client.isType(item, "Character")) {
      console.log("Character:", item.name);
    }
  }
  // You could save the token to call ContinueList later.
  return newToken;
}

CLI

#!/usr/bin/env bash

# begin-sample: update
stately item put \
  --store-id <store-id-goes-here> \
  --item-type 'Movie' \
  --item-data '{
    "id": "2hC3sMFFSlelJlFf9hRD9g",
    "title": "Starship Troopers",
    "rated": "R",
    "duration_seconds": 7740,
    "genre": "Sci-Fi",
    "year": 1997
  }'
# end-sample

# begin-sample: put
stately item put \
  --store-id <store-id-goes-here> \
  --item-type 'Movie' \
  --item-data '{
    "title": "Starship Troopers 2",
    "year": 2004,
    "genre": "Sci-Fi",
    "duration": 7880,
    "rating": "R",
  }'
# end-sample

# begin-sample: get
# There are no key path helpers for shell, so you need to
# manually base64-encode the UUID's bytes
stately item get \
  --store-id <store-id-goes-here> \
  --item-key '/movie-2hC3sMFFSlelJlFf9hRD9g'
# end-sample

# begin-sample: delete
# There are no key path helpers for shell, so you need to
# manually base64-encode the UUID's bytes
stately item delete \
  --store-id <store-id-goes-here> \
  --item-key '/movie-2hC3sMFFSlelJlFf9hRD9g'
# end-sample

# begin-sample: list
# There are no key path helpers for shell, so you need to
# manually base64-encode the UUID's bytes
stately item list \
  --store-id <store-id-goes-here> \
  --item-path-prefix '/movie-2hC3sMFFSlelJlFf9hRD9g'
# end-sample

Sort Direction

By default, the items returned in a List are sorted by their key paths, in ascending order. Namespaces, string IDs, and byte IDs are sorted lexicographically, while number IDs are sorted numerically. For example:

• /customer-1234
• /customer-1234/order-9
• /customer-1234/order-10
• /customer-1234/order-10/li-abc
• /customer-1234/order-10/li-bcd

You can specify a SortDirection option to reverse this order (from the default of Ascending to Descending).

Listing All Groups

Right now there is no API in StatelyDB for listing out your top level Groups. So, for example, if your Groups were based on customers, there would not be an API to list all customers. Because of the way StatelyDB distributes Groups amongst partitions within the database (and eventually, across different databases around the world), listing all of the top level Groups would be slow and expensive. Most applications will not need to perform an operation like this if the Group Key is chosen well - for example it is not frequently necessary to list out all users of an application in one place. However, we can help if you are looking to figure out how to work around this limitation.

Listing Across Client Upgrades

The list token you get from BeginList is specific to the schema version your client was built with. If you save that token, and then upgrade your client to another schema version, ContinueList will return a SchemaVersionMismatch error, since there’s no guarantee that the items you had cached are compatible with the new schema version. In this case you should discard the list token and start a new BeginList call.