Migrations

When you update your schema, you need to change your schema definition in your TypeScript files, but you also need to specify migrations that describe what changes you’ve made. This helps to double-check that you’ve made the changes you intend to, and also resolves some ambiguities - for example, did you mean to rename a field, or did you add a new field and delete an old field? This is part of what allows StatelyDB’s elastic schema to automatically convert between different schema versions and allow different versions of your clients to coexist.

Declaring Migrations

Whenever you update your schema, you need to declare a migration with the migrate function. Just like your schema types, the migration needs to be exported from your schema’s top level JavaScript module, and you can use modules to organize them into different files however you like.

The migrate function requires the schema version you’re migrating from (they start with 1), a human-readable description, and a function that applies one or more migration commands. You can look up the current version number of your schema on the console. Because these migration declarations have a version associated with them, you don’t need to delete them after they’ve been applied—you can keep them around as a record of changes to your schema, or you can clean them out of your schema files after they’re no longer needed.

1
import { migrate } from "@stately-cloud/schema";
2
// ... the rest of your schema, other migrations, etc.
3

4
// Declare a migration from version 1 to version 2 that describes a
5
// bunch of changes to the schema.
6
export const MyMigration = migrate(1, "An example migration", (m) => {
7
  m.addType("NewType");
8
  m.removeType("Url");
9
  m.renameType("Account", "Profile");
10
  m.changeType("Profile", (t) => {
11
    // was previously named "Account"
12
    t.addField("description");
13
    t.removeField("note");
14
    t.renameField("joinDate", "accountAge");
15
  });
16
});

Migration Commands

Migration commands are the individual instructions in each migration declaration that tell StatelyDB how your schema’s shape has changed. When you call stately schema put, these commands help StatelyDB understand how to migrate items between versions when reading or writing them. The available commands are:

AddType

You can declare that you’ve added a new objectType, itemType, or enumType by passing the name of your new type to the addType migration. This type must exist in your new schema version and not the old version.

Clients using this new version can read and write using this new type, while older versions won’t see it. StatelyDB will filter this new type out of List/SyncList results for clients on older versions.

1
export const NewType = itemType("NewType", {
2
  keyPath: "/newType-:name",
3
  fields: {
4
    name: { type: string },
5
  },
6
});
7

8
export const AddType = migrate(2, "An add type example", (m) => {
9
  m.addType("NewType");
10
});

RemoveType

When you remove a objectType, itemType or enumType, use the removeType command and pass the name of the type you have removed.

When you remove a type from your schema, newer schema versions will no longer see this type. The items or fields that reference the removed type are still accessible for older versions of the schema but newer versions of the schema cannot interact with them.

1
export const ExistingType = itemType("ExistingType", {
2
  keyPath: "/et-:id",
3
  fields: {
4
    id: { type: string },
5
  },
6
});
7

8
export const RemoveType = migrate(3, "A remove type example", (m) => {
9
  m.removeType("ExistingType");
10
});

RenameType

When you rename a type in your schema, use the renameType command to clarify that it’s the same type, and not a new type plus a removed type. If you reference this type in subsequent migration actions, you must use the new name.

When you rename a type, the data stored in that type doesn’t change. Clients using older schema versions will use the old name, while clients on newer version will use the new name, but they’re all interacting with the same data.

1
export const OldType = itemType("OldType", {
2
export const NewType = itemType("NewType", {
3
 keyPath: "/t-:name",
4
 fields: {
5
   name: { type: string },
6
 },
7
});
8

9
export const RenameType = migrate(4, "A rename type example", (m) => {
10
  m.renameType("OldType", "NewType");
11
});

AddField

When you add a new field to a type, use the addField command within a changeType command to declare the new field.

Clients using newer schema versions can read and write to this added field, while older versions won’t see this field at all. If a client using an older version updates an item that has added fields, those fields will be untouched. In other words, there is no way to replace the whole object—StatelyDB only updates the fields known to the caller’s schema version. If you want to completely replace an item, make sure to delete it first.

Adding required fields to an existing type must be accompanied by a readDefault to ensure newer clients can read items created by older clients that didn’t know about this field. The provided readDefault will be subject to any valid expression on the new field.

1
export const ExistingType = itemType("ExistingType", {
2
 keyPath: "/et-:id",
3
 fields: {
4
   id: { type: string },
5
   name: { type: string, readDefault: "John Doe" },
6
 },
7
});
8

9
export const AddField = migrate(5, "An add field example", (m) => {
10
  m.changeType("ExistingType", (i) => {
11
    i.addField("name");
12
  });
13
});

RemoveField

When you’ve removed a field from a type, use the removeField command within a changeType command.

Clients using older schema versions will still see the field, while newer clients cannot interact with the field but will not disturb any data already saved there. If you want to zero out this removed field for older clients, you will need to delete the item first.

Removing required fields from an existing type must be accompanied by a readDefault in the migration command to ensure older clients can read items created by newer clients that no longer know about this field. The provided readDefault will be subject to any valid expression on the removed field.

1
export const ExistingType = itemType("ExistingType", {
2
 keyPath: "/et-:id",
3
 fields: {
4
   id: { type: string },
5
   name: { type: string },
6
 },
7
});
8

9
export const RemoveField = migrate(6, "A remove field example", (m) => {
10
  m.changeType("ExistingType", (i) => {
11
    i.removeField("name", "John Doe");
12
  });
13
});

RenameField

If you’ve renamed a field in a type, use the renameField command in a changeType command, to distinguish a rename from an add and a remove. Renamed fields keep the same stored data and only change the name. This means that clients on different schema versions of the schema will read and write the same data, just with different field names.

1
export const ExistingType = itemType("ExistingType", {
2
 keyPath: "/et-:id",
3
 fields: {
4
   id: { type: string },
5
   oldName: { type: string },
6
   newName: { type: string },
7
 },
8
});
9

10
export const RenameField = migrate(7, "A rename field example", (m) => {
11
  m.changeType("ExistingType", (i) => {
12
    i.renameField("oldName", "newName");
13
  });
14
});

ModifyFieldReadDefault

If you’ve changed the default value of a field, by either adding, removing or modifying the readDefault value, use the modifyFieldReadDefault command within a changeType command. The provided readDefault will be subject to any valid expression on the field.

Note: You can never add or remove a required field’s readDefault value, only modify the existing value. This is because required fields must always have a value.

1
export const ExistingType = itemType("ExistingType", {
2
keyPath: "/et-:id",
3
fields: {
4
  id: { type: string },
5
  name: { type: string, readDefault: "John Doe" },
6
  name: { type: string, readDefault: "Jane Doe" },
7
},
8
});
9

10
export const ModifyFieldReadDefault = migrate(
11
  8,
12
  "A modify field readDefault example",
13
  (m) => {
14
    m.changeType("ExistingType", (i) => {
15
      i.modifyFieldReadDefault("name");
16
    });
17
  },
18
);

MarkFieldAsRequired

If you’ve changed a field to be required use the markFieldAsRequired command within a changeType command. Additionally, you must provide a readDefault on the field.

Newer clients will see this readDefault when reading items created by older clients that may have set the zero value for this field. The provided readDefault will be subject to any valid expression on the field.

1
export const ExistingType = itemType("ExistingType", {
2
keyPath: "/et-:id",
3
fields: {
4
  id: { type: string },
5
  name: { type: string, required: false },
6
  // required: true is implicit in schema
7
  name: { type: string, readDefault: "John Doe" },
8
},
9
});
10

11
export const MarkFieldAsRequired = migrate(
12
  9,
13
  "A required example",
14
  (m) => {
15
    m.changeType("ExistingType", (i) => {
16
      i.markFieldAsRequired("name");
17
    });
18
  },
19
);

MarkFieldAsNotRequired

If you’ve changed a field to not be required use the markFieldAsNotRequired command with a readDefault within a changeType command.

Older clients will see the readDefault when reading items modified by newer clients that set the zero value. This readDefault will be subject to any valid expression on the field.

1
export const ExistingType = itemType("ExistingType", {
2
 keyPath: "/et-:id",
3
 fields: {
4
   id: { type: string },
5
   // required: true is implicit in schema
6
   name: { type: string },
7
   name: { type: string, required: false },
8
 },
9
});
10

11
export const MarkFieldAsNotRequired = migrate(
12
  10,
13
  "A not required example",
14
  (m) => {
15
    m.changeType("ExistingType", (i) => {
16
      i.markFieldAsNotRequired("name", "John Doe");
17
    });
18
  },
19
);