Skip to content
Datanotes

Import existing notes

Your vault may already have notes with front matter, like a film diary, a reading list or meeting notes. The examples start from a Films/ folder of classic films written by hand, each with its year, runtime and Director. You turn it into a table without retyping anything: you infer the columns, create the table and adopt the notes. Then you decide which new notes become rows on their own.

Infer a table from notes

Section titled “Infer a table from notes”

infer_table reads the notes’ front matter and proposes a table, without writing anything. The proposal has:

  • A type for each key: an enum when a few short values repeat across enough notes
  • Shared keys: which front matter keys every note has
  • Valid column names: a key whose name isn’t a valid column name, like Director, gets a valid name and a from

infer_table

{ "folder": "Films", "name": "film" }
Answer (153 lines) 
{
  "files": 5,
  "paths": [
    "Films/Charade.md",
    "Films/His Girl Friday.md",
    "Films/Metropolis.md",
    "Films/Nosferatu.md",
    "Films/The Gold Rush.md"
  ],
  "skipped": [],
  "columns": [
    {
      "name": "director",
      "from": "Director",
      "type": "string",
      "description": null,
      "stats": {
        "present": 5,
        "filled": 5,
        "distinct": 5,
        "types": { "string": 5 },
        "examples": ["Stanley Donen", "Howard Hawks", "Fritz Lang"]
      },
      "required": true,
      "unique": true,
      "dense": true
    },
    {
      "name": "minutes",
      "type": "integer",
      "description": null,
      "stats": {
        "present": 5,
        "filled": 5,
        "distinct": 5,
        "types": { "integer": 5 },
        "examples": [113, 92, 153]
      },
      "required": true,
      "unique": true,
      "dense": true
    },
    {
      "name": "status",
      "type": "enum",
      "description": null,
      "stats": {
        "present": 5,
        "filled": 5,
        "distinct": 2,
        "types": { "string": 5 },
        "examples": ["watched", "watched", "to watch"]
      },
      "values": ["watched", "to watch"],
      "required": true,
      "dense": true
    },
    {
      "name": "year",
      "type": "integer",
      "description": null,
      "stats": {
        "present": 5,
        "filled": 5,
        "distinct": 5,
        "types": { "integer": 5 },
        "examples": [1963, 1940, 1927]
      },
      "required": true,
      "unique": true,
      "dense": true
    },
    {
      "name": "rating",
      "type": "integer",
      "description": null,
      "stats": {
        "present": 4,
        "filled": 4,
        "distinct": 2,
        "types": { "integer": 4 },
        "examples": [4, 5, 5]
      }
    },
    {
      "name": "watched_on",
      "type": "date",
      "description": null,
      "stats": {
        "present": 4,
        "filled": 4,
        "distinct": 4,
        "types": { "date": 4 },
        "examples": ["2026-09-12", "2026-07-20", "2026-03-14"]
      }
    }
  ],
  "proposal": {
    "name": "film",
    "description": null,
    "columns": [
      {
        "name": "director",
        "type": "string",
        "description": null,
        "required": true,
        "unique": true,
        "dense": true
      },
      {
        "name": "minutes",
        "type": "integer",
        "description": null,
        "required": true,
        "unique": true,
        "dense": true
      },
      {
        "name": "status",
        "type": "enum",
        "description": null,
        "values": ["watched", "to watch"],
        "required": true,
        "dense": true
      },
      {
        "name": "year",
        "type": "integer",
        "description": null,
        "required": true,
        "unique": true,
        "dense": true
      },
      { "name": "rating", "type": "integer", "description": null },
      { "name": "watched_on", "type": "date", "description": null }
    ]
  },
  "adopt": {
    "paths": [
      "Films/Charade.md",
      "Films/His Girl Friday.md",
      "Films/Metropolis.md",
      "Films/Nosferatu.md",
      "Films/The Gold Rush.md"
    ],
    "rename": { "Director": "director" },
    "relocate": false
  },
  "hints": [
    "no column matches the file names: set filename (e.g. \"{title}\") and adopt with relocate: false to keep the current names",
    "write the description of the table and of each column (null in the proposal) before create_table: it refuses them empty"
  ]
}

Read the proposal and its notes before you use it. The notes can’t say what the table and its columns mean. So description is null on the table and on every column, for you to write. Here five columns also need a fix:

  • year, minutes and director: came out unique because no two of these films share them, which won’t stay true
  • status: an enum with its values in the order the notes gave them, watched first, where you want an ordered enum from to watch to watched
  • rating: an integer with no range, where you want 1 to 5
  • title: missing, because the notes keep the title in the note’s name, not in the front matter

Create the table from the proposal

Section titled “Create the table from the proposal”

create_table refuses the proposal as it is, because its descriptions are null. The refusal names every one you still have to write:

create_table

{
  "name": "film",
  "description": null,
  "columns": [
    {
      "name": "director",
      "type": "string",
      "description": null,
      "required": true,
      "unique": true,
      "dense": true
    },
    {
      "name": "minutes",
      "type": "integer",
      "description": null,
      "required": true,
      "unique": true,
      "dense": true
    },
    {
      "name": "status",
      "type": "enum",
      "description": null,
      "values": ["watched", "to watch"],
      "required": true,
      "dense": true
    },
    {
      "name": "year",
      "type": "integer",
      "description": null,
      "required": true,
      "unique": true,
      "dense": true
    },
    { "name": "rating", "type": "integer", "description": null },
    { "name": "watched_on", "type": "date", "description": null }
  ]
}

Refused:

pass "time_column" or "filename" (e.g. "{name}") so rows get file names
Hint: Check the arguments of create_table: name (string, required), description (string), columns (list of objects, required), catalog (string), schema (string), time_column (string), filename (string), partition (string), exclude (list of strings), archive (string), aliases (list of strings), history (boolean), autolink_target (boolean), row_template (string), fields (object), tags (object), dry_run (boolean).
Docs: https://datanotes.dev/reference/operations/schema.md#create_table (offline: read_docs("reference/operations/schema#create_table"))

This call adds a description to the table and to each column, and corrects what the inference guessed wrong:

  • year, minutes and director: no longer unique
  • status: an ordered enum (to watch < watched)
  • rating: from 1 to 5
  • title: a new column that holds the note’s file name (identity: "filename")

create_table

{
  "name": "film",
  "description": "Classic films I watched or want to watch",
  "columns": [
    {
      "name": "title",
      "type": "string",
      "identity": "filename",
      "description": "The film's title, which is also its note's name"
    },
    {
      "name": "director",
      "type": "string",
      "description": "Who directed the film",
      "required": true,
      "unique": false,
      "dense": true
    },
    {
      "name": "minutes",
      "type": "integer",
      "description": "Running time in minutes",
      "required": true,
      "unique": false,
      "dense": true
    },
    {
      "name": "status",
      "type": "enum",
      "description": "Where the film is on my list",
      "values": ["to watch", "watched"],
      "required": true,
      "dense": true,
      "ordered": true
    },
    {
      "name": "year",
      "type": "integer",
      "description": "Year the film first came out",
      "required": true,
      "unique": false,
      "dense": true
    },
    {
      "name": "rating",
      "type": "integer",
      "min": 1,
      "max": 5,
      "description": "My rating, from 1 (poor) to 5 (a masterpiece)"
    },
    { "name": "watched_on", "type": "date", "description": "The day I last watched it" }
  ],
  "filename": "{title}"
}

Answer:

{
  "ok": true,
  "op": "create_table",
  "planned_texts": ["datanotes/metastore.yaml"],
  "commits": [{ "table": "main.public.film", "version": 0, "operation": "CREATE TABLE" }],
  "migration_id": "20261012T165228-create_table-create-table-main-public-film"
}

Preview the adoption with a dry run

Section titled “Preview the adoption with a dry run”

adopt_rows makes existing notes rows of a table and keeps their text and links. It sets table and an id on each note, and applies these arguments:

  • rename: renames front matter keys to column names
  • set: sets the same values on every note
  • drop: removes the front matter keys the table doesn’t have
  • relocate: moves the notes to the table’s folder and rewrites the links to them

With dry_run, it shows what it would do and writes nothing.

adopt_rows

{
  "table": "film",
  "paths": [
    "Films/Charade.md",
    "Films/His Girl Friday.md",
    "Films/Metropolis.md",
    "Films/Nosferatu.md",
    "Films/The Gold Rush.md"
  ],
  "rename": { "Director": "director" },
  "relocate": true,
  "dry_run": true
}
Answer (70 lines) 
{
  "ok": true,
  "op": "adopt_rows",
  "dry_run": true,
  "planned_edits": [
    {
      "path": "Films/Charade.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abd8-7236-b38e-4c708c8ceff0",
        "director": "Stanley Donen",
        "title": "Charade"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/His Girl Friday.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abd8-7a81-8b39-0d5d66d2125c",
        "director": "Howard Hawks",
        "title": "His Girl Friday"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/Metropolis.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abd9-73da-8768-b71ed3d29c27",
        "director": "Fritz Lang",
        "title": "Metropolis"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/Nosferatu.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abd9-7dd3-a500-c842233bf60f",
        "director": "F. W. Murnau",
        "title": "Nosferatu"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/The Gold Rush.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abd9-7569-accf-489099f7eb3f",
        "director": "Charlie Chaplin",
        "title": "The Gold Rush"
      },
      "remove": ["Director"]
    }
  ],
  "planned_moves": [
    { "from": "Films/Charade.md", "to": "datanotes/main/public/film/Charade.md" },
    {
      "from": "Films/His Girl Friday.md",
      "to": "datanotes/main/public/film/His Girl Friday.md"
    },
    { "from": "Films/Metropolis.md", "to": "datanotes/main/public/film/Metropolis.md" },
    { "from": "Films/Nosferatu.md", "to": "datanotes/main/public/film/Nosferatu.md" },
    {
      "from": "Films/The Gold Rush.md",
      "to": "datanotes/main/public/film/The Gold Rush.md"
    }
  ]
}

Adopt the notes into the table

Section titled “Adopt the notes into the table”

This is the same call without dry_run. The engine checks every note first: a note that doesn’t fit the table stops the whole adoption, and the answer lists why.

With thousands of notes, adopt them in calls of about 1,000 paths each. Each call is one journal entry, so a stop or a crash interrupts less, and undo_migration reverts one chunk at a time.

adopt_rows

{
  "table": "film",
  "paths": [
    "Films/Charade.md",
    "Films/His Girl Friday.md",
    "Films/Metropolis.md",
    "Films/Nosferatu.md",
    "Films/The Gold Rush.md"
  ],
  "rename": { "Director": "director" },
  "relocate": true
}
Answer (73 lines) 
{
  "ok": true,
  "op": "adopt_rows",
  "notes_changed": 5,
  "files_moved": 5,
  "planned_edits": [
    {
      "path": "Films/Charade.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abdf-770a-a3af-34b2b65e8d80",
        "director": "Stanley Donen",
        "title": "Charade"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/His Girl Friday.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abdf-7c4f-b7f9-ccb2da41e9a6",
        "director": "Howard Hawks",
        "title": "His Girl Friday"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/Metropolis.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abdf-7118-af41-764cb14d66a6",
        "director": "Fritz Lang",
        "title": "Metropolis"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/Nosferatu.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abdf-7dd8-8ee7-e93f52567739",
        "director": "F. W. Murnau",
        "title": "Nosferatu"
      },
      "remove": ["Director"]
    },
    {
      "path": "Films/The Gold Rush.md",
      "set": {
        "table": "main.public.film",
        "id": "01a13107-abdf-77ae-8225-da069c86634f",
        "director": "Charlie Chaplin",
        "title": "The Gold Rush"
      },
      "remove": ["Director"]
    }
  ],
  "planned_moves": [
    { "from": "Films/Charade.md", "to": "datanotes/main/public/film/Charade.md" },
    {
      "from": "Films/His Girl Friday.md",
      "to": "datanotes/main/public/film/His Girl Friday.md"
    },
    { "from": "Films/Metropolis.md", "to": "datanotes/main/public/film/Metropolis.md" },
    { "from": "Films/Nosferatu.md", "to": "datanotes/main/public/film/Nosferatu.md" },
    {
      "from": "Films/The Gold Rush.md",
      "to": "datanotes/main/public/film/The Gold Rush.md"
    }
  ],
  "commits": [{ "table": "main.public.film", "version": 1, "operation": "ADOPT ROWS" }],
  "migration_id": "20261012T165228-adopt_rows-adopt-5-notes-into-film"
}

The notes keep their text and names, so [[His Girl Friday]] in Charade still resolves:

read_note

{ "path": "datanotes/main/public/film/Charade.md" }

Answer:

---
table: main.public.film
id: 01a13107-abdf-770a-a3af-34b2b65e8d80
year: 1963
minutes: 113
status: watched
rating: 4
watched_on: 2026-09-12
director: Stanley Donen
title: Charade
---
Cary Grant again, twenty years after [[His Girl Friday]], and just as quick.

select_rows

{ "table": "film", "columns": ["title", "status", "rating"], "order_by": "title" }

Answer:

{
  "table": "main.public.film",
  "total": 5,
  "rows": [
    {
      "path": "datanotes/main/public/film/Charade.md",
      "title": "Charade",
      "status": "watched",
      "rating": 4
    },
    {
      "path": "datanotes/main/public/film/His Girl Friday.md",
      "title": "His Girl Friday",
      "status": "watched",
      "rating": 5
    },
    {
      "path": "datanotes/main/public/film/Metropolis.md",
      "title": "Metropolis",
      "status": "to watch",
      "rating": null
    },
    {
      "path": "datanotes/main/public/film/Nosferatu.md",
      "title": "Nosferatu",
      "status": "watched",
      "rating": 5
    },
    {
      "path": "datanotes/main/public/film/The Gold Rush.md",
      "title": "The Gold Rush",
      "status": "watched",
      "rating": 5
    }
  ]
}

Insert new rows through the engine

Section titled “Insert new rows through the engine”

title is the note’s file name (identity: "filename"). When you give it at insert, it names the new note, and the engine then keeps it equal to the note’s actual name.

insert_row

{
  "table": "film",
  "values": {
    "title": "Battleship Potemkin",
    "year": 1925,
    "minutes": 75,
    "director": "Sergei Eisenstein",
    "status": "to watch"
  }
}

Answer:

{
  "ok": true,
  "op": "insert_row",
  "files_created": 1,
  "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
  "commits": [{ "table": "main.public.film", "version": 2, "operation": "WRITE" }],
  "migration_id": "20261012T165228-insert_row-insert-into-film"
}

Set a value for notes made by hand with external_default

Section titled “Set a value for notes made by hand with external_default”

default fills a column on rows the engine writes. external_default fills it on notes that become rows without the engine writing them, like the note in the next example. Here added_by tells the two apart.

add_column

{
  "table": "film",
  "column": {
    "name": "added_by",
    "type": "enum",
    "values": ["agent", "by hand"],
    "default": "agent",
    "external_default": "by hand",
    "description": "Who added the film: an agent through the engine, or a person in an editor"
  }
}

Answer:

{
  "ok": true,
  "op": "add_column",
  "planned_texts": ["datanotes/metastore.yaml"],
  "commits": [{ "table": "main.public.film", "version": 4, "operation": "ADD COLUMNS" }],
  "migration_id": "20261012T165230-add_column-add-column-added_by-to-film"
}

insert_row

{
  "table": "film",
  "values": {
    "title": "Night of the Living Dead",
    "year": 1968,
    "minutes": 96,
    "director": "George A. Romero",
    "status": "to watch"
  }
}

Answer:

{
  "ok": true,
  "op": "insert_row",
  "files_created": 1,
  "planned_creates": ["datanotes/main/public/film/Night of the Living Dead.md"],
  "commits": [{ "table": "main.public.film", "version": 5, "operation": "WRITE" }],
  "migration_id": "20261012T165230-insert_row-insert-into-film"
}

datanotes/main/public/film/Night of the Living Dead.md now reads:

---
table: main.public.film
id: 01a13107-b1e6-7122-b565-804cd2fc0ad9
director: George A. Romero
minutes: 96
status: to watch
year: 1968
title: Night of the Living Dead
added_by: agent
---

A new note in a table’s folder becomes a row

Section titled “A new note in a table’s folder becomes a row”

The engine adopts a note you create by hand inside a table’s folder, for example with “new note” in a note editor such as Obsidian, or from a daily note template. The note gets:

  • table and an id
  • the values its name carries under the table’s filename pattern
  • the column defaults and the external_default values

Here the note’s name gives title, and added_by is by hand. A note that misses a required column or has a bad value stays as it is, and the engine logs a warning.

You edit datanotes/main/public/film/Sherlock Jr..md by hand, outside the engine. The note now reads:

---
year: 1924
minutes: 45
status: to watch
director: Buster Keaton
---

read_note

{ "path": "datanotes/main/public/film/Sherlock Jr..md" }

Answer:

---
table: main.public.film
id: 01a13107-b465-72da-a511-dbd2350ad5cb
year: 1924
minutes: 45
status: to watch
director: Buster Keaton
added_by: by hand
title: Sherlock Jr.
---

Keep a folder’s notes out of the table with exclude

Section titled “Keep a folder’s notes out of the table with exclude”

A note is a row when its front matter key table names the table, and a new note in the table’s folder becomes one. exclude lists folders whose notes never become rows. Here it lists the editor’s templates folder. The template already says table:, so the notes you make from it are rows, but the template itself must not be one.

alter_table

{ "table": "film", "exclude": ["datanotes/main/public/film/Templates"] }

Answer:

{
  "ok": true,
  "op": "alter_table",
  "planned_texts": ["datanotes/metastore.yaml"],
  "commits": [{ "table": "main.public.film", "version": 7, "operation": "ALTER TABLE" }],
  "migration_id": "20261012T165232-alter_table-alter-table-main-public-film"
}

You edit datanotes/main/public/film/Templates/New film.md by hand, outside the engine. The note now reads:

---
table: main.public.film
year:
status: to watch
---
## Why I want to watch it

The engine leaves the template as it is, with no id, and it isn’t a row:

read_note

{ "path": "datanotes/main/public/film/Templates/New film.md" }

Answer:

---
table: main.public.film
year:
status: to watch
---
## Why I want to watch it

select_rows

{
  "table": "film",
  "columns": ["title"],
  "where": { "path": { "contains": "Templates" } }
}

Answer:

{ "table": "main.public.film", "total": 0, "rows": [] }

See the table’s definition in metastore.yaml

Section titled “See the table’s definition in metastore.yaml”

This is the table as you inferred, corrected and extended it on this page. Its definition holds:

  • title with identity: filename
  • the columns from the notes’ front matter
  • added_by with its external_default
  • the exclude folder under properties

datanotes/metastore.yaml now reads:

# Metastore (Datanotes): catalogs, schemas, tables, columns, views.
id: 01a13107-abc2-7540-809d-762357ede111
config:
  history: true


catalogs:
  main:
    schemas:
      public:
        tables:
          film:
            description: Classic films I watched or want to watch
            options:
              filename: "{title}"
              archive: _archive
              exclude: datanotes/main/public/film/Templates
            columns:
              - name: id
                type: string
                unique: true
                identity: uuid
                required: true
                dense: true
                description: "Unique id of the row (UUIDv7): set when the row is created, never changes"
              - name: title
                type: string
                description: The film's title, which is also its note's name
                identity: filename
              - name: director
                type: string
                required: true
                description: Who directed the film
                dense: true
              - name: minutes
                type: integer
                required: true
                description: Running time in minutes
                dense: true
              - name: status
                type: enum
                required: true
                values:
                  - to watch
                  - watched
                description: Where the film is on my list
                ordered: true
                dense: true
              - name: year
                type: integer
                required: true
                description: Year the film first came out
                dense: true
              - name: rating
                type: integer
                min: 1
                max: 5
                description: My rating, from 1 (poor) to 5 (a masterpiece)
              - name: watched_on
                type: date
                description: The day I last watched it
              - name: added_by
                type: enum
                default: agent
                values:
                  - agent
                  - by hand
                external_default: by hand
                description: "Who added the film: an agent through the engine, or a person in an editor"