Version 4.0.0 · All versions

All pages

These are the docs as released with Solid Memo 4.0.0. Read the latest docs.

Deck library

The deck library is the set of ready-made decks and courses anyone can copy into their own instance. They are files in this repository's decks/ folder, published with the site and described in DCAT, a standard catalog format other tools can read. This page says how a deck version is written, checked and published, and how the app lists, imports and adds one from a link.

Ready-made decks that any user can copy into their own instance. The library lives in this repository's decks/ folder and is published with the site, next to the app (deployment.md), at https://solid-memo.com/decks/. It is described with DCAT and conforms to DCAT-AP: the library is a dcat:Catalog, each deck a dcat:DatasetSeries of its versions, each version a dcat:Dataset with its cards and a Turtle dcat:Distribution.

Path under decks/IRIWhat
index.ttlhttps://solid-memo.com/decks/index.ttlThe catalog the app lists, generated from the versions and committed: each deck a series (index.ttl#<name>), its current version described in full but for its cards, how it was made and any email, the publisher index.ttl#solid-memo, and the course for newcomers.
<name>/v<N>.ttlhttps://solid-memo.com/decks/<name>/v<N>.ttlVersion N of a deck, frozen once published; its cards are fragments of it (…/v1.ttl#sweden).

Nothing else is in decks/: no lock file, no sources, no scripts. A deck's name is lower-case letters, digits and dashes; its versions run v1.ttl, v2.ttl, … without gaps. Every IRI ends in .ttl, as the vocabulary's and the shapes' do: GitHub Pages serves a .ttl file as text/turtle but negotiates no content, so an address without the extension would be a 404. Each file states its own address as its @base.

Publishing a deck version
A new version is copied from the frozen one, indexed with npm run library, checked with npm run library:check and deployed with the site.
The diagram's source
flowchart LR
    accTitle: Publishing a deck version
    accDescr: A new version is copied from the frozen one, indexed with npm run library, checked with npm run library:check and deployed with the site.
    prev["decks/name/vN.ttl<br/>(published, frozen)"] -->|copy, edit| next["decks/name/vN+1.ttl"]
    next -->|npm run library| idx["decks/index.ttl<br/>dcat:Catalog"]
    next & idx -->|"npm run library:check<br/>-- --base ref"| ok["valid, index up to date,<br/>nothing published changed"]
    ok -->|deploy| site["solid-memo.com/decks/…"]

A version

A version is one sm:Deck (also a dcat:Dataset), the document itself (<>), in library deck format 5 (LibraryDeckV5, shapes.md) or, from vocabulary 1.18, format 6 (LibraryDeckV6, the same properties), with its cards as hash-fragment subjects in card format 4, 5 or 6. A version stays at the format it was published in; a new one is written at format 6, its series and publisher still in the index:

decks/greek-alphabet/v1.ttl is a small, complete example.

Courses

A version may be a course: a deck whose cards are taught in chapters of steps, each card asked as a multiple-choice question. Beside what every version has, a course has:

The index leaves the chapters, steps and distractors out, as it leaves the cards out. It keeps the release's schema:Course type, so the app can tell a course from a deck by the index alone (LibraryDeck.isCourse). The app reads the outline from the release when the learner opens the course. decks/solid-fundamentals/v1.ttl is the first course, and the first release written in Markdown.

A course's chapters, steps and distractors are never removed, as its cards are not: one that should go is retired (owl:deprecated true), so the learners' decks that follow the course keep their place in it.

The index names at most one course for newcomers: <> sm:newcomerCourse <#name> on the catalog, the series of a course the app offers to someone with no decks yet (courses.md). It is library configuration, not part of any release: NEWCOMER_COURSE in deckLibrary.ts names the deck (io.newcomerCourse in main, so its tests can name another), and undefined names none (and retires the journey that expects an offer, courses.md). The term belongs to no shape: CatalogV1 does not own it, and DCAT-AP ignores it.

Contact

A version names the library's contact, the project's issue tracker, as its dcat:contactPoint: a vcard:Kind node of the document, as DCAT-AP 3.0.1 describes one. It names no one's email: no subject states foaf:mbox.

@prefix vcard: <http://www.w3.org/2006/vcard/ns#> .

<> dcat:contactPoint <#contact> .

<#contact>
    a vcard:Kind ;
    vcard:hasURL <https://github.com/solid-memo/website/issues> .

The versions published before this rule name the maintainer's email instead (foaf:mbox on <#anton-wiklund>). They are frozen, so they keep it; a deck's next version drops it. The index never repeats an email, whichever version is current: keptInIndex (seriesEntry.ts) leaves out every foaf:mbox, and keeps the current version's contact. The command checks the rule on every version new since --base (contactProblems, Checks); LIBRARY_CONTACT in curationRules.ts names the tracker. The Studio's release check for the library asks the same of a draft (repoPolicy, studio.md), and its release screen writes the contact. A release in a pod (studio.md) is the author's own and may name an email. The app reads the contact's web page to say where a mistake in a release from a link is reported (Reporting a mistake).

Provenance

Each authored deck says how it was made, in PROV-O, inside its version file:

The record is condensed from the working notes kept while a deck was compiled; those notes' per-card evidence and queries are not part of the library. The few decks that were not researched card by card (generated from one source, such as the Swedish word decks and network ports, or written directly, such as the capitals) describe their sources and licenses, some the command that generated them, but have no review rounds. The index leaves the record out (every rdfs:comment, and the activities but for the compilation's type, which DCAT-AP asks for); the app shows the deck's sources and license on its page, and the version file has the rest.

Publishing a new version

A published version is never edited or removed, only followed by the next. To change a deck <name> whose latest version is N:

  1. Copy decks/<name>/v<N>.ttl to decks/<name>/v<N+1>.ttl and edit the copy: cards (retire, never delete), texts, sources, provenance.
  2. In the copy, change its @base to its own address (…/<name>/v<N+1>.ttl) and set
    • dcat:version "<N+1>",
    • dcat:prev and dcat:previousVersion <https://solid-memo.com/decks/<name>/v<N>.ttl>,
    • dcterms:issued to the release time, and dcterms:modified if the deck has it,
    • adms:versionNotes to what changed, in one string,
    • the contact: dcat:contactPoint <#contact> and its vcard:Kind node, and no foaf:mbox (an agent copied from a version published before the rule has one: remove it).
  3. npm run format:turtle, then npm run library, which validates the library and writes decks/index.ttl (the new version becomes the series' dcat:last and current version). Then npm run library:check -- --base origin/main, which checks the contact of every version new since origin/main.
  4. Look at it: npm run dev serves the working tree's decks/, so the library screen lists the new version and offers it to copies of the old one (migrations.md).
  5. Commit the new version and the index together, as a feat(decks) commit (CONTRIBUTING.md). The release it makes publishes them, once deployed (deployment.md).

A new deck is the same with decks/<name>/v1.ttl, without dcat:prev and dcat:previousVersion, its notes "First release.". It is also listed in REUSE.toml under the license its dcterms:license names, as "decks/<name>/*.ttl", and a version that changes the deck's license moves it there to the license's own annotation, or names that version alone. npm run check:reuse fails until it agrees (style.md).

Text over several lines. npm run format:turtle writes a literal whose text has a line feed as a long string, """…""" with its language tag, its lines verbatim and never indented (leading spaces can matter). A backslash and every control character but a tab or a line feed are escaped (a carriage return as \r), and so is any quote that would end or run into a delimiter, so the text reads back the same. Every other literal stays on one line.

The taxonomy deck

The deck of new and changed Swedish occupation names, swedish-labour-market-taxonomy-occupation-names, is generated, not written: one version per version of the labor market taxonomy of Arbetsförmedlingen, the Swedish Public Employment Service, which its JobTech taxonomy API serves under CC0 1.0, as the deck is. scripts/taxonomy_deck.py makes it, with Python's standard library alone. Its docstring says which names become cards, and why, and how it handles the taxonomy's inconsistencies.

The script reads the deck's newest version, v<N>.ttl, and writes v<N+1>.ttl beside it, covering every taxonomy version up to the newest (or --to). From v<N>.ttl it carries over, verbatim, what it cannot make itself: the title, description and keywords in every language of the app, the creator's name (never an email), and the review rounds that made those texts, by which the new version's <#generation> is prov:wasInformedBy. It keeps every card of v<N>.ttl, and retires one that no rule makes any longer. It writes the rest of what a new version needs: dcat:version, dcat:prev and dcat:previousVersion, dcterms:issued (--issued, by default today in UTC), the version notes, the publisher, the series, the contact and the Turtle distribution.

A version is reproducible. Its provenance records the script at the commit it ran from, with its SHA-256 and the Python and Unicode versions, the command line, and every taxonomy version read, with the query that fetched it and a checksum of the answer. Running that command at that commit makes the version again, byte for byte. So the script refuses to run unless it is committed and unmodified (--allow-uncommitted writes a draft), and stops when a taxonomy version the deck records no longer matches its checksum.

To make a version by hand, from the repository root on an up-to-date main:

python3 scripts/taxonomy_deck.py --verify --cache-dir ~/.cache/taxonomy
python3 scripts/taxonomy_deck.py --cache-dir ~/.cache/taxonomy
npm run format:turtle
npm run library
npm run library:check -- --base origin/main

The script prints what it included, left out and adjusted, then the version notes. Commit the new version with the index as a feat(decks) commit that names its issue (CONTRIBUTING.md), as for any new version.

The Taxonomy workflow does this every Monday, and by hand (Actions › Taxonomy › Run workflow). Its check job runs --check. When the taxonomy has a version the deck lacks, its update job runs --verify, makes the next version, and checks it as CI's library and crosscheck jobs do. Only then does it open the issue "Taxonomy version N is out" (label deck), unless one is open already, and push a commit to the branch taxonomy-deck that closes it, and the issues of earlier taxonomy versions still open, with Closes: lines. It opens a pull request from that branch, or updates the one already open. A pull request already open for the newest taxonomy version is left alone: merge it, or close it to have it made again. Nothing merges it on its own. Once it is merged through the merge queue (testing.md), the next release publishes the version, as it does any other. When a job fails, the failed job opens the issue "The weekly taxonomy deck update failed", or comments on the one open, with a link to the run.

The taxonomy app

The update job pushes and opens the pull request with the token of a GitHub App, not the workflow's own. A pull request opened with the workflow's token starts no workflow, so CI would never run on it, ci-ok would never report, and it could not be merged. The App's token starts workflows as anyone's does. Only this workflow can use it: the App's private key is a secret of the labour-market-deck environment, which allows only main, as the deploy key's environment does.

A maintainer sets it up once:

  1. Make a GitHub App owned by the solid-memo organization (Organization settings › Developer settings › GitHub Apps › New GitHub App), named solid-memo-labour-market-deck, with no webhook ("Active" unchecked), and the repository permissions Contents, Issues and Pull requests, each "Read and write" (Metadata "Read" comes with them), and no other.
  2. Install it on solid-memo/website alone ("Only select repositories").
  3. Generate a private key in the App's settings.
  4. Make the environment labour-market-deck (Settings › Environments › New environment), or open it if a run has made it already: GitHub makes it, with no rule, when a job first names it. Under "Deployment branches and tags", choose "Selected branches and tags" and add the branch main, and nothing else. Add the App's client ID as its environment variable LABOUR_MARKET_DECK_CLIENT_ID, and the private key as its environment secret LABOUR_MARKET_DECK_PRIVATE_KEY. Then delete the key's file, and leave no repository variable or secret of either name: a job whose environment lacks one reads the repository's.

The job makes the token only after the script and the checks have run, for the steps that open the issue, push and open the pull request, and the token lasts an hour. The App needs no bypass of the ruleset on main: it pushes only the branch taxonomy-deck. Until the App is set up, the update job fails at the step that makes the token, and the failure issue links that run. To change the key, generate a new one in the App's settings, replace the environment's secret, then delete the old key there.

Authoring Markdown

A card, a course step or a chapter may be written in Markdown (markdown.md): CommonMark with pipe tables, for code, tables and lists. Text is plain unless its subject says otherwise, so a release that says nothing reads exactly as before.

<#q-staged> a solid-memo:Card ;
    solid-memo:formatVersion 5 ;
    solid-memo:textFormat solid-memo:markdown ;
    solid-memo:front "Which Git command shows **staged** changes?"@en ;
    solid-memo:back "`git diff --staged`"@zxx ;
    solid-memo:backNote """Without `--staged`, `git diff` compares the working tree with the index:

| Command | Compares |
|---|---|
| `git diff` | working tree and index |
| `git diff --staged` | index and `HEAD` |"""@en .

npm run library checks what is written in Markdown by the Markdown rules.

Checks

packages/shacl/node/deckLibrary.ts is both commands. npm run library writes the index; npm run library:check writes nothing and fails when the index is not what the versions make. Both check:

Where the rules live

The rules that read a release's data live in the domain, in packages/domain/src/release/, so the Studio can run them in the browser as the command does:

FileWhat
releaseModel.tsA release as plain data: its metadata, every card, chapter, step and distractor (retired ones too), how it was made.
problems.tsReleaseProblem: a code, a severity, the subject and field it is in, and what it needs to be worded. No text.
libraryRules.tsThe layout and each version's metadata against its place.
continuityRules.tsA version against the one before it.
courseRules.tsThe course rules; and what a release needs that its draft may lack (readinessProblems), which the Studio's check adds.
markdownFields.tsWhich fields are Markdown, and by which rule; the check itself is passed in, since the domain reads no Markdown.
curationRules.tsA LibraryPolicy: what a library asks beyond the data. repoPolicy asks a title, description and keywords in every language the app speaks (LIBRARY_LANGUAGES, from LOCALES) and the theme EDUC; podPolicy asks nothing.
seriesEntry.tsWhat the index says of a deck.
releaseCheck.tsThe Studio's release check: the rules a policy runs, where a library publishes a release (libraryReleaseUrl, librarySeriesUrl, which this command uses too), and where each problem is fixed.
draftModel.tsA Studio draft as a release model, at its own address or another.

The command builds each release's model from its quads (quadsToReleaseModel.ts), words each problem in English (releaseMessages.ts), and checks itself what only the text or the shapes can: @base, Turtle syntax, the shapes and the profiles. It does not run the curation rules: LibraryDeckV5 and LibraryDeckV6 state English and EDUC for this library. Nor does it name a version that is not the one before it plus one (versionNotNext): here the path fixes each version, and the metadata check names one other than its path says. A golden test (deckLibrary.golden.test.ts) keeps what the rules say of published releases with faults put in.

Course rules

courseProblems (courseRules.ts) checks what the shapes cannot say about a course. It runs on every version, and finds nothing in a plain deck:

Retired steps and chapters do not count toward the rules on positions or on how a card is asked.

Markdown rules

markdownProblems (markdownFields.ts, with the markdown package's check) checks, field by field, the text of every card, step and chapter that states sm:textFormat sm:markdown, by the rules for a release. In short:

It also checks that only a card, a step or a chapter states sm:textFormat (a distractor's text is written as its card's, and any other subject's is plain), and only a concept of sm:TextFormats. A text not marked is not checked: plain text is shown as written.

None of this is what keeps the app safe, which shows any text safely (markdown.md): it is that the text shows as its author meant.

With -- --base <git ref> they also compare with the library at that commit: a version published there that is gone or differs by a byte fails, and so does a version not published there that states foaf:mbox or lacks the library's contact (contactProblems). Without a base nothing tells a new version from a published one, so that rule does not run. npm run check runs library:check without a base, as a Turbo task of the shacl package (turbo.json), cached until a file it reads (ns/, decks/, its packages' code) changes. CI's build job runs npm run check:ci, which leaves it out, since the library job checks the same and more. CI's library job (ci.yml, testing.md) passes a base, on pull requests, in the merge queue and on pushes to the default branch, not on other branches' pushes, so a pull request's commits are checked once per push; ci-ok waits for it like every other job. On a push to the default branch it passes the commit the push moved it from; in the merge queue, the commit of the default branch the queue built on (testing.md); on a pull request, the commit where the change left the default branch, so an edit to a published version is caught before it is merged, while a version not merged yet can still be fixed. A branch with no pull request is not compared; open one, or check it locally before pushing:

npm run library:check -- --base origin/main

pySHACL checks the index and every version against DCAT-AP once more (validation.md), and libraryReleases.test.ts reads every version as the app does, its cards and a course's outline (testing.md).

Reading it

It is public, so the app reads it with a plain fetch, without logging in. appUseCases.ts names the index; during npm run dev and npm start the site's own addresses are read from the local server (siteFetch), so the working tree's library is what the app shows. VITE_LIBRARY_INDEX_URL names another index at build time (configuration.md):

VITE_LIBRARY_INDEX_URL=https://example.org/decks/index.ttl npm run dev

The library conforms to the shapes in shapes.md: CatalogV1, LibraryDeckSeriesV3, and LibraryDeckV5 or LibraryDeckV6, each release against its own format's shape. The app migrates a format-5 release to format 6 in memory.

A release outside the library

A release need not be in decks/: from library deck format 6, one Turtle document anywhere (a release published in a pod) is whole without an index. Beside the release and its cards, the document then describes:

The same shapes check it (LibraryDeckV6, LibraryDeckSeriesV3, AgentV1), and DCAT-AP accepts it with only the reference data beside it (library-deck/v6/valid is an example, which npm run crosscheck checks too). Nothing in the data says who may publish one, and the app reads its content as any release's (toLibraryDeckContent).

Such a release is made from a draft in the creator's pod (studio.md): every subject of the draft moved to the release's address, @base that address, its release and change time set, at library deck format 6. A series the draft starts (its own <#series>) is described in it, the release its first, last and current version; a series an earlier release described goes on being described, the release added as its last and current version; a series an index describes, as this library's, is left to the index. The next version of such a release describes it, though: the draft copies from the index its series, its publisher and its earlier versions (readLinked, linkedDescription), so the release it makes is whole too. So a release downloaded for decks/ from a next version carries the index's statements of its series, publisher and earlier versions. Only what a release needs is copied: of the series, what an index says of one; of a publisher, its class and name. Every release in decks/ made a draft and published again comes back as it was, apart from what publishing sets (a test of its own, testing.md).

In the app

The library in the app
The library screens and the newcomer offer call use cases that read the library with plain fetches and write imported decks to the pod with authenticated requests.
The diagram's source
flowchart LR
    accTitle: The library in the app
    accDescr: The library screens and the newcomer offer call use cases that read the library with plain fetches and write imported decks to the pod with authenticated requests.
    ui["LibraryContainer / LibraryScreen<br/>#/library?instance=…"] --> uc["listLibraryDecks<br/>importLibraryDeck"]
    page["LibraryDeckContainer / LibraryDeckScreen<br/>#/library-deck?instance=…&deck=&lt;series&gt;"] --> uc
    offer["NewcomerCourseContainer<br/>on an empty deck list"] --> start["listLibraryDecks<br/>startCourse"]
    start --> lib
    start --> repo
    uc --> lib["DeckLibrary port<br/>(solidDeckLibrary.ts)"]
    uc --> repo["DeckRepository.importDeck<br/>(solidDeckRepository.ts)"]
    lib -->|plain fetch| idx["solid-memo.com/decks/index.ttl"]
    lib -->|plain fetch| doc["solid-memo.com/decks/name/vN.ttl"]
    repo -->|authenticated| pod["catalog.ttl + decks/deck-id.ttl"]

A deck or a course published anywhere, a release outside the library in its creator's pod say, is added by its address: Add from a link, on the deck list and the library, opens #/import-url (routing.md). Nothing about it is special to one creator, host or series: any release this app can read is added the same way.

Importing a release by URL
The import screen reads a release from any host, validates it against the shapes, and imports it into the learner's instance.
The diagram's source
flowchart LR
    accTitle: Importing a release by URL
    accDescr: The import screen reads a release from any host, validates it against the shapes, and imports it into the learner's instance.
    ui["ImportUrlContainer / ImportUrlScreen<br/>#/import-url?instance=…&url=…"] --> read["readReleaseFromLink"]
    read --> lib["DeckLibrary.readRelease<br/>(solidDeckLibrary.ts)"]
    read --> shapes["ShapeValidator.validateDocument<br/>(context library)"]
    ui --> add["importReleaseFromUrl"]
    add --> imp["importLibraryDeck / startCourse"]
    lib -->|plain fetch| rel["any host: …/name/vN.ttl"]
    shapes -->|plain fetch| rel
    imp -->|authenticated| pod["the learner's instance"]

Reporting a mistake

A learner who sees a wrong card reports it from where they see it: Report a mistake is under a library card's page, a library deck's page, a release's page from a link, and each card studied from a copy that its release has (ReportMistake in ReportMistake.tsx). A card the learner added to their copy is theirs to fix, and has no link. The review of a version checks its form, not its facts (CONTRIBUTING.md), so these reports are how the library's mistakes are found. Where the link leads is mistakeReportUrl (mistakeReport.ts), from the release, the card's fragment and where the release is reported (ReportPlace):