ADR 14: Declarative Kpimark validation

Context

Document structures of former XML formats allowed us to easily perform input document validation. There were 3 basic types of document structures: scenario, subject_info and page, which are described here.

In Kpimark, there are currently no standardized validation tools. We wanted to achieve some kind of validation for Kpimark files in order to “force” users to stick to good practices like consistency during writing process. A simple declarative DSL could potentionally be a great solution, which allows easy validation rules creation. Thanks to using mistletoe library as Makrdown parser, now we can operate on in-memory tree of tokens using custom Renderer.

Decision

We’ve decided to create two ways of validation:

1. Using simple DSL language to perform mostly appearing/ordering related checks.
2. Using custom Python functions to perform other validation logic.

For DSL functionality, we decided to define two basic concepts:

1. Actions, serves much like type checking of parent’s children tokens. You can define token’s ordering or whitelist/blacklist children in parent token and more.
2. Properties, which server more like content modifiers. You can set if token is required or can be found multiple times.

The main difference between actions and properties is that properties are meant to perform it’s check on parent itself. On the other hand actions are checking parent’s content (children).

For example, let’s take a look at scenario structure DSL schema:

from it4kt.kpimark.validation.validator import *

Shape(Document).contains_order([
    Shape(MetaBlock).required,
    Shape(ObjectivesBlock).contains_all_of(ObjectiveItem).required,
    IntroductionBlock,
    Shape(InstructionsBlock).contains_all_of(StepBlock).required,
    SummaryBlock,
    ResourcesBlock,
    AdditionalTasksBlock,
    AdditionalResourcesBlock,
])

We decided to Wrap tokens, that needs to be validated, in declarative schema inside Shape object. These arguments then acts like parents. Whenever any document token is found within tokens tree (for docuemnt of scenario type), schema checking is going to be recursively performed.

These schemas are called directly from special kind of renderer named Validator, where schema checking and custom validation logics lives coupled together.

We’ve decided to provide an optional meta key for Kpimark users validation, which can be set to “None” or “null” in order to turn off the validaiton for specifiied file. We’ve also added type meta key, which allows users to override type of a document, which is by default defined in course’s configuration for every specific file or whole folder.

We also decided to include Logger extensions in order to provide relevant output from validation for users. It should always point user to parsed file, where an exception was raised and provide specific feedback about validation error.

Status

Accepted.

Consequences

It is now possible to assign structure to every Kpimark document or token. It is also possible to perform validation on Kpimark document in two ways: custom validation logic and DSL schema. Users are able to override file’s type directly from metadata. Users are also able to disable validation for specific file using metadata.