Links

Asynchronous resource validation

Use RPC to run a validation operation to check a resource conformance

Asynchronous Batch Validation draft

This is a draft API. We appreciate your feedback and ideas in this discussion
It may happen that you updated your profiles when data is already in your database or you want to do efficiently load a batch of data and validate it later. API consists of 4 procedures and a couple of resources:

Prepare data

To illustrate let's create some invalid data in Aidbox:
POST /Patient
content-type: text/yaml
id: 'pt1'
birthDate: '1980-03-05'
Break data from DB Console:
update patient
set resource = resource || '{"ups": "extra"}'
where id = 'pt1'
returning *

aidbox.validation/batch-validation

You can validate your existing data with our new rpc aidbox.validation/batch-validation:
POST /rpc
content-type: text/yaml
method: aidbox.validation/batch-validation
params:
# resourceType to validate
resource: Patient
id: pt-validation-run-1
# you can limit number of resources to validate
limit: 100
# you can stop process on specific number of invalid resources
errorsThreshold: 10
# where section of resources query
filter: "resource#>>'{birthDate}' is not null"
## run validation asynchronously
# async: true
## specify profiles to validate
# profiles: ['profile-url-1', 'profile-url-2']
## specify zen schemas to validation
#schemas: ['myapp/Patient', 'us-core.patient/Patient']
# response
result:
id: pt-validation-run-2
valid: 0
invalid: 1
duration: 15
problems:
- resource:
id: pt1
ups: extra
meta:
createdAt: '2021-08-05T16:36:37.723008+03:00'
versionId: '1224'
lastUpdated: '2021-08-05T16:36:37.723008+03:00'
birthDate: '1980-03-05'
resourceType: Patient
errors:
- path:
- ups
message: extra property

aidbox.validation/resources-batch-validation-task

You can run validation workflow with rpc method, which creates task for every resource provided in rpc's params fields include or exclude:
POST /rpc
accept: text/yaml
content-type: text/yaml
method: aidbox.validation/resources-batch-validation-task
params:
include: ['patient', 'observation']
# response
params:
tables:
- patient
- observation
status: in-progress
definition: aidbox.validation/resource-types-batch-validation-workflow
id: >-
7addda33-003e-4892-a1d9-0faffbedf86d
resourceType: AidboxWorkflow
You can also run workflow directly to achieve the same result.
If you specify include param, only types you passed will be validated.
If you specify exclude param, all types will be validated except the ones you passed.
include and exclude params cannot be used together.
You can check a progress of workflow in Aidbox UI or by rpc method:
POST /rpc
accept: text/yaml
content-type: text/yaml
method: awf.workflow/status
params:
id: 7addda33-003e-4892-a1d9-0faffbedf86d
#response
result:
resource:
params:
tables:
- patient
- observation
result: Finished
status: done
outcome: succeeded
definition: aidbox.validation/resource-types-batch-validation-workflow
id: >-
7addda33-003e-4892-a1d9-0faffbedf86d
resourceType: AidboxWorkflow

aidbox.validation/batch-validation-result

If you run validation in async mode or aidbox.validation/resources-batch-validation-task, it will respond instantly and run validation in the background. You can get validation results with RPC aidbox.validation/batch-validation-result
POST /rpc?_format=yaml
content-type: text/yaml
method: aidbox.validation/batch-validation-result
params:
id: pt-validation-run-1
# response
status: 200
result:
valid: 1543
invalid: 2
duration: 3293
problems:
- resource: {....}
errors: [{...}, {...}]
aidbox.validation/batch-validation-result method requires `resourceType` param, which has a default value `BatchValidationRun`.
So, if you want to get the result from aidbox.validation/resources-batch-validation-task you need pass "AidboxWorkflow" to `resourceType` param.

aidbox.validation/clear-batch-validation

When you do not need results of this validation you can clean up resources with:
POST /rpc?_format=yaml
content-type: text/yaml
method: aidbox.validation/clear-batch-validation
params:
id: pt-validation-run-1

BatchValidationRun & BatchValidationError Resources

When you run validation operation aidbox internally creates resource BatchValidationRun and put errors of validation in BatchValidationError. You can access these resources through standard CRUD/Search API
GET /BatchValidationError?.run.id=pt-validation-run-2&_format=yaml&_result=array
# response
- run:
id: pt-validation-run-2
resourceType: BatchValidationRun
errors:
- path:
- ups
message: extra property
resource:
id: pt1
resourceType: Patient
id: pt-validation-run-2-Patient-pt1
If you restart Aidbox you have to start validation over
Last modified 2mo ago