Asynchronous resource validation

Use RPC to run a validation operation to check a resource conformance

Asynchronous Batch Validation draft

It may happen that you updated your profiles when data is already in your database or you want to do efficiently load a batch of data and validate it later. API consists of 4 procedures and a couple of resources:

Prepare data

To illustrate let's create some invalid data in Aidbox:

POST /Patient
content-type: text/yaml

id: 'pt1'
birthDate: '1980-03-05'

Break data from DB Console:

update patient 
set resource = resource || '{"ups": "extra"}'
where id = 'pt1' 
returning *

aidbox.validation/batch-validation

You can validate your existing data with our new rpc aidbox.validation/batch-validation:

POST /rpc
content-type: text/yaml

method: aidbox.validation/batch-validation
params:
  # resourceType to validate
  resource: Patient
  id: pt-validation-run-1
  # you can limit number of resources to validate
  limit: 100 
  # you can stop process on specific number of invalid resources
  errorsThreshold: 10 
  # where section of resources query
  filter: "resource#>>'{birthDate}' is not null"
  ## run validation asynchronously
  # async: true
  ## specify profiles to validate
  # profiles: ['profile-url-1', 'profile-url-2']
  ## specify zen schemas to validation
  #schemas: ['myapp/Patient', 'us-core.patient/Patient']
  
  
  
# response
result:
  id: pt-validation-run-2
  valid: 0
  invalid: 1
  duration: 15
  problems:
    - resource:
        id: pt1
        ups: extra
        meta:
          createdAt: '2021-08-05T16:36:37.723008+03:00'
          versionId: '1224'
          lastUpdated: '2021-08-05T16:36:37.723008+03:00'
        birthDate: '1980-03-05'
        resourceType: Patient
      errors:
        - path:
            - ups
          message: extra property

aidbox.validation/resources-batch-validation-task

You can run validation workflow with rpc method, which creates task for every resource provided in rpc's params fields include or exclude:

POST /rpc
accept: text/yaml
content-type: text/yaml


method: aidbox.validation/resources-batch-validation-task
params:
  include: ['patient', 'observation']
  
  
# response
params:
  tables:
    - patient
    - observation
status: in-progress
definition: aidbox.validation/resource-types-batch-validation-workflow
id: >-
  7addda33-003e-4892-a1d9-0faffbedf86d
resourceType: AidboxWorkflow

You can also run workflow directly to achieve the same result.

You can check a progress of workflow in Aidbox UI or by rpc method:

POST /rpc
accept: text/yaml
content-type: text/yaml


method: awf.workflow/status
params:
  id: 7addda33-003e-4892-a1d9-0faffbedf86d

#response
result:
  resource:
    params:
      tables:
        - patient
        - observation
    result: Finished
    status: done
    outcome: succeeded
    definition: aidbox.validation/resource-types-batch-validation-workflow
    id: >-
      7addda33-003e-4892-a1d9-0faffbedf86d
    resourceType: AidboxWorkflow

aidbox.validation/batch-validation-result

If you run validation in async mode or aidbox.validation/resources-batch-validation-task, it will respond instantly and run validation in the background. You can get validation results with RPC aidbox.validation/batch-validation-result

POST /rpc?_format=yaml
content-type: text/yaml

method: aidbox.validation/batch-validation-result
params:
  id: pt-validation-run-1
   
# response
status: 200
result:
  valid: 1543
  invalid: 2
  duration: 3293
  problems:
    - resource: {....}
      errors: [{...}, {...}]

aidbox.validation/clear-batch-validation

When you do not need results of this validation you can clean up resources with:

POST /rpc?_format=yaml
content-type: text/yaml

method: aidbox.validation/clear-batch-validation
params:
  id: pt-validation-run-1

BatchValidationRun & BatchValidationError Resources

When you run validation operation aidbox internally creates resource BatchValidationRun and put errors of validation in BatchValidationError. You can access these resources through standard CRUD/Search API

GET /BatchValidationError?.run.id=pt-validation-run-2&_format=yaml&_result=array

# response
- run:
    id: pt-validation-run-2
    resourceType: BatchValidationRun
  errors:
    - path:
        - ups
      message: extra property
  resource:
    id: pt1
    resourceType: Patient
  id: pt-validation-run-2-Patient-pt1

Last updated

Was this helpful?