Asynchronous resource validation

Asynchronous Batch Validation draft
This is a draft API. We appreciate your feedback and ideas in this discussion​
It may happen that you updated your profiles when data is already in your database or you want to do efficiently load a batch of data and validate it later.  API consists of 3 procedures and a couple of resources:
​aidbox.validation/batch-validation - run validation
​aidbox.validation/batch-validation-result - inspect results (useful for async mode)
​aidbox.validation/clear-batch-validation - clear validation results
Prepare data
To illustrate let's create some invalid data in Aidbox:
1
POST /Patient
2
content-type: text/yaml
3
​
4
id: 'pt1'
5
birthDate: '1980-03-05'
Copied!
Break data from DB Console:
1
update patient 
2
set resource = resource || '{"ups": "extra"}'
3
where id = 'pt1' 
4
returning *
Copied!
aidbox.validation/batch-validation
You can validate your existing data with our new rpc aidbox.validation/batch-validation:
1
POST /rpc
2
content-type: text/yaml
3
​
4
method: aidbox.validation/batch-validation
5
params:
6
  # resourceType to validate
7
  resource: Patient
8
  id: pt-validation-run-1
9
  # you can limit number of resources to validate
10
  limit: 100 
11
  # you can stop process on specific number of invalid resources
12
  errorsThreshold: 10 
13
  # where section of resources query
14
  filter: "resource#>>'{birthDate}' is not null"
15
  ## run validation asynchronously
16
  # async: true
17
  ## specify profiles to validate
18
  # profiles: ['profile-url-1', 'profile-url-2']
19
  ## specify zen schemas to validation
20
  #schemas: ['myapp/Patient', 'us-core.patient/Patient']
21
  
22
  
23
  
24
# response
25
result:
26
  id: pt-validation-run-2
27
  valid: 0
28
  invalid: 1
29
  duration: 15
30
  problems:
31
    - resource:
32
        id: pt1
33
        ups: extra
34
        meta:
35
          createdAt: '2021-08-05T16:36:37.723008+03:00'
36
          versionId: '1224'
37
          lastUpdated: '2021-08-05T16:36:37.723008+03:00'
38
        birthDate: '1980-03-05'
39
        resourceType: Patient
40
      errors:
41
        - path:
42
            - ups
43
          message: extra property
Copied!
aidbox.validation/batch-validation-result
If you run validation in async mode, it will respond instantly and run validation in the background. You can get validation results with  RPC aidbox.validation/batch-validation-result
1
POST /rpc?_format=yaml
2
content-type: text/yaml
3
​
4
method: aidbox.validation/batch-validation-result
5
params:
6
  id: pt-validation-run-1
7
   
8
# response
9
status: 200
10
result:
11
  valid: 1543
12
  invalid: 2
13
  duration: 3293
14
  problems:
15
    - resource: {....}
16
      errors: [{...}, {...}]
Copied!
aidbox.validation/clear-batch-validation
When you do not need results of this validation you can clean up resources with:
1
POST /rpc?_format=yaml
2
content-type: text/yaml
3
​
4
method: aidbox.validation/clear-batch-validation
5
params:
6
  id: pt-validation-run-1
Copied!
BatchValidationRun & BatchValidationError Resources
When you run validation operation aidbox internally creates resource BatchValidationRun and put errors of validation in BatchValidationError. You can access these resources through standard CRUD/Search API
1
GET /BatchValidationError?.run.id=pt-validation-run-2&_format=yaml&_result=array
2
​
3
# response
4
- run:
5
    id: pt-validation-run-2
6
    resourceType: BatchValidationRun
7
  errors:
8
    - path:
9
        - ups
10
      message: extra property
11
  resource:
12
    id: pt1
13
    resourceType: Patient
14
  id: pt-validation-run-2-Patient-pt1
Copied!
If you restart Aidbox you have to start validation over

Extend an IG with a custom zen profile

Next - Profiling and validation

Profiling with AidboxProfile

Last modified 4mo ago

Copy link

Contents

Asynchronous Batch Validation draft