aidbox.bulk data import

aidbox.bulk/* RPC operations

Overview

aidbox.bulk API uses Aidbox JSON RPC API

Features of the aidbox.bulk API:

• Asynchronous validation of uploaded resources & references

• Human readable validation errors

• Can process Aidbox & FHIR format

RPC methods

aidbox.bulk/import-start

Starts aidbox.bulk import.

Parameters

Object with the following structure:

• format input resources format, fhir or aidbox (default: fhir)

• meta meta data that will be attached to each of the imported resources

• on-conflict action to resolve id uniqueness constraint violation, update or override

• input array of objects

  • url string with input source url

Result

Returns input params object with following attributes added:

• status in-progress, finished, failed

• count overall number of imported resources

• time import execution time

• input the import status of each specific input

  • status loaded, validated, failed

  • count number of loaded resources

  • errors number of validation errors

  • time input execution time

Error

• message "There is running import - wait until it finish or cancel it with aidbox.bulk/import-cancel"

• import object identical to result object

Resource requirements for aidbox.bulk/import-start:

Operation	id	resourceType
aidbox.bulk/import-start	Required	Required

Example

Request

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-start
params:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz'}

Response

status: 200

result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz', status: loaded, count: 124, errors: 1, time: 157}
  status: failed
  errors: 1
  time: 241

aidbox.bulk/import-status

Returns latest aidbox.bulk import info

Parameters

Expects no parameters

Result

Same as aidbox.bulk/import-start result

Error

message "No active imports"

Example

Request

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-status
params: {}

Response

Status: 200

result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta:
    source: app1
  input:
    - url: >-
        https://storage.googleapis.com/aidbox-public/synthea/100/Patient.ndjson.gz
      status: loaded
      count: 124
      errors: 0
      time: 1205
  status: finished
  count: 0
  time: 1452

aidbox.bulk/import-errors

Returns latest aidbox.bulk import detailed errors list

Parameters

Object with the following structure:

• omit-resources? (default: false)

Result

Result is an array of objects with following structure:

• input_no

• line_no

• type

• id

• resource

• errors

Error

message "No active imports"

Example

Request

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-errors
params:
  omit-resources?: true

Response

Status: 200

result:
  - input_no: 0
    line_no: 1
    type: Patient
    id: e9adac47-eb98-4fce-b871-512226086c97
    errors:
      - path:
          - name
          - 0
          - given
        message: expected array
  - input_no: 0
    line_no: 20
    type: Patient
    id: b2d58f0f-4499-4392-ad3a-1c2141c8a6c1
    errors:
      - path:
          - address
          - 0
          - line
        message: expected array

aidbox.bulk/import-cancel

Cancels latest aidbox.bulk import

Parameters

Expects no parameters

Result

Returns object with following structure:

• message "Import canceled"

• import Same as aidbox.bulk/import-start result

Error

message "No active imports"

Example

Request

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-cancel

Response

Status: 200

result:
  message: "Import Canceled"
  import: <. . .>

Import local file

Sometimes you want to import local file into local Aidbox. Possible solutions for local development:

Add volume to the aidboxone container (not aidboxdb):

volumes:
- ./Encounter.ndjson.gz:/resources/Encounter.ndjson.gz
# url: file:///resources/Encounter.ndjson.gz

Use tunneling e.g. ngrok:

python3 -m http.server 
ngrok http 8000
# url: https://<...>.ngrok-free.app/Encounter.ndjson.gz