Run Aidbox locallyRun Aidbox in SandboxTalk to us Ask community

Import flat file (CSV)

Import flat file via REST

If you have a terminology distributed as a flat file, for example CSV, you can use this API to import it as a set of Concept resources and later use them with Terminology API.

Concepts should have unique code property. In this regard Aidbox follows FHIR constraints defined for CodeSystems. If there are multiple Concepts with the same code, only the last one will be loaded.

Import by URL

Aidbox provides RPC API to load terminology from a flat file by url.

RPC aidbox.terminology.import-flat/import-from-url

This method accepts an URL pointing to a flat file with a terminology, file format description, columns to concept mapping, CodeSystem resource and ValueSet resource. For hierarchical terminology it can build hierarchy materialized paths.

When upload is done, rows of the flat file are loaded as Concept resources into Aidbox. Also CodeSystem and ValueSet resources are created, these resources describe the full terminology from the file, i.e. the ValueSet composed of only this CodeSystem and all Concepts have system set to the CodeSystem.url.

Also check "Request with comments" and "Parameters" tabs for the full request structure description.

POST /rpc
content-type: application/edn
accept: application/edn

{:method aidbox.terminology.import-flat/import-from-url
 :params
 {:source-url "<https://example.com/path/to/your/file.csv>"

  :format "csv"
  :csv-format {:delimiter "<char>"
               :quote "<char>"}

  :header     <true|false>
  :header-row <integer>
  :data-row   <integer>
  :hierarchy  <true|false>

  ;; If `:header true` put the <column> as a string,
  ;; otherwise put an integer as a column index (indexing starts with 0)
  :mapping {:concept {:code        {:column <column>}
                      :display     {:column <column>}
                      :deprecated? {:column <column>
                                    :true-values ["<string>"]}
                      :parent-id    {:column <column>}
                      :hierarchy-id {:column <column>}}
            :property {"<property-name>" {:column <column>}}}

  :code-system {:url "<CodeSystem url>"}
  :value-set {:url "<ValueSet url>"}}}

Examples

Here are 2 examples importing a fragment of ICD-10. The first one specifies as few parameters as possible and uses a CSV file without a header. The second one uses a CSV with a header and also specifies a deprecation criteria, hierarchy parameters and a concept properties mapping. Both examples have commentaries of used parameters and contents of imported CSV files.

  • CSV file delimiter is ;.This file contains no quoted strings, but since :quote parameter is required ' character is used

  • This file contains no header and data starts at the row 0.

  • Concept.code is mapped to the column with the index 2

  • Concept.display is mapped to the column with the index 3

  • Request will create

    • a CodeSystem/icd10 resource with :url http://hl7.org/fhir/sid/icd-10

    • a ValueSet/icd10 resource with :url http://hl7.org/fhir/ValueSet/icd-10

Request

POST /rpc
accept: application/edn
content-type: application/edn

{:method aidbox.terminology.import-flat/import-from-url
 :params
 {:source-url "https://storage.googleapis.com/aidbox-public/documentation/icd10_example_no_header.csv"
  :format      "csv"
  :csv-format  {:delimiter ";"
                :quote "'"}

  :header   false
  :data-row 0
  :mapping  {:concept {:code    {:column 2}
                       :display {:column 3}}}

  :code-system {:id "icd10", :url "http://hl7.org/fhir/sid/icd-10"}
  :value-set   {:id "icd10", :url "http://hl7.org/fhir/ValueSet/icd-10"}}}

Response:

{:result
  {:count 8,
   :code-system-id "0be7ce48-edab-497c-bb52-186a9ac64746",
   :value-set-id "f4418460-cdac-4847-99be-6ae4c3a64e87"}}

CSV file content

File is also available here

10344;20;XX;External causes of morbidity and mortality;;;1;
16003;2001;V01-X59;Accidents;10344;;1;
15062;20012;W00-X59;Other external causes of accidental injury;16003;;1;10/07/2020
11748;2001203;W50-W64;Exposure to animate mechanical forces;15062;;1;
11870;2001203W64;W64;Exposure to other and unspecified animate mechanical forces;11748;;1;
11871;2001203W640;W64.0;Exposure to other and unspecified animate mechanical forces home while engaged in sports activity;11870;;1;
11872;2001203W641;W64.00;Exposure to other and unspecified animate mechanical forces, home, while engaged in sports activity;11871;;1;
11873;2001203W641;W64.01;Exposure to other and unspecified animate mechanical forces, home, while engaged in leisure activity;11871;;1;

Imported Concept resources

Loaded resources sample (some output parts are omitted)

[{:resourceType "Concept"
  :code "XX"
  :display "External causes of morbidity and mortality"
  :system "http://hl7.org/fhir/sid/icd-10"
  ...}
 ...
 {:resourceType "Concept"
  :code "W00-X59"
  :display "Other external causes of accidental injury"
  :system "http://hl7.org/fhir/sid/icd-10"
  ...}
 ...
 {:resourceType "Concept"
  :code "W64.01"
  :display "Exposure to other and unspecified animate mechanical forces, home, while engaged in leisure activity"
  :system "http://hl7.org/fhir/sid/icd-10"
  ...}]
PreviousImport via FTRNext$import operation

Last updated