AidboxTopicSubscription NATS tutorial

Objectives

• Learn how to integrate AidboxTopicSubscriptions with core NATS / NATS JetStream

What is NATS?

NATS is a high-performance, open-source messaging broker designed for communication between services and microservices. It provides simple and fast message delivery using the publish/subscribe (pub/sub) pattern and supports various application interaction scenarios.

JetStream vs Core NATS

Core NATS is a lightweight pub/sub system where messages are delivered to subscribers only at the moment of publishing (best-effort delivery). If a subscriber is offline, the message is lost.

JetStream is an extension of NATS that provides message persistence, replay, and acknowledgment (at least once delivery). It allows you to create streams, store messages, manage consumers, and ensure that important events are not lost even in the case of failure.

In Aidbox, create AidboxTopicDestination with http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-core-best-effort profile to integrate with Core NATS and http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-jetstream-at-least-once to integrate with NATS JetStream.

Setting up

1. Create a directory structure.

mkdir nats && cd nats
mkdir jwt-auth
mkdir jetstream

1. Go to the getting started guide. Create docker-compose.yaml and add these lines:

  aidbox:
    extra_hosts:
    # connect Aidbox container and NATS from localhost
    - "host.docker.internal:host-gateway"
    volumes:
    # module jar to turn on nats support
    - ./topic-destination-nats-2505.1.jar:/topic-destination-nats-2505.1.jar
    # creds used in authentication
    - ./jwt-auth/creds:/creds
    # ...
    environment:
      BOX_MODULE_LOAD: io.healthsamurai.topic-destination.nats.core
      BOX_MODULE_JAR: "/topic-destination-nats-2505.1.jar"
      # ... other envs ...

1. Download .jar NATS module file from our bucket and place it next to docker-compose.yaml.

curl -O https://storage.googleapis.com/aidbox-modules/topic-destination-nats/topic-destination-nats-2505.1.jar

1. Install NATS CLIs: nats-server and nsc.

go install github.com/nats-io/nats-server/v2@latest
go install github.com/nats-io/nsc/v2@latest

1. Start Aidbox.

docker compose up

Now, in AidboxUI, go to FHIR Packages -> io.healthsamurai.topic and make sure that NATS profiles are present.

Basic Usage

1. Start NATS.

nats-server

1. Subscribe to the foo subject.

nats sub foo

1. Check that publishing and subscribing work.

nats pub foo "Hello, NATS"

1. Go to AidboxUI and create a topic that triggers if Patient.name exists.

POST /fhir/AidboxSubscriptionTopic
content-type: application/json
accept: application/json

{
  "resourceType": "AidboxSubscriptionTopic",
  "url": "patient-topic",
  "status": "active",
  "trigger": [
    {
      "resource": "Patient",
      "fhirPathCriteria": "name.exists()"
    }
  ]
}

1. Create AidboxTopicDestination with http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-core-best-effort profile.

POST /fhir/AidboxTopicDestination
content-type: application/json
accept: application/json

{
  "id": "basic",
  "resourceType": "AidboxTopicDestination",
  "meta": {
    "profile": [
      "http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-core-best-effort"
    ]
  },
  "kind": "nats-core-best-effort",
  "topic": "patient-topic",
  "parameter": [
    {
      "name": "url",
      "valueString": "nats://host.docker.internal:4222"
    },
    {
      "name": "subject",
      "valueString": "foo"
    },
    {
      "name": "sslContext",
      "valueString": "none"
    }
  ]
}

1. Create a patient with a name.

POST /fhir/Patient

name:
- family: smith

1. See the output in your terminal that the Patient is created.

[#3] Received on "foo"
content-type: application/json

{"topic":"foo","value":{"resourceType":"Bundle","type":"history","timestamp":"2025-05-05T09:54:29Z","entry":[{"resource":{"resourceType":"AidboxSubscriptionStatus","status":"active","type":"event-notification","notificationEvent":[{"eventNumber":1,"focus":{"reference":"Patient/5e78f7b8-e6dc-404b-93b5-2356ed1e48b6"}}],"topic":"patient-topic","topic-destination":{"reference":"AidboxTopicDestination/basic"}}},{"request":{"method":"POST","url":"/fhir/Patient"},"fullUrl":"http://localhost:8080/fhir/Patient/5e78f7b8-e6dc-404b-93b5-2356ed1e48b6","resource":{"name":[{"family":"smith"}],"id":"5e78f7b8-e6dc-404b-93b5-2356ed1e48b6","resourceType":"Patient","meta":{"lastUpdated":"2025-05-05T09:54:29.496342Z","versionId":"17","extension":[{"url":"https://aidbox.app/ex/createdAt","valueInstant":"2025-05-05T09:54:29.496342Z"}]}}}]}}

NATS JetStream

1. Change working directory.

cd jetstream

1. Create stream.

nats stream add EVENTS \
  --subjects="patients.>" \
  --storage=file \
  --defaults

1. Publish a message to the stream.

nats pub patients.test "hello JetStream"

1. Check the stream saves the message.

nats stream ls

The output:

╭─────────────────────────────────────────────────────────────────────────────╮
│                                   Streams                                   │
├────────┬─────────────┬─────────────────────┬──────────┬──────┬──────────────┤
│ Name   │ Description │ Created             │ Messages │ Size │ Last Message │
├────────┼─────────────┼─────────────────────┼──────────┼──────┼──────────────┤
│ EVENTS │             │ 2025-05-05 13:49:16 │ 1        │ 58 B │ 2.14s        │
╰────────┴─────────────┴─────────────────────┴──────────┴──────┴──────────────╯

1. Create AidboxTopicDestination with http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-jetstream-at-least-once profile.

POST /fhir/AidboxTopicDestination
content-type: application/json
accept: application/json

{
  "resourceType": "AidboxTopicDestination",
  "meta": {
    "profile": [
      "http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-jetstream-at-least-once"
    ]
  },
  "kind": "nats-jetstream-at-least-once",
  "id": "jetstream",
  "topic": "patient-topic",
  "parameter": [
    {
      "name": "url",
      "valueString": "nats://host.docker.internal:4222"
    },
    {
      "name": "subject",
      "valueString": "patients.created"
    },
    {
      "name": "sslContext",
      "valueString": "none"
    }
  ]
}

1. Create a patient.

POST /fhir/Patient

name:
- family: smith

1. Create a stream consumer.

nats consumer add EVENTS my-consumer --defaults --pull

1. Pull the first available message.

nats consumer next EVENTS my-consumer

It is our message published from CLI:

[13:59:58] subj: patients.test / tries: 1 / cons seq: 1 / str seq: 1 / pending: 1

hello JetStream

Acknowledged message

1. Pull next message.

nats consumer next EVENTS my-consumer

It is the message from Aidbox:the

[14:00:21] subj: patients.created / tries: 1 / cons seq: 2 / str seq: 2 / pending: 0

Headers:

  content-type: application/json

Data:


{"topic":"patients.created","value":{"resourceType":"Bundle","type":"history","timestamp":"2025-05-05T10:57:54Z","entry":[{"resource":{"resourceType":"AidboxSubscriptionStatus","status":"active","type":"event-notification","notificationEvent":[{"eventNumber":1,"focus":{"reference":"Patient/598ef035-89f7-4b19-9ad7-fa4ad8f38681"}}],"topic":"patient-topic","topic-destination":{"reference":"AidboxTopicDestination/jetstream"}}},{"request":{"method":"POST","url":"/fhir/Patient"},"fullUrl":"http://localhost:8080/fhir/Patient/598ef035-89f7-4b19-9ad7-fa4ad8f38681","resource":{"name":[{"family":"smith"}],"id":"598ef035-89f7-4b19-9ad7-fa4ad8f38681","resourceType":"Patient","meta":{"lastUpdated":"2025-05-05T10:57:54.908063Z","versionId":"35","extension":[{"url":"https://aidbox.app/ex/createdAt","valueInstant":"2025-05-05T10:57:54.908063Z"}]}}}]}}

Acknowledged message

JWT authentication

1. Turn off the previous nats-server (Ctrl+C).

2. Change working directory.

cd jwt-auth

1. Set up NATS operator to use the generated signing key:

nsc add operator --generate-signing-key --sys --name local

nsc edit operator --require-signing-keys \
  --account-jwt-server-url "nats://localhost:4222"

1. Create APP account and generate a new signed key for it.

nsc add account APP
nsc edit account APP --sk generate

1. Create server.conf and resolver.conf files

nsc generate config --nats-resolver --sys-account SYS > resolver.conf

cat <<- EOF > server.conf
include resolver.conf
EOF

Now the server config includes your operator, accounts, and JWT resolver settings.

1. Upload your APP account to the NATS server, so NATS knows what APP is.

nsc push -a APP

1. Create user joe and save it in the APP. Joe can publish to subjects that start with "joe".

nsc add user --account APP joe --allow-pub 'joe.>'

nats context save joe --nsc nsc://local/APP/joe

1. Do the same to pam. Pam can publish to subjects that start with "pam".

nsc add user --account APP pam --allow-pub 'pam.>'

nats context save pam --nsc nsc://local/APP/pam

1. Create user admin that can do everything.

nsc add user --account APP admin \
  --allow-pub '>' \
  --allow-sub '>'

nats context save admin --nsc nsc://local/APP/admin

1. Move creds next to docker-compose.yaml file to mount them to use in Aidbox.

mkdir -p creds
# make sure the path to creds is right 
cp -r ~/.local/share/nats/nsc/keys/creds/local/APP/* creds

1. Start NATS server.

nats-server -c server.conf 

1. Try to push to joe.message using Joe's credentials.

nats pub joe.message "hello from joe" --context joe
# same thing using --creds
# nats pub joe.message "hello from joe" --creds creds/joe.creds

1. Try to push to pam.message using Joe's credentials.

nats pub pam.message "hello from joe" --context joe

Access denied, the error:

nats: error: nats: permissions violation: Permissions Violation for Publish to "pam.message"

1. Create joe-to-pam AidboxTopicDestination. Note that Aidbox, as a Client, has no idea about rights in best-effort mode.

POST /fhir/AidboxTopicDestination
content-type: application/json
accept: application/json

{
  "id": "joe-to-pam",
  "resourceType": "AidboxTopicDestination",
  "meta": {
    "profile": [
      "http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-core-best-effort"
    ]
  },
  "kind": "nats-core-best-effort",
  "topic": "patient-topic",
  "parameter": [
    {
      "name": "url",
      "valueString": "nats://host.docker.internal:4222"
    },
    {
      "name": "subject",
      "valueString": "pam.a"
    },
    {
      "name": "connectionName",
      "valueString": "connectionname"
    },
    {
      "name": "credentialsFilePath",
      "valueString": "/creds/joe.creds"
    },
    {
      "name": "sslContext",
      "valueString": "none"
    }
  ]
}

1. Use Joe's credentials to publish to joe subject.

POST /fhir/AidboxTopicDestination
content-type: application/json
accept: application/json

{
  "id": "joe-to-joe",
  "resourceType": "AidboxTopicDestination",
  "meta": {
    "profile": [
      "http://aidbox.app/StructureDefinition/aidboxtopicdestination-nats-core-best-effort"
    ]
  },
  "kind": "nats-core-best-effort",
  "topic": "patient-topic",
  "parameter": [
    {
      "name": "url",
      "valueString": "nats://host.docker.internal:4222"
    },
    {
      "name": "subject",
      "valueString": "joe.message"
    },
    {
      "name": "connectionName",
      "valueString": "connectionname"
    },
    {
      "name": "credentialsFilePath",
      "valueString": "/creds/joe.creds"
    },
    {
      "name": "sslContext",
      "valueString": "none"
    }
  ]
}

1. From your terminal, subscribe to joe.message from admin.

nats sub joe.message --creds ./creds/admin.creds

You can try to subscribe to the subject using Joe's or Pam's credentials, but they do not have the right to do so.

1. Post the patient with a name.

POST /fhir/Patient

name:
- family: smith

See the output in the terminal. The error from joe-pam TopicDestination trying to publish without rights:

Publish Violation - Nkey "UDVE47SBIUZZM76JDTR6T2RBYF2AANUZH7WTJZYHHZF36GACGXQUGMUF", Subject "pam.a"