Settings View Source Astarte in 5 minutes

This tutorial will guide you through bringing up your Astarte instance, creating a realm and streaming your first data from a device simulator (or a real device) before your cup of tea is ready.

before-you-begin

Before you begin

First of all, please keep in mind that this setup is not meant to be used in production: by default, no persistence is involved, the installation does not have any recovery mechanism, and you will have to restart services manually in case something goes awry. This guide is great if you want to take Astarte for a spin, or if you want to use an isolated instance for development.

You will need a machine with at least 4GB of RAM, a recent 64-bit operating system with Docker, Docker Compose and astartectl installed. If you don't have astartectl installed on your machine yet, you should install it by following the instructions in astartectl's README

Also, on the machine(s) or device(s) you will use as a client, you will need either Docker, or a Qt5 installation with development components if you wish to build and run components locally.

Due to ScyllaDB requirements, if you're working on a Linux machine you should make sure that aio-max-nr is at least 1048576:

cat /proc/sys/fs/aio-max-nr
1048576

If it's less than that, you'll need to edit your /etc/sysctl.conf file

fs.aio-max-nr = 1048576

and to persist this configuration

sudo sysctl -p

checking-prerequistes

Checking prerequistes

Docker version >= 19 is recommended:

$ docker -v
Docker version 19.03.8

Docker compose version >= 2.21 is recommended:

$ docker compose version
Docker Compose version v2.21.0

astartectl >= 22.11 is recommended:

$ astartectl version
astartectl 22.11.02

This procedure has been tested on several systems, and is validated and maintained against Ubuntu 22.04 and macOS 10.15 Catalina, but any other modern operating system should work.

install-astarte

Install Astarte

To get our Astarte instance running as fast as possible, we will install Astarte's standalone distribution. It includes a tunable Docker Compose which brings up Astarte and every companion service needed for it to work. To do so, simply clone Astarte's main repository and use its scripts to bring it up:

$ git clone https://github.com/astarte-platform/astarte.git -b v1.1.1 && cd astarte
$ docker run -v $(pwd)/compose:/compose astarte/docker-compose-initializer:1.1
$ docker compose pull
$ docker compose up -d

docker-compose-initializer will generate a root CA for devices, a key pair for Housekeeping, and a self-signed certificate for the broker (note: this is a really bad idea in production). You can tune the compose file further to use legitimate certificates and custom keys, but this is out of the scope of this tutorial.

Compose might take some time to bring everything up, but usually within a minute from the containers creation Astarte will be ready. You can reach Astarte at the following addresses:

• api.astarte.localhost: Astarte API, in detail:
  • api.astarte.localhost/appengine: AppEngine
  • api.astarte.localhost/housekeeping: Housekeeping
  • api.astarte.localhost/pairing: Pairing
  • api.astarte.localhost/realmmanagement: Realm Management
• broker.astarte.localhost: VerneMQ broker
• dashboard.astarte.localhost: Astarte Dashboard

Moreover, Compose will forward the following ports to your machine:

• 80: HTTP
• 8883: MQTTS

To check everything went fine, use docker ps to verify relevant containers are up: Astarte itself, VerneMQ, CFSSL, RabbitMQ and ScyllaDB should be now running on your system. If any of them isn't up and running, docker ps -a should show it stopped or failed. In those cases, it is advised to issue docker compose up -d again to fix potential temporary failures.

create-a-realm

Create a Realm

Now that we have our instance up and running, we can start setting up a Realm for our device. We'll call our Realm test. Given we have no SSO or Authentication mechanism set up, we're just going to generate a public key to sign our JWTs with. You can create one with astartectl:

$ astartectl utils gen-keypair test

Also, we will need a JWT token to authenticate against Housekeeping. generate-compose-files.sh created a keypair automatically, which is in compose/astarte-keys/housekeeping_{private,public}.pem. To perform all of our Astarte interactions, we will use astartectl.

Use astartectl to create a new Realm:

$ astartectl housekeeping realms create test --astarte-url http://api.astarte.localhost --realm-public-key test_public.pem -k compose/astarte-keys/housekeeping_private.pem

This creates a test realm, which should be ready to be used almost immediately. To ensure your realm is available and ready, check if it exists in Astarte by issuing:

$ astartectl housekeeping realms ls --astarte-url http://api.astarte.localhost -k compose/astarte-keys/housekeeping_private.pem

install-interfaces

Install interfaces

We will use Astarte's Qt5 Stream Generator to feed data into Astarte. However before starting, we will have to install the org.astarte- platform.genericsensors.Values and the org.astarte-platform.genericcommands.ServerCommands interfaces into our new realm. To do that, we can use astartectl again:

$ astartectl realm-management interfaces sync standard-interfaces/org.astarte-platform.genericsensors.Values.json standard-interfaces/org.astarte-platform.genericcommands.ServerCommands.json --astarte-url http://api.astarte.localhost -r test -k test_private.pem -y

Now org.astarte-platform.genericsensors.Values and org.astarte- platform.genericcommands.ServerCommands should show up among our available interfaces:

$ astartectl realm-management interfaces ls --astarte-url http://api.astarte.localhost -r test -k test_private.pem

Our Astarte instance is now ready for our devices.

install-a-trigger

Install a trigger

We will also test Astarte's push capabilities with a trigger. This will send a POST to a URL of our choice every time the value generated by stream_test is above 0.6.

Due to how triggers work, it is fundamental to install the trigger before a device connects. Doing otherwise will cause the trigger to kick in at a later time, and as such no events will be streamed for a while.

Replace $TRIGGER_TARGET_URL with your target URL in the example below, you can use a Postbin service like Mailgun Postbin to generate a URL and see the POST requests. The resulting trigger would be:

{
  "name": "my_trigger",
  "action": {
    "http_url": "$TRIGGER_TARGET_URL",
    "http_method": "post"
  },
  "simple_triggers": [
    {
      "type": "data_trigger",
      "on": "incoming_data",
      "interface_name": "org.astarte-platform.genericsensors.Values",
      "interface_major": 1,
      "match_path": "/streamTest/value",
      "value_match_operator": ">",
      "known_value": 0.6
    }
  ]
}

Replace $TRIGGER_TARGET_URL with the URL your Trigger will target. Assuming you saved this as my_trigger.json, you can now install it through astartectl:

$ astartectl realm-management triggers install my_trigger.json --astarte-url http://api.astarte.localhost -r test -k test_private.pem

You can now check that your trigger is correctly installed:

$ astartectl realm-management triggers ls --astarte-url http://api.astarte.localhost -r test -k test_private.pem

stream-data

Stream data

If you already have an Astarte compliant device, you can configure it and connect it straight away, and it will just work with your new installation - provided you skip SSL checks on the broker's certificate. If you don't, you can use Astarte's stream-qt5-test to emulate an Astarte device and generate a datastream. You can do this either on the same machine where you are running Astarte, or from another machine or device on the same network.

using-a-container-for-stream-qt5-test

Using a container for stream-qt5-test

Astarte's stream-qt5-test can be pulled from Docker Hub with:

$ docker pull astarte/astarte-stream-qt5-test:1.0.4

Its most basic invocation (from your astarte repository tree) is:

$ docker run --net="host" -e "DEVICE_ID=$(astartectl utils device-id generate-random)" -e "PAIRING_URL=http://api.astarte.localhost/pairing" -e "REALM=test" -e "PAIRING_JWT=$(astartectl utils gen-jwt pairing -k test_private.pem)" -e "IGNORE_SSL_ERRORS=true" astarte/astarte-stream-qt5-test:1.0.4

This will generate a random datastream from a brand new, random Device ID. You can tweak those parameters to whatever suits you better by having a look at the Dockerfile. You can spawn any number of instances you like, or you can have the same Device ID send longer streams of data by saving the container's persistency through a Docker Volume. If you wish to do so, simply add -v /persistency:<your persistency path> to your docker run invocation.

Refer to stream-qt5-test README for more details on which variables can be passed to the container.

Also, please note that the --net="host" parameter is required to make localhost work. If this is not desirable, you can change PAIRING_URL to an host reachable from within the container network. Obviously, that parameter isn't required if you're running the container on a different machine and PAIRING_URL is pointing to a different URL.

grab-your-tea

Grab your tea

Congratulations! Your devices or fake devices are now communicating with Astarte, and your tea should be ready by now. You can check if everything is working out by invoking AppEngine APIs to get some values. In case you are using stream-qt5-test, you can get the last sent value with astartectl:

$ astartectl appengine devices get-samples <your device id> org.astarte-platform.genericsensors.Values /streamTest/value --count 1  --astarte-url http://api.astarte.localhost -r test -k test_private.pem

If you get a meaningful value, congratulations - you have a working Astarte installation with your first datastream coming in!

Moreover, Astarte's Docker Compose also installs Astarte Dashboard, from which you can manage your Realms and install Triggers, Interfaces and more from a Web UI. It is accessible by default at http://dashboard.astarte.localhost - remember that if you are not exposing Astarte from localhost, you have to change Realm Management API's URL in Dashboard's configuration file, to be found in compose/astarte-dashboard/config.json in Astarte's repository. You can generate a token for Astarte Dashboard, as usual, through astartectl utils gen-jwt all-realm-apis -k test_private.pem. By default, astartectl will generate a token valid for 8 hours, but you can set a specific expiration by using the -e <seconds> parameter.

From here on, you can use all of Astarte's APIs and features from your own installation. You can add devices, experiment with interfaces, or develop your own applications on top of Astarte's triggers or AppEngine's APIs. And have a lot of fun!

cleaning-up

Cleaning up

When you're done with your tests and developments, you can use docker compose again to tear down your Astarte instance simply by issuing:

$ docker compose down

Unless you add the -v option, persistencies will be kept and next time you will docker compose up the cluster will come back in the very same state you left it last time. docker compose down -v is extremely useful during development, especially if you want a clean slate for testing your applications or your routines every time.

final-notes

Final notes

Running Astarte through docker compose is the fastest way for going from zero to hero. However, please keep in mind this setup is unlikely to hold for long in production, and is by design broken for large installations. We can't stop you from running such a thing in production, but do so as long as you know you voided your warranty by doing so.

This method is great for development and for trying out the system. If you wish to deploy Astarte in a more robust environment, have a look at Astarte Enterprise or, if you want to go the DIY way, make sure that at least every service which requires persistency has reliable storage and adequate redundancy beneath it.