# How to send data to AWS Kinesis

<div align="left"><figure><img src="https://2165942204-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMKXTFIN5ZlLOjBlfC4%2Fuploads%2F6hIEzbQsA3N0eEgc2BBF%2Fenterprise.png?alt=media&#x26;token=a2ef5bf8-1e13-4c49-bf5e-2d00005cbd3c" alt="" width="103"><figcaption></figcaption></figure></div>

![AWS Kinesis Output Connector](https://2165942204-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMKXTFIN5ZlLOjBlfC4%2Fuploads%2FiTAPbqu1oreujN50zfJb%2Fakenza-kinesis%20\(1\).png?alt=media\&token=671830e8-15bf-4a38-bfc3-f37c50510e82)

{% hint style="success" %}
**Benefits of the AWS Kinesis output connector**:

* Directly stream device data from akenza to **AWS Kinesis** and subsequently process it in [AWS Kinesis](https://aws.amazon.com/kinesis/data-firehose/?nc1=h_ls\&kinesis-blogs.sort-by=item.additionalFields.createdDate\&kinesis-blogs.sort-order=desc)
* Easily process **low-power device** data e.g. LoRaWAN device data in a **AWS Kinesis**
  {% endhint %}

In this tutorial, you will learn, how to send data to **AWS Kinesis**. The following steps are explained in more detail:

1. [Setup an **AWS Kinesis** stream](#1.-setup-an-amazon-kinesis-stream)
2. [Setup a **Data Flow** in akenza](#2.-setup-a-data-flow-in-akenza)&#x20;
3. [Monitor events](#3.-check-events)
4. [Use of further AWS services](#4.-use-of-further-kinesis-services)
5. [Useful Links](#5.-useful-links)

### 1. Setup an AWS Kinesis stream

First, you will need to set up AWS kinesis in order to successfully process data from akenza in AWS services.

* Sign in to the [Amazon portal](https://aws.amazon.com/kinesis/data-firehose/?nc1=h_ls\&kinesis-blogs.sort-by=item.additionalFields.createdDate\&kinesis-blogs.sort-order=desc)
* Create a new **AWS Kinesis stream resource**, by searching for Kinesis and creating a data stream
  * copy the **stream ARN** for later use
* Create a new **API** user by searching for IAM
  * create a new policy with access to the Kinesis service
  * Expand the tab *Write* and select&#x20;
    * *PutRecord*
    * *PutRecords*&#x20;
    * *ListShards*
  * Add the ARN of the newly created kinesis stream as *resources*&#x20;
  * &#x20;create a new user which uses access key as the credentials type and the previously created policy

<figure><img src="https://2165942204-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMKXTFIN5ZlLOjBlfC4%2Fuploads%2FcbvMkyC7udiedfnCWM1U%2Fscreely-1733848538408.png?alt=media&#x26;token=2f0705f9-67a6-4ab7-8643-33f745e99c2b" alt=""><figcaption><p>Kinesis Policy</p></figcaption></figure>

### 2. Setup a Data Flow in akenza

In akenza, create a new **Data Flow** with **AWS Kinesis** as Output Connector. Define a **Connector Name**, enter the **IAM user id** and **key** as well as the Kinesis **data stream ARN**. Save the connector and Data Flow accordingly.

<figure><img src="https://2165942204-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MMKXTFIN5ZlLOjBlfC4%2Fuploads%2FTCkFzE9RRCGmDNQHVxBj%2Fscreely-1733848899379.png?alt=media&#x26;token=369e9beb-7b24-469f-b1bb-893b16bd044a" alt=""><figcaption><p>New AWS Kinesis Data Flow</p></figcaption></figure>

[Create a new device](https://docs.akenza.io/akenza.io/how-to-create-a-working-environment/how-to-connect-a-device) using the Data Flow and start sending data.

{% hint style="info" %}
Optionally: The custom payload can be selected for making use of templated payload in the desired structure. If the custom payload is not selected, the whole data sample will be forwarded.
{% endhint %}

### 3. Monitor Events

In order to monitor uplinks that are sent to AWS Kinesis, you can use [AWS cli](https://docs.aws.amazon.com/streams/latest/dev/kinesis-tutorial-cli-installation.html). To check if the uplinks arrived execute as a first step the following command and copy the return value. This value is the shard iterator used in the following commands.

```
aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type LATEST --stream-name {streamName} --query 'ShardIterator'
```

Send data to your device and execute the following command accordingly.

```
aws kinesis get-records --shard-iterator {shardIterator}
```

You should now see your base64 encoded records as well as the next shard iterator. If there is no data repeat the previous command with the next shard iterator.&#x20;

### 4. Use of further AWS services

Once a device is connected to the kinesis, device data can be further used on various **AWS services:**

{% hint style="success" %}
**Possible services are:**

* [AWS kinesis firehouse](https://aws.amazon.com/kinesis/data-firehose/?kinesis-blogs.sort-by=item.additionalFields.createdDate\&kinesis-blogs.sort-order=desc)

  Real-time data stream processing
* [AWS S3](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank\&all-free-tier.sort-order=asc\&awsf.Free%20Tier%20Categories=categories%23storage\&trk=ps_a134p000006pabbAAA\&trkCampaign=acq_paid_search_brand\&sc_channel=PS\&sc_campaign=acquisition_DACH\&sc_publisher=Google\&sc_category=Storage\&sc_country=DACH\&sc_geo=EMEA\&sc_outcome=acq\&sc_detail=amazon%20s3\&sc_content=S3_e\&sc_matchtype=e\&sc_segment=495037812850\&sc_medium=ACQ-P|PS-GO|Brand|Desktop|SU|Storage|S3|DACH|EN|Text|xx|Non-EU\&s_kwcid=AL!4422!3!495037812850!e!!g!!amazon%20s3\&ef_id=CjwKCAiAp8iMBhAqEiwAJb94z9jJGT0S1873IRB7RaQMMvlDMxWaTy12Ve3HtFSLR2wL0O0_YC5AjRoCQMwQAvD_BwE:G:s\&s_kwcid=AL!4422!3!495037812850!e!!g!!amazon%20s3\&awsf.Free%20Tier%20Types=*all)

  Object storage is built to retrieve any amount of data from anywhere
* [AWS Lambda](https://aws.amazon.com/lambda/)

  Run code without thinking about servers or clusters
  {% endhint %}

### 5. Useful Links

{% embed url="<https://eu-central-1.console.aws.amazon.com/kinesis/home?region=eu-central-1#/home>" %}

{% embed url="<https://aws.amazon.com/s3?nc2=type_a>" %}

{% embed url="<https://aws.amazon.com/lambda>" %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.akenza.io/akenza.io/tutorials/create-enterprise-solutions/how-to-send-data-to-aws-kinesis.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
