Direct Device Uploads

Devices can upload directly to data streams over any of the existing transports (TCP, UDP, SMS, and Satellite). The path specified in the data service message begins with DataPoint and the rest of the message is mapped to a data stream appended to the device ID.

For example, if the device sends a data point file specifying the filename DataPoint/temp1, the data point is added to the data stream /temp1. The file must follow one of the expected formats and must specify the format via the file extension. The following types are supported for a given extension:

Data limits related to direct device uploads

To maximize the speed and throughput of Remote Manager, limitations have been imposed on device uploads.

• Maximum number of data points allowed per request: 250
• Maximum size of Send Data requests: 2MB
• Maximum size of replies to Device Requests: 2MB
• Maximum number of binary data points allowed: 64KB

When devices push data points up to Remote Manager, the description included refers to the data point, not the data stream. To view the description, you must retrieve data point via web services.

XML

XML format uses the same format used in /ws/DataPoint PUT. The stream id is ignored since it is provided by the path. Also, any streams listed in the forwardTo field will be normalized to the device’s stream. This is done to prevent one device from uploading data into another device’s stream.

<DataPoint>
    <data>42</data>
    <!-- Everything below this is optional -->
    <description>Temperature at device 1</description>
    <location>0.0, 0.0, 0.0</location> 
    <quality>99</quality>
    <dataType>float</dataType>
    <units>Kelvin</units>
</DataPoint>

For multiple data points in one message:

<list>
  <DataPoint>
    <data>42</data>
    <timestamp>1234566</timestamp>
  </DataPoint>
  <DataPoint>
    <data>43</data>
  </DataPoint>
</list>

CSV

An optional upload format is to specify the data in UTF-8 encoded comma separated values. Each line ('\n' terminated) specifies a data point. The default order is:

DATA, TIMESTAMP, QUALITY, DESCRIPTION, LOCATION, DATATYPE, UNITS, FORWARDTO

Meaning the following file:

data, 1,99,"my description",,INTEGER,kelvins,"stream1,stream2"
data2,2,50,"my description"
data3,3,25,"my description"

Would create 3 data points, set the stream’s units/type to kelvins/Integers, and have the data points with the data “data”, “data2”, and “data3”, using the epoch timestamps of 1, 2, and 3.

Note that location was omitted in the above example. You can omit values by leaving them empty or stopping before the end. For example:

**Empty values:**data,1,,,99

**Ending early:**data,1

Order can be overridden. You can define a header on the first line by starting it with a ‘#’ character, for example:

#TIMESTAMP,DATA
1, data
2, data2
3, data3

Will create 3 data points 1ms apart starting at epoch (1970).

Multiple datapoints for multiple streams from a device can be inserted in one message using the STREAMID value. When the STREAMID value is specified, the file name is no longer used for the stream name.

For example:

#STREAMID,DATA,TIMESTAMP
sensor1/port1,97,1
sensor1/port2,98,1
sensor2/port1,42,1
sensor2/port2,0,2

Will create 4 data points, one in each of 4 separate streams for the device. The first 3 data points are at 1ms after the epoch (1970) and the final data point is 1ms later.

The XML version is as follows:

<list>
<DataPoint><streamId>sensor1/port1</streamId><data>97</data><timestamp>1</timestamp></DataPoint>
<DataPoint><streamId>sensor1/port2</streamId><data>98</data><timestamp>1</timestamp></DataPoint>
<DataPoint><streamId>sensor2/port1</streamId><data>42</data><timestamp>1</timestamp></DataPoint>
<DataPoint><streamId>sensor2/port2</streamId><data>0</data><timestamp>2</timestamp></DataPoint>
</list>

Binary Concise Alternative Format

The disadvantage to using the XML format is that it is very verbose. This binary alternative format can be used to be more concise. You can specify a simple value instead of XML or CSV data. When the value is pushed to /DataPoint, it is stored in complete as-is in time-series data (in the exact binary format as provided). For details on endianness, bit lengths, and so on for supported data types see the dataType in the Data Streams section. However, data types are not required. Data can be 1 byte status indicators or 10k images but Remote Manager will not be able to provide rollups on things which do not use the specified formats.

For instance, the following data service message:

Will result in a new data point with a value of “42” (in binary).

Note: The binary concise mechanism has the following limitations:

• Only single values can be uploaded per data service message
• Data must be smaller than 64k

Deciding which format to use when inserting data

Whitespace characters for the data value are preserved in all formats. Use quotes around the string for CSV format to preserve break lines. For binary data, we recommend you to use binary concise format. Binary concise format however can’t be used to create multiple data points in a single request. To create multiple binary data points in a single request, we recommend you to use a base64 encoded string.