Help / Slogging Data via HTTP POST

Note: As we go through some examples, we're assuming you've already created a project and a data table(s) to slog data to. Also, the following code snippets can be generalized and adapted for other languages other than Perl and UNIX.

For the impatient, we'll start with a basic example of how you can slog a CSV file via an HTTP POST.

Slogging CSV File (UNIX) Example

If you have a traditional CSV file with a header and all, you can use the below code. All you have to do is replace EMAIL, PASSWORD, PROJECT_ID, and TABLE_NAME with your own SensorBase information. You also will need to change the data_file field to the correct path (yeah, the "@" is required) to your CSV file. 

curl --connect-timeout 10 --max-time 120 -s -S \
   --form 'email=EMAIL' --form 'password=PASSWORD' \
   --form 'project_id=PROJECT_ID' --form 'table_name=TABLE_NAME' \
   --form 'data_file=@path/to/data.csv' --form 'type=csv' \
   http://sensorbase.org/upload.php

Slogging XML File (UNIX) Example

Similar to the above CSV example, you can also slog XML files in the following format:

<table>
    <row>
        <field name="User_Name">myUser</field>

        <field name="Demo_Name">urbanCens</field>
        <field name="Demo_Author">author_name</field>
        <field name="Affilliation">ucla</field>

        <field name="Num_Of_Visitors">3</field>
        <field name="Rate_Of_Demo">8</field>
        <field name="Comment">great</field>

    </row>
    <row>
        .
        .
        .
    </row>
</table>

When you have XML files in this format, you can use something similar to that of the CSV upload: 

curl --connect-timeout 10 --max-time 120 -s -S \
    --form 'email=EMAIL' --form 'password=PASSWORD' \
    --form 'project_id=PROJECT_ID' --form 'table_name=TABLE_NAME' \
    --form 'data_file=@path/to/data.xml' --form 'type=xml' \
    http://sensorbase.org/upload.php

Notice that the only difference between uploading CSV and XML is the type field and the file that you are uploading.

Slogging Image Files (Perl) Example

The following Perl example is courtesy of Shaun Ahmadian (thanks, Shaun!). Again, you have to replace EMAIL, PASSWORD, PROJECT_ID, and TABLE_NAME with your own SensorBase information. 
#!/usr/bin/perl

use strict;

my $dir=$ARGV[0];
my $url="http://sensorbase.org/upload.php";
my $project_id=PROJECT_ID;
my $sensorbase_user=EMAIL;
my $sensorbase_password=PASSWORD;
my $table_name=TABLE_NAME; 

foreach my $imageFile (<$dir/*>) {
    if ($imageFile=~/mote(\d+)_date\d+_(\d{4})_(\d{2})_(\d{2})_(\d{2})_(\d{2})/) {
        my($mote_id, $year,$mon,$day, $hour,$min)=($1,$2,$3,$4,$5,$6);
        if ($year<2004)
        { next; } # We don't want 1970!

        my $datetime="$year-$mon-$day $hour:$min:00";
             
        my $CURL="curl --connect-timeout 10 --max-time 120 -s -S 
            --form 'email=$sensorbase_user' --form 'password=$sensorbase_password' 
            --form 'project_id=$project_id' --form 'table_name=$table_name' 
            --form 'MAX_FILE_SIZE=2097152' --form 'blob=1' --form 'datetime=$datetime' 
            --form 'location_id=$mote_id' --form 'image=\@$imageFile' $url";
        my $result=`$CURL 2>&1`;
        
        my $error=`echo $?`;
        print $CURL;
        if ($error > 0){
            print STDERR "File: $imageFile : $error \n";
            die "Executing of $CURL failed:\n$result\n";
        }
    print "RESULT: $result\n";
    }
}

The main portion of the above code is where the $CURL statement is formed. The beginning parts of the statement are the same as previous examples where email, password, project_id, and table_name are specified.

After table_name is MAX_FILE_SIZE and blob, which must be specified, where the latter indicates that you are uploading media (e.g. images, audio). The remaining fields are specific to the table where data was being slogged. In this example, an image is getting slogged to a table with fields named "datetime", "location_id", and "image". Note that the "image" field in this example is of media type and so is a pointer to an image file.