coreHTTP Basic S3 Upload Demo
Single Threaded Vs Multi Threaded
There are two coreHTTP usage models, single threaded and multithreaded (multitasking). Although the demo on this page runs the HTTP library in a thread, it is actually demonstrating how to use coreHTTP in a single threaded environment (only one task uses the HTTP API in the demo). Whereas single threaded applications must repeatedly call the HTTP library, multithreaded applications instead can execute sending HTTP requests in the background within an agent (or daemon) task.
Introduction
This example demonstrates sending a PUT request to the AWS S3 HTTP server and uploading a small file. It also performs a GET request to verify the size of the file after the upload. This example uses a network transport interface that uses mbedTLS to establish a mutually authenticated connection between an IoT device client running coreHTTP and AWS S3 HTTP server.
The core HTTP S3 upload demo project uses the
FreeRTOS Windows port, so you can
build and evaluate it with the free Community version of Visual Studios on Windows, without the need for any particular MCU hardware.
Source Code Organization
The demo project is called http_s3_upload_demo.sln and can be found in the FreeRTOS-Plus/Demo/coreHTTP_Windows_Simulator/HTTP_S3_Upload directory of the main FreeRTOS download (and in the coreHTTP_Windows_Simulator
repository on Github)
Building the Demo Project
The demo project uses the
free community edition
of Visual Studio. To build the demo:
- Open the '
http_s3_upload_demo_demo.sln
' Visual Studio solution file from within the Visual Studio IDE.
- Select '
Build Solution
' from the IDE's 'Build
' menu.
Note: If you are using Microsoft Visual Studio 2017 or earlier, then you must select a 'Platform
Toolset
' compatible with your version: 'Project -> RTOSDemos Properties -> Platform Toolset
'.
Configuring the Demo Project
The demo uses the
FreeRTOS-Plus-TCP TCP/IP stack, so follow the instructions provided for the
TCP/IP starter project to ensure you:
- Have the pre-requisite
components installed (such as WinPCap).
- Optionally set a static or
dynamic IP address, gateway address and netmask.
- Optionally set a MAC address.
- Select an Ethernet
network interface on your host machine.
- ...and importantly
test your network connection before attempting to run the HTTP demo.
As delivered, the TCP/IP stack is configured to use a dynamic IP address.
Configuring the AWS S3 HTTP Server Connection
This demo uses a presigned URL to connect the the AWS S3 HTTP server and authorize access to the object to download. The AWS S3 HTTP server's TLS connection uses
server authentication only. At the application level, access to the object is authenticated with parameters in the presigned URL query. The presigned URL generated
using the instructions below expires after one hour, so if an hour or more has elapsed since you generated the URL, you will need to generate the URL again in order
to run the demo application.
Follow the steps below to configure your connection to AWS.
- Set up an Amazon Web Services (AWS) account:
- If you have not already, create and activate an AWS account (which includes a free tier).
- Accounts and permissions are set using AWS Identity and Access Management (IAM). IAM allows you to manage the permissions for each user. By default, no users have permissions until granted by the root owner.
- To add an IAM user to your AWS account, see the IAM User Guide.
- Set permissions for your AWS account to access FreeRTOS and AWS IoT by adding the policies below:
- Create a bucket in S3 by following the steps provided on AWS Docs
- Update a file to S3 by following the steps provided on AWS Docs
- Generate a presigned URL using the script located at
FreeRTOS-Plus/Demo/coreHTTP_Windows_Simulator/Common/presigned_url_generator/presigned_urls_gen.py
. See FreeRTOS-Plus/Demo/coreHTTP_Windows_Simulator/Common/presigned_url_generator/README.md
for usage instructions.
Functionality
The demo first connects to the AWS S3 HTTP server with TLS server authentication. Then it creates an HTTP request to upload the data specified in democonfigDEMO_HTTP_UPLOAD_DATA
. After uploading the file, it checks that file was successfully uploaded by requesting for the size of the file. The structure of the demo can be found in S3UploadHTTPExample.c on Github.
Connecting to the AWS S3 HTTP Server
The function connectToServerWithBackoffRetries()
attempts to make a TCP connection to the HTTP server. If the
connection fails, it retries after a timeout. The timeout value will exponentially increase until the maximum number of
attempts are reached or the maximum timeout value is reached. connectToServerWithBackoffRetries()
returns
a failure status if the TCP connection cannot be established to the server after the configured number of attempts. The source code for connectToServerWithBackoffRetries()
can be found in http_demo_utils.c
on Github.
The function 'prvConnectToServer()
' demonstrates how to establish a connection to
the AWS S3 HTTP server using server authentication only. It uses the mbedTLS-based transport interface that is implemented in the file
'FreeRTOS-Plus/Source/Application-Protocols/network_transport/freertos_plus_tcp/using_mbedtls/using_mbedtls.c
'.
The definition of 'prvConnectToServer()
' can be found in S3UploadHTTPExample.c on Github.
Upload data
The function 'prvUploadS3ObjectFile
' demonstrates how to create a PUT request and specify the file to upload. The AWS S3 bucket to upload to and the name of file upon upload is specified in the presigned URL. To save memory, the same buffer is used for both the request headers and the for receiving the response. The response is received synchronously using API function 'HTTPClient_Send()
'. A 200 OK
response status-code is expected from the AWS S3 HTTP server; any other status-code received is an error.
The source code for 'prvUploadS3ObjectFile
' can be found in S3UploadHTTPExample.c on Github.
Verifying upload
The function 'prvVerifyS3ObjectFileSize
' calls 'prvGetS3ObjectFileSize
' to retrieve the size of the object in the S3 bucket. The S3 HTTP server does not currently support HEAD requests using a presigned URL, so the 0th byte is requested. The size of the file is contained in the response's Content-Range
header field. A 206 Partial Content
response is expected from the server; any other response status-code received is an error.
The source code for 'prvGetS3ObjectFileSize
' can be found in S3DownloadHTTPExample.c on Github.
Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.