Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem


Loading

FreeRTOS_select starvation

Posted by hannes23 on June 1, 2018

Hello, at my project I've setup a TCP server based on the one found in FreeRTOS-Plus. The hardware is a Zynq-7000 variant, and the FreeRTOS Version is FreeRTOSv10.0.0.

The TCP server calls a HTTP worker. The server has the FreeRTOS_select() function to wait for incoming TCP packets. This function is called with a blocking time of 100 ms. The job of the HTTP-worker is to receive a HTTP url followed by sending an amount of 10 MByte data. This is done continuosly. I've changed the HTTP-worker according to my needs in that, that the prvSendFile() routine gives to a semaphore and returns . The semaphore is then taken by a separate task (named prvHTTPClientTask), which does the sending. Each new client gets a new prvHTTPClientTask. So HTTP clients on th PC can work better in in parallel. But the problem, which I describe here, also arises with a single client.

The whole job is running quiet well for a long time, (some gigabyte sent), if the task priority of the TCP-server ( 6 ) is lower than the priority of the prvHTTPClientTask ( 7 ). If I change the priority of the TCP-server to equal or higher piority, the job will starve after a short time. Maybe 2 - 20 of the 10-MByte sendings. What I observe is that the FreeRTOS_select() function never returns, although a blocking time of 100 ms is given.

Can anybody give a hint here? Thanks and greetings.


FreeRTOS_select starvation

Posted by heinbali01 on June 1, 2018

Hi Johannes,

it is difficult to tell what goes wrong in your application without seeing it. But I'll give it a try.

So HTTP clients on the PC can work better in in parallel.

Every task runs on the same CPU (core), so I do not think that your solution will be more efficient.

You are using FreeRTOS_select(). When you start up a new task to handle a connection, do you remove the socket from the select mask?

A problem with select() in general is that it sometimes returns immediately without blocking.

Suppose you have the TX bit enable for some socket, either you must send data, or you must disable the TX bit. Could it be that the "starvation" is caused by select() constantly returning without ever blocking?

Please report your findings here!


FreeRTOS_select starvation

Posted by hannes23 on June 4, 2018

Hi Hein, thank you for your comments.

The mechanism to handle TCP connections and recv/transmit is the same as shown in the file 'FreeRTOSTCPserver.c', which I took as a template. The procedure is, that a) FreeRTOSselect() waits for TCP-events, then if the socket is already active b) runs through FreeRTOSaccept(), followed by b1) prvReceiveNewClient(), when it's a new connection or b2) handles the rx or tx in a worker-routine.

In the original files FreeRTOSTCPserver and FreeRTOSHTTPserver.c work so, that upon each received data (URL) a file is sent back to the client. And all clients are handled one by one. So if a clients awaits a big amount of data, then the next clients have to wait . To avoid this I spawn a new task for each client, which does the probably long-standing sending.

I do not remove a socket from the select-list, but only, when the connection is closed. Just like in the original file FreeRTOSTCPserver.

And yes, like in the original FreeRTOSHTTPserver.c file I set the tx-bit, when the sending routine returns, because no tx-space is available. The send will then be continued after the next successful return from FreeRTOS_select().

The question is simply, how can it be, that FreeRTOS_select() blocks forever, although a blocking time is given in the call.

The other question is how should I generally setup a solution for my TCP / HTTP communication needs. I will have 1 to 4 clients requesting data exchange with possibly big amount of data, 10 MByte may be. I took as template the mentioned files, but think about to use a solution, where one task is responsible for connection to clients and spawning a worker-task for each connection. This would avoid for each data to go through FreeRTOS_accept() each time and wouldn't use the select() routine. Which situation has an advantage to use select()?

Any comments welcome and thank you in advance.


FreeRTOS_select starvation

Posted by hannes23 on June 5, 2018

Is there an error in FreeRTOS_select() ?

Since many days I'm fighting with a problem that under certain conditions FreeRTOS_select() not returns, although the blocking-time is max. 100 ms.

The probability that this happens is high, when the task wich calls FreeRTOS_select() has a priority that is equal or higher than other tasks which send or receive TCP packets. Is the priority lower, then the server has to transfer many packets until the fault happens.

To find out, where the task finally stops in the faulty situation, I entered a debug variable with different values at certain points of the called functions.

Thereby I found, that FreeRTOSselect() calls prvFindSelectedSocket() which then calls xEventGroupWaitBits(), with portMAXDELAY as timeout, and then in this function the debug variable was finally set before calling xTaskResumeAll().

Is this probably a bug? :-( Comments and hints are greatly welcome.


FreeRTOS_select starvation

Posted by hannes23 on June 5, 2018

Is there an error in FreeRTOS_select() ?

Since many days I'm fighting with a problem that under certain conditions FreeRTOS_select() not returns, although the blocking-time is max. 100 ms.

The probability that this happens is high, when the task wich calls FreeRTOS_select() has a priority that is equal or higher than other tasks which send or receive TCP packets. Is the priority lower, then the server has to transfer many packets until the fault happens.

To find out, where the task finally stops in the faulty situation, I entered a debug variable with different values at certain points of the called functions.

Thereby I found, that FreeRTOSselect() calls prvFindSelectedSocket() which then calls xEventGroupWaitBits(), with portMAXDELAY as timeout, and then in this function the debug variable was finally set before calling xTaskResumeAll().

Is this probably a bug? :-( Comments and hints are greatly welcome.


FreeRTOS_select starvation

Posted by rtel on June 5, 2018

Can you please try with ipconfigTCPHANGPROTECTION set to 1 in FreeRTOSIPConfig.h - thanks.


FreeRTOS_select starvation

Posted by hannes23 on June 6, 2018

Hello Richard, the hang-protection was already active, but the function FreeRTOSselect() didn't return. What I indeed find strange is that this routine can be given a max. blocking-time, one of the inside called functions however is called with portMAXDELAY. When this should be overruled by the hang-protection mechanism, I think it didn't work. What I meanwhile found is that the problem seems to vanish when the task-priority of the task, which calls FreeRTOSselect(), (TCP-Server) , is set to tskIDLEPRIORITY. Thanks and Greetings.


FreeRTOS_select starvation

Posted by heinbali01 on June 6, 2018

Hi Johannes, FreeRTOS_select() will wait for an event group. Event groups are used all over the library, also in the API's.

the problem seems to vanish when the task-priority of the task which calls FreeRTOSselect(), (TCP-Server) , is set to tskIDLEPRIORITY

That makes me wonder if your diagnosis is correct: lowering the priority of a task calling a blocking function would only lower the chances of returning ( with or without a time-out ). And that will happen if the CPU is too busy with running tasks of a higher priority.

Please make sure that the idle task gets plenty of running time, i.e. make sure that there is no task that is running constantly running or runnable ( unless you give that task the lowest possible priority ).

Also please make sure you have understood my texts here above about using [FreeRTOS_]select(). If you use the TX mask, make sure that you either send data, or you disable the TX mask.

FreeRTOS_select() and xEventGroupWaitBits()() have already been used extensively in many projects and I'm not aware of problems. It is hard to guess what goes wrong in your project without seeing the source code. Best,


FreeRTOS_select starvation

Posted by hannes23 on June 7, 2018

Hi Hein, of course I'm not sure, whether I understood everything right and also for sure I cannot code error-free :-) But I try. And for a better understanding I've attached the hopeful necessary files.

Some aspects of this problem I have meanwhile understood a bit. In alteration of the original files 'FreeRTOSTCPserver.c' and 'FreeRTOSHTTPserver.c' I start an extra client task after a TCP connection is made, which should be responsible for the sending. And here the task priority gets important. If the TCP-server which calls select() has a higher priority, than the extra sending task isn't active as should be necessary. When the communication stucks, the select function in this case returns the event-bits. The tx-bit is set, but due to priority-scheme the sending task will not be running and the whole thing is stuck. If the TCP-server has the same priority, then the select() function, when the communication is stuck, doesn't return as mentioned above. But if the priority of the TCP-server is lower, zero in my case, the communication is running well.

So the only question which remains is, why the select() function doesn't return if the priorities of the TCP-server and the client-sending-task are the same. Thanks and greetings.

Attachments

FreeRTOSIPConfig.h (22206 bytes)
FreeRTOS_TCP_server.c (12402 bytes)

FreeRTOS_select starvation

Posted by heinbali01 on June 7, 2018

HI Johannes, would you mind to send me an email and we'll try to get through this by email. My email address is hein [at] htibosch [dot] net


FreeRTOS_select starvation

Posted by heinbali01 on June 7, 2018

[ sorry, I pressed twice on the Post button ]


[ Back to the top ]    [ About FreeRTOS ]    [ Privacy ]    [ Sitemap ]    [ ]


Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.

Latest News

FreeRTOS v10.2.1 is available for immediate download. MIT licensed, includes 64-bit RISC-V, NXP Cortex-M33 demo & Nuvoton Cortex-M23 demo.

NXP tweet showing LPC5500 (ARMv8-M Cortex-M33) running FreeRTOS.

View a recording of the "OTA Update Security and Reliability" webinar, presented by TI and AWS.


Careers

FreeRTOS and other embedded software careers at AWS.



FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

Cadence Tensilica Cortes

Espressif ESP32

IAR Partner

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

Mediatek

Renesas

RISC-V

SiFIve RISC-V

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

OpenRTOS and SafeRTOS

Xilinx Microblaze and Zynq partner