Strange lockup with queues
Today, I’ve experienced a strange behaviour with FreeRTOS on STM32, using the SVN version and similar in 5.2.0 and 5.1.2.
I’ve added an ISR handler on USART3 receiving characters from a Zigbee module and sending them into a queue. Oddly enough, the system seems to loop indefinitely in vListInsert when queues are heavily used:
#0 0x20001d2a in vListInsert (pxList=0x20003c94, pxNewListItem=0x20002ab0) at ../../Source/list.c:135
#1 0x200018f0 in vTaskPlaceOnEventList (pxEventList=0x20003c94, xTicksToWait=4294967295) at ../../Source/tasks.c:1435
#2 0x20001226 in xQueueGenericReceive (pxQueue=0x20003c70, pvBuffer=0x20002cab, xTicksToWait=4294967295, xJustPeeking=0) at ../../Source/queue.c:925
#3 0x20000b56 in zigbee_task (pvParameters=<value optimized out>) at adc.c:31
#4 0x00000000 in ?? ()
(gdb) fr 0
#0 0x20001d2a in vListInsert (pxList=0x20003c94, pxNewListItem=0x20002ab0)
at ../../Source/list.c:135
135 for( pxIterator = ( xListItem * ) &( pxList->xListEnd ); pxIterator->pxNext->xItemValue <= xValueOfInsertion; pxIterator = pxIterator->pxNext )
(gdb) p *pxIterator
$1 = {xItemValue = 3, pxNext = 0x20002ab0, pxPrevious = 0x20002ab0,
pvOwner = 0x20002a98, pvContainer = 0x20003c94}
As you see, pxIterator next and previous fields points onto itself, while its xItemValue is 3, which means that it’s not the latest item in the list (which should have an xItemValue of portMAX_DELAY, i.e. 0xffffffff). Of course, it loops indefinitely here (lines number may be a little different than from the original because of my added checks in an attempt to isolate this problem).
The ISR handler produces characters at a rate of approximately 1000 cps, and the consumer task consumes them as soon as possible.
For the context, there are no memory allocation after the scheduler has started, and all the previous object allocations have been checked. Nothing fancy here, only an annoying lockup. The more tasks there are in the system, the faster the lockup happens.
If you have any idea of what may happen here, I would be interested.
Strange lockup with queues
Btw, there is something else I might add. The two queues exhibiting problems are queues with multiple writers and a single reader:
– a queue of "short", whose access are protected by a binary semaphore (characters to write to a LCD display) ;
– a queue of "long", representing backlight information to use.
Those queues are never written into from the ISR handlers. However, the serial ISR handler is the sole poster to a third queue, and a task reads those characters, accumulate them, and then uses the other two tasks to drive the LCD, along with other tasks.
Strange lockup with queues
Make sure the interrupt priority is equal to or below configMAX_SYSCALL_INTERRUPT priority remembering that the Cortex M3 has 0 as the highest priority NOT the lowest priority.
Also make sure you are not calling any API functions that don’t end in FromISR from an ISR.
Strange lockup with queues
Thanks MEdwards.
I have the following in FreeRTOSConfig.h, that I got from one of the examples, and which looks right to me:
#define configKERNEL_INTERRUPT_PRIORITY 255
#define configMAX_SYSCALL_INTERRUPT_PRIORITY 191 /* equivalent to 0xa0, or priority 5. */
I have not changed the default priorities for NVIC interrupts, so the greatest priority level I use is 47 for EXTI15_10. It means that all the NVIC interrupts from my peripherals cannot be interrupted by the FreeRTOS kernel handlers (which is required I think so that the PendSV handler really switches tasks context, not stacked interrupt ones). And the only active one at the time of the lockup is USART3, whose interrupt level is 46.
Concerning the API functions I call from the ISR handlers, only xSemaphoreGiveFromISR, xQueueReceiveFromISR, xQueueSendFromISR and portEND_SWITCHING_ISR are called.
Strange lockup with queues
Also, a check in vListInsert shows that the element being inserted sometimes has a non-NULL pvContainer field, as if it already belonged to another list (when vListInsert is called from vTaskPlaceOnEventList). Since pvContainer is set to NULL in both vListInitialiseItem and vListRemove, this alone looks suspicious to me.
Strange lockup with queues
Just to be as precise as possible (sorry about the fragmented additional information, I just thought about it), I need to add that my tasks do have large enough stacks (128 32-bits words, same for the main [interrupt] stack, few basic local variables, no structs and arrays on the stack), and that the FreeRTOS stack checker (set at level 2) doesn’t trigger. And the behaviour is the same whether I compile with -O3 and with -Os. Same result also with GCC 4.4.0 (to be released in a few days) and GCC 4.3.2.
Strange lockup with queues
Ouch, of course the default priorities were much too high. I skipped the FreeRTOS page explaining exactly this.
The issue is moot, FreeRTOS works perfectly fine as usual :)
Strange lockup with queues
I have experienced a similar problem. Unfortunately I have not found the FreeRTOS page explaining exactly what to do. Can you point me in the right direction please? I have a strange lock-up problem that I can’t really isolate.
Thanks in advance for your help.
Strange lockup with queues
Philip: look at http://www.freertos.org/a00110.html#kernel_priority
There is note concerning the CORTEX M3.
Strange lockup with queues
I’m experiencing exactly the same problem.
The info in this thread helped me on the right track again.
Thanks!
The info in this thread helped me on the right track again.
Thanks!