Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT




Loading

Queue crash problem

Posted by Alan on October 27, 2010
I am developing a project using an STM32 ARM processor and FreeRTOS. there are 3 USARTS, and the RX ISR for each gives a semphore which is taken by task vUSART_RXcharReceived:


void vUSART_RXcharReceived(void* pvParameters){
portTickTypeTimeOut;
u8Port;

TimeOut = portMAX_DELAY;
Port = *(u8*)pvParameters;
vSemaphoreCreateBinary(prvHIO.USART_RXcharReceived[Port]);
//Make sure the semaphore is cleared initially
xSemaphoreTake(prvHIO.USART_RXcharReceived[Port], 0);

while (1){
if (xSemaphoreTake(prvHIO.USART_RXcharReceived[Port], TimeOut) == pdPASS){//If we received a char...
TimeOut = 35000/SysConfig.COMport[Port].BaudRate;//Time for 3.5 chars
if (TimeOut < 2)
TimeOut = 2;
}else{//End of message
TimeOut = portMAX_DELAY;
xQueueSend(HIO.CheckMessage, &Port, portMAX_DELAY);
}
}
}


There are three insances of this task, one for each USART, and as you see they wait for a break in transmission then add an item to queue HIO.CheckMessage to identify which port now contains a message.

The queue is received by vCheckMessage:


void vCheckMessage(void* pvParameters){
s16MsgLen, Response = DONE;
u8DiagnosticsDisplayed;
u8Port;

HIO.CheckMessage = xQueueCreate(15, sizeof(u8));

while (1){
xQueueReceive(HIO.CheckMessage, &Port, portMAX_DELAY);
|
|
|
}
}


I am testing it using traffic on only one USART. The programme works correctly for the first six messages, then fails, giving a Hard Fault exception, which means it has tried to access an invalid memory address. The fault always occurs as the code is running xQueueSend(HIO.CheckMessage, &Port, portMAX_DELAY) in vUSART_RXcharReceived((). I have pinpointed to error to task vListRemove( ) in FreeRTOS list.c. This copies a value to a pointer

pxList = ( xList * ) pxItemToRemove->pvContainer;

However, when the system fails it is because the value of this pointer is 0 (NULL), so when it is subsequently used the code attempts to write to Flash code space.

I obviously cannot change the code within FreeRTOS, yet I can't see anything I am doing wrong to cause the problem. When the failure occurs I can look at the memory location of pxItemToRemove and see that all elements appear to hold sensiboe values except pvContainer, which is 0. I can't put a watch on it to trap when it is set to 0, since it is constantly being updated when the code is running.

Any suggestions?

Regards
Alan Rouse

RE: Queue crash problem

Posted by Richard Damon on October 27, 2010
While I am not familiar with this particular port, the issue sounds like what can happen if the interrupt isn't setup right and doesn't respect the Critical Section code. The key here is that the priority of the interrupt needs to be set properly. If it can interrupt during a Critical Section or causes unexpected interrupt nesting then the interrupts can corrupt the data structures.

The other possibility is that an interrupt is using a non FromISR routine which can also cause this sort of issue.

RE: Queue crash problem

Posted by Alan on October 27, 2010
Thanks for your ideas. I have already repeatedly checked to confirm that all semaphore and queue accesses from ISR's use the correct call format. However, the queue that is failing has no connection with interrupts. It is simply passing a queue from one task to another.

Regarding your first thought, do you mean the FreeRTOS priorities or the ARM priorities? On the ARM the Systick intterupt is higehst (0) and the USART interrupts are lowest (4). On FreeRTOS the USART task priority is set to 3, so it is higher than the CheckMessage() task, which is set to priority 2. I believe that taskENTER_CRITICAL() disables all interrupts, so I can't understand why there should be any problem if an interrupt occurs during critical code sections.

Since posting the problem I checked which version of FreeRTOS is being used and found it is V5.4.2. I have now downloaded the latest version, 6.1.0 in the hope that it may fix the problem. However, I can't yet get it to work: vTaskStartScheduler() calls vPortStartFirstTask(), which comprises a small number of assembler instructions. When it executes the last instruction, svc 0, the processor jumps to the default handler exception. If I revert back to FreeRTOS V5.4.2 the programme starts properly, but exhibits the original problem. It would appear, therefore, that there is a compatibility problem between V5 and V6 of FreeRTOS.

RE: Queue crash problem

Posted by Richard on October 28, 2010
“Regarding your first thought, do you mean the FreeRTOS priorities or the ARM priorities? ”


I think richard_damon is referring to the interrupt priorities.

“On the ARM the Systick intterupt is higehst (0) and the USART interrupts are lowest (4). ”


This does not sound right and if so will cause the sort of problem you are seeing. The Systick would normally be the lowest (255, or 15, depending on how you want to define it). The UART interrupts can be above the (lower numeric values) but must have a lower or equal interrupt priority than whatever configMAX_SYSCALL_INTERRUPT_PRIORITY is set to if you are calling FreeRTOS API functions from the interrupt.

“ I have now downloaded the latest version, 6.1.0 in the hope that it may fix the problem.”


From what you have said, it looks like an interrupt priority problem. Changing FreeRTOS versions is not going to fix that.

“ When it executes the last instruction, svc 0, the processor jumps to the default handler exception”


Presumably therefore you have not installed an SVC handler. The way the kernel is started changed at some point, I cannot remember when but this will be documented in the change history (http://www.freertos.org/History.txt).


Please go to the documentation page for the port you are using on the FreeRTOS web site and read the "RTOS Port specific configuration" section that describes how to set up the interrupt priorities.


Regards.

RE: Queue crash problem

Posted by Alan on October 28, 2010
Richard

Thanks for your response. Since my original posting and richard_damon’s response I have looked closely into interrupt priorities. I was wrong in assuming the SysTick interrupt was set to priority 0. It is in fact 255 as you say. I have realised the following:

1.On ARM processors the priority level is 8-bits, divided into Pre-emption Priority (the most significant part) and Sub-Priority (the least significant part). NVIC_PriorityGroupConfig defines the split, between 7:1 and 0:8. An interrupt at a given Pre-emption Priority Level will pre-empt any in higher-numbered interrupt. Simultaneous interrupts at the same Pre-emption Priority Level are processed in Sub-Priority sequence (lowest first), but I can see no purpose in Sub-Priority level.

2.The STM32 only implements 16 Pre-emption Priority levels so expects the NVIC_PriorityGroupConfig to define the split as 4:4 (i.e. the most significant 4 bits define Pre-emption Priority and the least significant 4 bits define Sub-Priority). However, it doesn’t use the latter, since whatever value is written the value read back is 0.

3.The FreeRTOS definitions for configKERNEL_INTERRUPT_PRIORITY and configMAX_SYSCALL_INTERRUPT_PRIORITY must reflect this structure, so should bet set to 0xN0 where N is the required level.

4.FreeRTOS disables interrupts by copying configMAX_SYSCALL_INTERRUPT_PRIORITY to its BASEPRI register. This register is not very well documented by either ARM or ST, but if set to any value other than 0 it disables only interrupts set to higher Pre-emption Priority levels.

Bearing this in mind I have set configMAX_SYSCALL_INTERRUPT_PRIORITY to 0x30 and configKERNEL_INTERRUPT_PRIORITY to 0xFF. I have then set all the interrupts I use to values greater than 0x30. I can confirm the settings by examining the relevant registers within the NVIC block.

However, I still get the same problem, namely that the system crashes with a Hard Fault on the 7th message received via a USART. pxCurrentTCB shows the failure at the vUSART_RXcharReceived task. I have again confirmed that the fault is due to pxList being set to 0. I cannot see what is wrong with my code to cause this problem, so I am still looking for ideas.

Regarding FreeRTOS V6:

FreeRTOS v5.4.2/Source/portable/GCC/ARM_CM3/port.c includes
#define vPortSVCHandler SVC_Handler
#define xPortPendSVHandler PendSV_Handler
#define xPortSysTickHandler SysTick_Handler
followed by definitions of the functions.

However, FreeRTOS v6.1.0/Source/portable/GCC/ARM_CM3/port.c defines the functions but doesn’t include the #defines. If I copy them from the older version the system works. However, it exhibits the same problem with pxList, as expected.

Regards

RE: Queue crash problem

Posted by Alan on October 29, 2010
Update:

I have now modified my code in a way that I hoped would overcome the problem, but it now manifests itself in a different way:

I have added diagnostics functions SetGPIO() and ClearGPIO() to force digital states on spare CPU output pins that can be monitored on an oscilloscope. I have also implemented re-triggerable timers in the USART receive interrupt routines to eliminate the need for intermediate task vUSART_RXcharReceived(). When a timer lapses it generates an interrupt which adds the port number to the CheckMessage queue:


voidTIM2_IRQHandler(void){
portBASE_TYPE xHigherPriorityTaskWokenByPost;
u8Port;

ClearGPIO(TP1);
|
|
Port = COM3;
xQueueSendFromISR(HIO.CheckMessage, &Port, &xHigherPriorityTaskWokenByPost);
}


Port TP1 is set every time a character is received and cleared when the message is added to the CheckMessage queue;

I have also added diagnostics in vCheckMessage():


void vCheckMessage(void* pvParameters){
u8Port;

HIO.CheckMessage = xQueueCreate(15, sizeof(u8));

while (1){
ClearGPIO(TP2);
prvHIO.MessagesInQueue = uxQueueMessagesWaiting(HIO.CheckMessage);
xQueueReceive(HIO.CheckMessage, &Port, portMAX_DELAY);
SetGPIO(TP2);

prvHIO.NoOfMessages++;
|
|
}
}


I should therefore be able to see on the oscilloscope a pulse on TP1 when a message is received and a pulse on TP2 when it is processed. Furthermore, NoOfMessages and MessagesInQueue allow me to check how many messages have been processed and the number waiting in the queue.

The diagnostics show that after correctly processing 5 messages vCheckMessage() remains permanently blocked. TIM2_IRQHandler() continues to receive messages but can’t add them to the queue because it is full.

Can anyone give me any ideas?

RE: Queue crash problem

Posted by Alan on November 1, 2010
Cracked it!

I eventually found the problem:

Under certain circumstances my CheckMessage() routine could execute a FreeRTOS xTaskResumeAll() without first executing vTaskSuspendAll(). The effect of this is apparently to corrupt message handling.

RE: Queue crash problem

Posted by Jesper Matthiesen on November 9, 2010
so how were you able to create such a problem by just using the FreeRTOS API functions? Looking at your checkmessage function, I would say that if it doesn't work, it indicates a bug in FreeRTOS.

how did you solve the problem, exactly?

thanks.

RE: Queue crash problem

Posted by Alan on November 9, 2010
Some messages processed by CheckMessage() modify global data using read-modify-write so I need to ensure to other tasks can interrupt them. I therefore use vTaskSuspendAll() within an if{} statment, carry out some common functionality, then include xTaskResumeAll().within another if{} statement. Unfortunately the if{} statements were very slightly different, so some messages did not satisfy the first but did satisfy the second. Hence these messages caused xTaskResumeAll().to be executed without a preceding vTaskSuspendAll(), even though CheckMessage() exectued without any apparent errors.

I tracked down the problem by dogged persistence! I was looking for problems in queue handling so it took me a long time to realise that this can be affected by other API tasks.


[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.

Latest News

FreeRTOS kernel V10 is available for immediate download. Now MIT licensed.


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

IAR Partner

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

OpenRTOS and SafeRTOS