Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT




Loading

rtos fails to withstand longtern test

Posted by eprogrammer on September 24, 2008
Hello,

I am using FreeRTOS with Cortex M3 core. The system has 3 interrupt and one task. All the interrupts set a semaphore to wake up this task. One interrupt has higher preemption priority than other two. this means it can interrupt other two interrupts.
The program runs great for hours!!!
Then all of a sudden it stop working normally. The ISRs set the semaphore but the task is not waken up. Thus semaphore remains always set.
Findings:
1. it was found that the semaphore given from ISR routine expected that the routine could never be interrupted as it was called from ISR. But this is false in case of cortex-m3. It could be preempted. So i disabled the interrupt for this routine.

2. It was found then that the task which was supposed to wakeup is already in the readyList but the topReadyListPriority was somehow turned to 0. As the context switching routine doesnt scan through whole radyList but just wakeup the first task it finds in readyList having the priority of topReadyListPriority, our task is never been called. To overcome this problem i set the topReadyListPriority to be highestTaskPriority so that contextswitch routine scans the whole ready list. In other word i remove the optimization and so whole readyList is always scaned through to find the highest priority task.
Now my program runs for more hours. but still there is problem.

3. It was then found that the task is not more in readyList and the semaphore to which it blocks is already set.

I have inserted lots of assertions in the freeRTOS source files but couldnt find anything.

ANY SUGGESTION WILL BE HIGHLY APPRECIATED


RE: rtos fails to withstand longtern test

Posted by Richard on September 24, 2008
> I am using FreeRTOS with Cortex M3 core. The system has 3
> interrupt and one task. All the interrupts set a semaphore to
> wake up this task.

What type of semaphore is being used?

> One interrupt has higher preemption
> priority than other two. this means it can interrupt other
> two interrupts.

Can you let us know:

+ What configKERNEL_INTERRUPT_PRIORITY is set to.
+ What configMAX_SYSCALL_INTERRUPT_PRIORITY is set to.
+ What priority your interrupt are running at.


Also, have you checked that you are not overflowing a task stack?

> The program runs great for hours!!!
> Then all of a sudden it stop working normally. The ISRs set
> the semaphore but the task is not waken up. Thus semaphore
> remains always set.
> Findings:
> 1. it was found that the semaphore given from ISR routine
> expected that the routine could never be interrupted as it
> was called from ISR.


The semaphore give from ISR function ***DOES*** include protection against being preempted by another task using the same semaphore.

> But this is false in case of cortex-m3.
> It could be preempted. So i disabled the interrupt for this routine.


As long as you have set the priorities correctly as per my questions above it is perfectly fine for the interrupt to be preempted.



> 2. It was found then that the task which was supposed to
> wakeup is already in the readyList but the
> topReadyListPriority was somehow turned to 0. As the context
> switching routine doesnt scan through whole radyList but just
> wakeup the first task it finds in readyList having the
> priority of topReadyListPriority, our task is never been
> called. To overcome this problem i set the
> topReadyListPriority to be highestTaskPriority so that
> contextswitch routine scans the whole ready list.


This is just masking a problem that exists elsewhere.

Regards.


RE: rtos fails to withstand longtern test

Posted by Dave on September 24, 2008
Which version of FreeRTOS are you using?

RE: rtos fails to withstand longtern test

Posted by eprogrammer on September 25, 2008
Thanks for quick reply.

1) FreeRTOS version - 5.0.2

2) Semaphore used: xSemaphoreGiveFromISR(ProtocolMainLoopSemaphore, &xHigherPriorityTaskWoken). The xHigherPriorityTaskWoken is set false before this call.

3) + configKERNEL_INTERRUPT_PRIORITY is set to: 255
+ configLIBRARY_KERNEL_INTERRUPT_PRIORITY is set to 15
BUT...
+ configMAX_SYSCALL_INTERRUPT_PRIORITY is set NOT set. This means the SVCall has the default priority of 0(highest). But i thought that it is ment to be like this as with the cortex-m3 demoRTOS also it is like this.
+ priorities my interrupts running at:
Three interrupts which set the semaphore have priorities: 144, 160 and 16. Here interrupt with prio 16 can interrupt other two. There is one more timer interrupt with priority 0. But it doesnot set any semaphore.

4) task overflow is checked. And overflow is far far away. No posibility.

5) semaphore give from ISR function::::
----------------------------------------------------------------------------------------------------------------------------
signed portBASE_TYPE xQueueGenericSendFromISR( xQueueHandle pxQueue, const void * const pvItemToQueue, signed portBASE_TYPE *pxHigherPriorityTaskWoken, portBASE_TYPE xCopyPosition )
{
signed portBASE_TYPE xReturn;
unsigned portBASE_TYPE uxSavedInterruptStatus;

/* Similar to xQueueGenericSend, except we don't block if there is no room
in the queue. Also we don't directly wake a task that was blocked on a
queue read, instead we return a flag to say whether a context switch is
required or not (i.e. has a task with a higher priority than us been woken
by thispost). */
uxSavedInterruptStatus = portSET_INTERRUPT_MASK_FROM_ISR();
{
if( pxQueue->uxMessagesWaiting < pxQueue->uxLength )
{
traceQUEUE_SEND_FROM_ISR( pxQueue );

prvCopyDataToQueue( pxQueue, pvItemToQueue, xCopyPosition );

/* If the queue is locked we do not alter the event list. This will
be done when the queue is unlocked later. */
if( pxQueue->xTxLock == queueUNLOCKED )
{
if( !listLIST_IS_EMPTY( &( pxQueue->xTasksWaitingToReceive ) ) )
{
u32 interrupt_base = NVIC_GetBASEPRI();
assert((interrupt_base==0xf0));
if( xTaskRemoveFromEventList( &( pxQueue->xTasksWaitingToReceive ) ) != pdFALSE )
{
/* The task waiting has a higher priority so record that a
contextswitch is required. */
*pxHigherPriorityTaskWoken = pdTRUE;
}
}
}
else
{
/* Increment the lock count so the task that unlocks the queue
knows that data was posted while it was locked. */
++( pxQueue->xTxLock );
}

xReturn = pdPASS;
}
else
{
traceQUEUE_SEND_FROM_ISR_FAILED( pxQueue );
xReturn = errQUEUE_FULL;
}
}
portCLEAR_INTERRUPT_MASK_FROM_ISR( uxSavedInterruptStatus );
return xReturn;
}
----------------------------------------------------------------------------------------------------------------------------------
Here portSET_INTERRUPT_MASK_FROM_ISR() is not defined in my program. So essentially higher priority interrupt may interrupt this routine if it is first called by lower priority interrupt. Please correct me if im telling something wrong.


Very Excited for the replies :)

RE: rtos fails to withstand longtern test

Posted by Richard on September 25, 2008
> 1) FreeRTOS version - 5.0.2

Ok - that explains something. The configMAX_SYSCALL_INTERRUPT_PRIORITY parameter was not added to the CM3 port until V5.0.3.



> 2) Semaphore used: xSemaphoreGiveFromISR(ProtocolMainLoopSemaphore,
> &xHigherPriorityTaskWoken). The xHigherPriorityTaskWoken is
> set false before this call.


I was actually meaning, is it binary, counting, recursive or mutex, but it probably doesn't actually matter come to think of it.


> 3) + configKERNEL_INTERRUPT_PRIORITY is set to: 255
> + configLIBRARY_KERNEL_INTERRUPT_PRIORITY is set to 15 BUT...
> + configMAX_SYSCALL_INTERRUPT_PRIORITY is set NOT set.


I find the way ST define the interrupt priorities very confusing. You should take a look at the code (in the ST library) that converts the 15 to a Cortex M3 priority, if you can work out what the hell it is doing let me know!


> This means the SVCall has the default priority of 0(highest).

SVCall is only used once, to start the scheduler running. After that context switches are set by PendSV, which does the same thing just not necessarily synchronously. The PendSV priority is set to configKERNEL_INTERRUPT_PRIORITY by the kernel itself.


> But i thought that it is ment to be like this as with the
> cortex-m3 demoRTOS also it is like this.
> + priorities my interrupts running at:
> Three interrupts which set the semaphore have
> priorities: 144, 160 and 16. Here interrupt with prio 16 can
> interrupt other two. There is one more timer interrupt with
> priority 0. But it doesnot set any semaphore.

This is definitely where your problem lies. I didn't realise you were using V5.0.2 in my other reply. In this version anything interrupt that makes a system call must use the same priority as the kernel (255). If you use the latest versions then you have must more flexibility as described here: http://www.freertos.org/a00110.html#kernel_priority . It should be simple enough to upgrade, just drop in the newer files and set configMAX_SYSCALL_INTERRUPT_PRIORITY to something appropriate.

I can't remember how many bits of priority the STM32 has. The Stellaris CM3 has 3 bits, which are the top three bits, so I do something like this (this is probably different for the STM32).

#define configKERNEL_INTERRUPT_PRIORITY ( 7 << 5 ) /* Priority 7, or 255 as only the
top three bits are implemented. This
is the lowest priority. */

#define configMAX_SYSCALL_INTERRUPT_PRIORITY ( 5 << 5 ) /* Priority 5, or 160. */

Then interrupts that make system calls can use priorities ( 5 << 5 ), ( 6 << 5 ) and ( 7 << 5 ).

Regards.

RE: rtos fails to withstand longtern test

Posted by eprogrammer on September 25, 2008
Thank Richard,

That was really quick and enlightening.

I will give you the result after i upgrade my project to new version.

Thnaks again



[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.

Latest News

FreeRTOS kernel V10 is available for immediate download. Now MIT licensed.


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

IAR Partner

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

OpenRTOS and SafeRTOS