Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Real time embedded FreeRTOS mailing list 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT Training




Loading

Windows port and prvProcessSimulatedInterrupts()

Posted by daskew on December 4, 2015

We're using the windows port to allow us to run a firmware code base in an emulated hardware environment. While running, I've what I believe to be an issue with prvProcessSimulatedInterrupts(). It appears that this function allows interrupt handlers to execute at the same time as FreeRTOS tasks which can lead to unpredictable behavior in some circustances.

Our emulated environment layer uses windows resources to emulate a UART, GPIO and an RTC. These emulated peripherals use the vPortGenerateSimulatedInterrupt() API whenever they need to cause our firmware code base to see characters, GPIO interrupts, or clock ticks.

Occasionally we've seen some configASSERTs inside freeRTOS, usually while freeRTOS is removing an item from a list due to a list being unexpectedly empty. We've also seen Visual Studio report first chance exceptions while processing interrupts.

While investigating these problems, I noticed that the prvProcessSimulatedInterrupts() implementation in the windows port allows a task to continue executing while interrupt handlers are also called. prvProcessSimulatedInterrupts() only halts the current task from executing if a context switch is necessary after running the interrupt handler.

To better emulate single-core processors, shouldn't prvProcessSimulatedInterrupts() suspend the running task prior to calling interrupt handlers? This appears to be especially important if the interrupt handlers are using the fromISR variants of freeRTOS APIs.

I inserted a call to SuspendThread() prior to calling all interrupt handlers, and followed with up with: pxThreadState = (xThreadState )((size_t *)pxCurrentTCB); ResumeThread(pxThreadState->pvThread);

This seems to have helped with our stability issues. I haven't yet further investigated the source of the configASSERT() and windows exceptions that our debugger was catching.

This code change seems to better emaulate a single core CPU.

Thoughts?

-Dave


Windows port and prvProcessSimulatedInterrupts()

Posted by rtel on December 4, 2015

I will have to re-familiarise myself with how this functions. Please attach your modified version to a forum post so I can see your changed.

Normally (on real hardware) interrupts execute outside the context of a task. Critical sections are used to avoid the two separate threads of execution (the task and the interrupt) from accessing critical RTOS data at the same time. The critical section effectively holds a subset of interrupts off until it is safe for them to execute.

As I recall, in the windows port, the function that executes the simulated interrupts executes in its own Windows (not FreeRTOS) thread - in fact it is the main thread which just runs into a for( ;; ) loop that blocks on Windows semaphores to wait for interrupts to process. In that case it is analogous to the real hardware scenario. Critical sections prevent (or should prevent) the Windows thread that processes simulated interrupts from leaving the Blocked state.

We know from experience that all the Windows threads that are FreeRTOS tasks have to run on the same core. I'm wondering if there is anything that forces the thread that processes simulated interrupts to also be on the same core - that is probably very important and needs checking.

Also, which FreeRTOS version are you using, as the following code was added into the process simulated interrupts function quite recently:

~~~~ /* Ensure the thread is actually suspended by performing a synchronous operation that can only complete when the thread is actually suspended. The below code asks for dummy register data. */ xContext.ContextFlags = CONTEXT_INTEGER; ( void ) GetThreadContext( pxThreadState->pvThread, &xContext ); ~~~~


Windows port and prvProcessSimulatedInterrupts()

Posted by rtel on December 4, 2015

... Should have looked first - the main thread does get its afinity set.


Windows port and prvProcessSimulatedInterrupts()

Posted by daskew on December 4, 2015

Looks like we were running 8.2.1. I've just updated to 8.2.3, so I now have the code change you referenced above.

I do understand your comments that processor affinity along with critical sections should protect FreeRTOS data structures. However, it looks like the FromISR versions of FreeRTOS functions do not enter a critical section.

The way my project is using FreeRTOS may be leading to issues. I have created a very simple OS layer in my product. One example of an OS function is a function called osQueuePost() that posts to a queue. This function performs a check to determine if it is being called from an ISR or not and calls either xQueueSendFromISR() or xQueueSend() depending upon context.

The way this is implemented on an ARM processor, is a register is read to determine whether we are within an ISR or not. On our windows port, we have a global variable that gets set indicating whether an ISR is executing or not.

Here is one theory about what may be going on:

  1. Application task is running
  2. Interrupt comes in and the interrupt handler sets the IN_ISR global variable
  3. Interrupt handler then calls osQueuePost() which calls xQueueSendFromISR()
  4. At the same time as this is executing, windows does a context switch and schedules the application task. The application task calls osQueuePost(). Because the global variable IN_ISR is set to true, the xQueueSendFromISR() API is called.
  5. At this point, there are two instances of xQueueSendFromISR() running at the same time.

My osQueuePost() function assumes that we are running a single core processor and that if an ISR is running, application code will not be running as we only have one CPU execution unit. The ISR will set the INISR global variable run to completion, then clear the INISR global. No matter what the application task is doing at this time when it is interrupted, it will call the appropriate FreeRTOS API.

To prevent this, it made sense to me to modify the Windows FreeRTOS port to do the following in the for loop inside :

~~~~~~ for(;;) { DWORD result = WaitForMultipleObjects( sizeof( pvObjectList ) / sizeof( void * ), pvObjectList, TRUE, INFINITE );

	/* Used to indicate whether the simulated interrupt processing has
	necessitated a context switch to another task/thread. */
	ulSwitchRequired = pdFALSE;

	/* Suspend the running thread. */
	pxThreadState = (xThreadState *)*((size_t *)pxCurrentTCB);
	SuspendThread(pxThreadState->pvThread);

	/* Ensure the thread is actually suspended by performing a
	synchronous operation that can only complete when the thread is
	actually suspended.  The below code asks for dummy register
	data. */
	xContext.ContextFlags = CONTEXT_INTEGER;
	(void)GetThreadContext(pxThreadState->pvThread, &xContext);

	/* For each interrupt we are interested in processing, each of which is
	represented by a bit in the 32bit ulPendingInterrupts variable. */
	for( i = 0; i < portMAX_INTERRUPTS; i++ )
	{
		/* Is the simulated interrupt pending? */
		if( ulPendingInterrupts & ( 1UL << i ) )
		{
			/* Is a handler installed? */
			if( ulIsrHandler[ i ] != NULL )
			{
				/* Run the actual handler. */
				if( ulIsrHandler[ i ]() != pdFALSE )
				{
					ulSwitchRequired |= ( 1 << i );
				}
			}

			/* Clear the interrupt pending bit. */
			ulPendingInterrupts &= ~( 1UL << i );
		}
	}

	if( ulSwitchRequired != pdFALSE )
	{
		/* Select the next task to run. */
		vTaskSwitchContext();
	}

	/* Obtain the state of the task now selected to enter the
	Running state. */
	pxThreadState = (xThreadState *)(*(size_t *)pxCurrentTCB);
	ResumeThread(pxThreadState->pvThread);

	ReleaseMutex( pvInterruptEventMutex );
}

~~~~~~

Here is my osQueuePost function:

~~~~~~ COMMONSTATUS osQueuePost( OSQueueHandlet handle, void *item, uint32t timeoutMsecs ) { COMMONSTATUS status = COMMONSTATUSERROR;

ASSERT(handle != NULL);
ASSERT(item != NULL);

if ((handle != NULL) && (item != NULL))
{
	// FreeRTOS uses different interfaces to post a queue item if you are in an interrupt
	if (bspCPUInHandler())
	{
		BaseType_t yieldRequired;

		// In an interrupt context
		BaseType_t osResult = xQueueSendFromISR((QueueHandle_t)handle, item, &yieldRequired);

		if (osResult == pdTRUE)
		{
			status = COMMON_STATUS_OK;
		}
		else
		{
			status = COMMON_STATUS_BUSY;
		}

		osSetYield(yieldRequired);
	}
	else
	{
		// Not in an interrupt context
		do {
			TickType_t ticksToWait;
			if (timeoutMsecs != OS_WAIT_FOREVER)
			{
				ticksToWait = timeoutMsecs / portTICK_PERIOD_MS;
			}
			else
			{
				ticksToWait = portMAX_DELAY;
			}

			BaseType_t osResult = xQueueSend((QueueHandle_t)handle, item, ticksToWait);

			if (osResult == pdTRUE)
			{
				status = COMMON_STATUS_OK;
				break;
			}

			if (timeoutMsecs != OS_WAIT_FOREVER)
			{
				status = COMMON_STATUS_TIMEOUT;
				break;
			}
		} while (timeoutMsecs == OS_WAIT_FOREVER);
	}
}

ASSERT(status != COMMON_STATUS_ERROR);
return status;

} ~~~~~~


Windows port and prvProcessSimulatedInterrupts()

Posted by rtel on December 5, 2015

Critical sections are only required in interrupts if interrupts can nest, so ports that do not support interrupt nesting do not provide the "FromISR" versions of the critical section macros. In the case of the Windows port, all [simulated] interrupts run in the same [Windows] thread so they cannot nest.

The scenario you describe assumes the application task can start to execute while a [simulated] interrupt is executing, but that should not be possible. There is an emphasis on the 'should', because perhaps that is what is happening.

The rational for saying it should not be possible is that the Windows thread that executes the interrupts is given a priority of THREADPRIORITYNORMAL, while the Windows threads that run FreeRTOS tasks are given a priority of THREADPRIORITYIDLE.

Another question - are you making any Windows system calls from any of your FreeRTOS tasks or interrupts? In particular, are you writing to the console (printf), a disk, or using the Windows TCP/IP stack? You can get away with these things occasionally (as we do in the demo application for the Windows port), but because such system calls can block the Windows threads in a way that is outside of the control of the FreeRTOS scheduler (which thinks it knows what is running when) they can causes crashes under load. We have some techniques for getting around this by using thread safe circular buffers to communicate between Windows threads that run FreeRTOS tasks and Windows threads to which IO operations are delegated.

Now I will look at the code you posted...

Regards.


Windows port and prvProcessSimulatedInterrupts()

Posted by daskew on December 7, 2015

I am not particularly familar with the Windows scheduler so I didn't realize that a THREADPRIORITYNORMAL thread would always be scheduled by Windows to run instead of THREADPRIORITYIDLE. The Windows port is now making more sense!

However, we are making Windows system calls from FreeRTOS tasks, and also access a windows mutex inside interrupts. I can see how such calls would break the assumptions you have spelled out above. I'll look a little bit closer at using thread safe circular buffers to queue up our I/O operations.


Windows port and prvProcessSimulatedInterrupts()

Posted by rtel on December 7, 2015

Perhaps too complex for your case, but attached is a file that we use in the Windows version of the FreeRTOS+TCP demo application. The stream buffer (the code for which is also in the FreeRTOS Labs download) is the thread safe circular buffer.

In the demo you can send debug and application log messages to UDP (I think?), the console, or a disk files. The FreeRTOS threads call the logging function, and the logging function writes to the stream buffer from the context of the task. A separate Windows thread that is unknown to the FreeRTOS scheduler then reads from the stream buffer and peforms the necessary IO.

Like I said, probably too complex for your scenario, but hopefully you will get the idea.

Attachments

demo_logging.c (20804 bytes)

Windows port and prvProcessSimulatedInterrupts()

Posted by daskew on December 9, 2015

We're adapting some of the ideas in demo_logging.c to our code base.

I am wondering why this uses Windows thread priorities as a critical section. Based upon my code reading it appears to me that vLoggingPrintf() will only be called from FreeRTOS tasks. If that is the case, why not use taskENTERCRITICAL()/taskEXITCRITICAL() to protect the buffer used by vLoggingPrintf?


Windows port and prvProcessSimulatedInterrupts()

Posted by rtel on December 10, 2015

I presume you are referring to this code:

~~~~ /* First write in the length of the data, then write in the data itself. Raising the thread priority is used as a critical section as there are potentially multiple writers. The stream buffer is only thread safe when there is a single writer (likewise for reading from the buffer). */ SetThreadPriority( GetCurrentThread(), THREADPRIORITYNORMAL ); lStreamBufferAdd( xLogStreamBuffer, 0, ( const uint8t * ) &( xLength ), sizeof( xLength ) ); lStreamBufferAdd( xLogStreamBuffer, 0, ( const uint8t * ) cOutputString, xLength ); SetThreadPriority( GetCurrentThread(), THREADPRIORITYIDLE ); ~~~~

...the answer is I'm not sure, but think perhaps it is because in that particular demo there are also other non FreeRTOS Windows threads that, for test purposes, are communicating with the FreeRTOS+TCP TCP/IP stack (so the FreeRTOS tack can communicate with a third party stack within the same test application) and those non FreeRTOS Windows threads might use the same logging function.


[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) 2004-2010 Richard Barry. Copyright (C) 2010-2016 Real Time Engineers Ltd.
Any and all data, files, source code, html content and documentation included in the FreeRTOSTM distribution or available on this site are the exclusive property of Real Time Engineers Ltd.. See the files license.txt (included in the distribution) and this copyright notice for more information. FreeRTOSTM and FreeRTOS.orgTM are trade marks of Real Time Engineers Ltd.

Latest News:

FreeRTOS V9.0.0 is now available for download.


Free TCP/IP and file system demos for the RTOS


Sponsored Links

⇓ Now With No Code Size Limit! ⇓
⇑ Free Download Without Registering ⇑


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

Renesas Electronics Gold Alliance RTOS Partner.jpg

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

Atmel RTOS partner supporting ARM Cortex-M3 and AVR32 microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Xilinx Microblaze and Zynq partner

Silicon Labs low power RTOS partner

Altera RTOS partner for Nios II and Cortex-A9 SoC

Freescale Alliance RTOS Member supporting ARM and ColdFire microcontrollers

Infineon ARM Cortex-M microcontrollers

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

Cypress RTOS partner supporting ARM Cortex-M3

Fujitsu RTOS partner supporting ARM Cortex-M3 and FM3

Microsemi (previously Actel) RTOS partner supporting ARM Cortex-M3

Atollic Partner

IAR Partner

Keil ARM Partner

Embedded Artists