Trying to figure out why this happens, according to the comments in the code,:
/* The following assertion will fail if a service routine (ISR) for
an interrupt that has been assigned a priority above
configMAXSYSCALLINTERRUPTPRIORITY calls an ISR safe FreeRTOS API
function. ISR safe FreeRTOS API functions must only be called
from interrupts that have been assigned a priority at or below
so I changed this HALNVICSetPriority to configMAXSYSCALLINTERRUPTPRIORITY, but it still fail.
This code is part of USB host, for STM32F4, code is the library by ST.
HALNVICSetPriority(OTGFSIRQn, configMAXSYSCALLINTERRUPTPRIORITY, 0);
Any ideas and tips how to debug this will help.
configMAXSYSCALLINTERRUPTPRIORITY is probably a full 8 bit value, whereas NVICSetPriority() requires a shifted priority. I know this is complex but it is how the Cortex-M works. I tried explaining it on this page: http://www.freertos.org/RTOS-Cortex-M3-M4.html
Most newer projects have a conversion in the FreeRTOSConfig.h file, something like the following:
/* Cortex-M specific definitions. */
/* __BVIC_PRIO_BITS will be specified when CMSIS is being used. */
#define configPRIO_BITS __NVIC_PRIO_BITS
#define configPRIO_BITS 4 /* 15 priority levels */
/* The lowest interrupt priority that can be used in a call to a "set priority"
#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY 0xf
/* The highest interrupt priority that can be used by any interrupt service
routine that makes calls to interrupt safe FreeRTOS API functions. DO NOT CALL
INTERRUPT SAFE FREERTOS API FUNCTIONS FROM ANY INTERRUPT THAT HAS A HIGHER
PRIORITY THAN THIS! (higher priorities are lower numeric values. */
#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 5
/* Interrupt priorities used by the kernel port layer itself. These are generic
to all Cortex-M ports, and do not rely on any particular library functions. */
#define configKERNEL_INTERRUPT_PRIORITY ( configLIBRARY_LOWEST_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )
/* !!!! configMAX_SYSCALL_INTERRUPT_PRIORITY must not be set to zero !!!!
See http://www.FreeRTOS.org/RTOS-Cortex-M3-M4.html. */
#define configMAX_SYSCALL_INTERRUPT_PRIORITY ( configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )
So in that case configLIBRARYLOWESTINTERRUPTPRIORITY is set by hand as a shifted value (and therefore easier to understand for a human) and configKERNELINTERRUPTPRIORITY is then generated automatically. Likewise configLIBRARYMAXSYSCALLINTERRUPTPRIORITY and configMAXSYSCALLINTERRUPTPRIORITY.
The 'LIBRARY' prefix denoting that it is the version to be used in library function calls. So in your code you want to use configLIBRARYMAXSYSCALLINTERRUPTPRIORITY to prevent the assert triggering.
I read the page at http://www.freertos.org/RTOS-Cortex-M3-M4.html as it is in the comment above the assert line, but did not understand that I need to use configLIBRARYLOWESTINTERRUPT_PRIORITY.
This solved the problem.