FreeRTOS TCP/IP stack vs lwIP

HI, I have a working project based on FreeRTOS + lwIP. I noticed that FreeRTOS has it’s own native TCP/IP stack and be fore I go deeply into studding (and maybe porting) it I would like to get a brief information about advantages (or disadvantages) of FreeRTOS stack vs lwIP. In other words is it worth to pass to native stack or keep using lwIP? The main disadvantages of lwIP stack is luck of ZERO COPY operation and luck of dynamic scale-ability.
By last I mean that all memory allocation in lwIP are made static (in compilation time) and I can’t configure specific device without reprogramming it. I would like to know if this features are implemented in FreeRTOS stack. Thanks a lot.

FreeRTOS TCP/IP stack vs lwIP

FreeRTOS+TCP is designed from the ground up to have a standard and familiar interface – Berkeley sockets – and be thread safe. This is intended to make it as easy to use as possible. There are actually other interfaces which are faster than the Berkeley sockets interface (what we refer to as ‘expert’ interfaces), but these are not documented yet because we don’t want to confuse people or overwhelm them with information. They will be documented in time, and we have a few add-on components (also not released yet) which use the expert interfaces which will serve as a reference. With regards to zero copy:
  • UDP packets can be zero copy end to end if the MAC driver itself supports zero copy. That is you can fill a buffer in the application, send that buffer to the TCP/IP stack using the sockets interface (there is a special socket option on the sendto() function to say its a zero copy), have the buffer passed through the stack, and sent to the driver. The driver can then make one of its DMA descriptors point to the buffer for transmission – nothing is copied anywhere.
  • With TCP it is not so easy because TCP packets form streams. You could send three small TCP packets to a socket, and internally the TCP stack would add the three packets into a single Ethernet frame to minimise calls to the driver and minimise packets on the network. There is a way of zero copying TCP packets too, but likewise this is not documented yet as it is more complex than the UDP case because you end up with pointers to data that is within a stream – which means the user and the writer of the driver need to take great care.
FreeRTOS+TCP keeps buffer management in the portable layer, so you choose the implementation you want:
  • The simplest but least deterministic implementation is not to allocate any memory at all, and the stack will allocate and free memory from the FreeRTOS heap when it needs it. This is of course also the slower and least deterministic implementation, but great for simplicity and resource constrained systems. If you want to send a packet but there is not enough heap then the socket will block (assuming it is a blocking socket) until it can obtain the memory (or timeout as the case may be). To use this method you need to be using heap4.c or heap5.c as those two implementations guard against memory fragmentation.
  • The fastest implementation allocates the buffers up front. Fast and deterministic, but potentially consumes more memory, especially when only a subset of the buffers are in use at any one time.
Which chip do you want to port it to? It might be we have something you could start with. Regards.

FreeRTOS TCP/IP stack vs lwIP

Hi. Thanks a lot for your comprehensive reply. I’m working with STM32F2xx. I wrote my own netif driver for lwIP cause one provided by ST is buggy and limited. This make me thing that I have a good understanding of low level – ETH module, RMII and PHY functionality. Your information makes me believe that using native stack would be more efficient compared to lwIP which has to stay compatible with arbitrary any OS. What is the current status of your development? Is it a time to start looking inside or better to wait for official release? My project is commercial and it is already in production, we can not take chances. Currently lwIP suits us but as development is going further and new features are added I see that RAM usage becomes our big problem and we will need a more scalable solution with zero copy and dynamic configuration per specific customer. Thanks. Sent from my iPad On Nov 21, 2014, at 9:11 PM, “Real Time Engineers ltd.” rtel@users.sf.net wrote:
FreeRTOS+TCP is designed from the ground up to have a standard and familiar interface – Berkeley sockets – and be thread safe. This is intended to make it as easy to use as possible. There are actually other interfaces which are faster than the Berkeley sockets interface (what we refer to as ‘expert’ interfaces), but these are not documented yet because we don’t want to confuse people or overwhelm them with information. They will be documented in time, and we have a few add-on components (also not released yet) which use the expert interfaces which will serve as a reference. With regards to zero copy: UDP packets can be zero copy end to end if the MAC driver itself supports zero copy. That is you can fill a buffer in the application, send that buffer to the TCP/IP stack using the sockets interface (there is a special socket option on the sendto() function to say its a zero copy), have the buffer passed through the stack, and sent to the driver. The driver can then make one of its DMA descriptors point to the buffer for transmission – nothing is copied anywhere. With TCP it is not so easy because TCP packets form streams. You could send three small TCP packets to a socket, and internally the TCP stack would add the three packets into a single Ethernet frame to minimise calls to the driver and minimise packets on the network. There is a way of zero copying TCP packets too, but likewise this is not documented yet as it is more complex than the UDP case because you end up with pointers to data that is within a stream – which means the user and the writer of the driver need to take great care. FreeRTOS+TCP keeps buffer management in the portable layer, so you choose the implementation you want: The simplest but least deterministic implementation is not to allocate any memory at all, and the stack will allocate and free memory from the FreeRTOS heap when it needs it. This is of course also the slower and least deterministic implementation, but great for simplicity and resource constrained systems. If you want to send a packet but there is not enough heap then the socket will block (assuming it is a blocking socket) until it can obtain the memory (or timeout as the case may be). To use this method you need to be using heap4.c or heap5.c as those two implementations guard against memory fragmentation. The fastest implementation allocates the buffers up front. Fast and deterministic, but potentially consumes more memory, especially when only a subset of the buffers are in use at any one time. Which chip do you want to port it to? It might be we have something you could start with. Regards. FreeRTOS TCP/IP stack vs lwIP Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/freertos/discussion/382005/ To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

FreeRTOS TCP/IP stack vs lwIP

While I was typing the response below, Richard was quicker… But it still adds some information, it’s a bit more specific:
I would like to get a brief information about advantages (or disadvantages) of FreeRTOS stack vs lwIP. In other words is it worth to pass to native stack or keep using lwIP?
brief? FreeRTOS+TCP was created with the aim:
  • to get a close integration with FreeRTOS
  • simple to use, a small configuration file with few items, easy bug-tracking
  • fast, low-RAM, configurable memory foot-print
  • Compatible with ultra-low-power, tickless kernel
As Richard wrote, +TCP has two ways to allocate network buffers:
  • BufferAllocation_1.c : all buffers are allocated in advance
  • BufferAllocation_2.c : buffers are allocated using pvPortMalloc() only when needed
The amount of memory needed per TCP socket depends on the buffer-sizes that you assign. These sizes are configurable using sockopt’s, thus yes, they are dynamic. Once a TCP socket is connected, the buffer size is fixed, reserved, and can not be changed any more. The memory is freed when FreeRTOS_closesocket() is called. This guarantees quality of transmission: you will rarely see a TCP transmission fail half-way because malloc() was running low 🙂
The main disadvantages of lwIP stack is lack of ZERO COPY operation and lack of dynamic scale-ability.
I think lwIP does have a possibility for zerocopy if you use the call-back methods? +TCP also has call-back methods, for instance:
BaseType_t onTcpReceive (xSocket_t xSocket, void * pData, size_t xLength )
{
    /* Here pData is a direct pointer to the internal stream buffer. */
}
BaseType_t onUdpReceive (xSocket_t xSocket, void * pData, size_t xLength,
    const struct freertos_sockaddr *pxFrom,
    const struct freertos_sockaddr *pxDest )
{
    /* pData points to the message received. A reply can be sent here */
}
Such functions can be bound to a socket with sockopts. Please note that all call-back functions (also within lwIP) are risky because you’re writing code which is executed by the Network task. This task has a different and maybe limited stack, different rights / ownerships etc. When using TCP, there is a zero-copy possibility while receiving data, not yet documented:
/* The "zero-copy" method for TCP: */
xRc = FreeRTOS_recv( pxClient->xTransferSocket, ( void * ) &pcBuffer,
    0x20000u, FREERTOS_ZERO_COPY );
if( xRc <= 0 )
{
    break;
}
/* pcBuffer points to an internal stream buffer. 'xRc' bytes may be read. */
pxClient->ulRecvBytes += xRc;
/* Use the data, e.g. write them to a file */
/* Tell the driver the data ('xRc' bytes) may be flushed from the buffer. */
FreeRTOS_recv( pxClient->xTransferSocket, ( void * ) NULL, xRc, 0 );
The disadvantage of these call-backs of course is that they are not standard, it will be more difficult to port your code to another platform.
By last I mean that all memory allocation in lwIP are made static (in compilation time) and I can’t configure specific device without reprogramming it. I would like to know if this features are implemented in FreeRTOS stack.
So yes, the sockopt calls will let you configure most of the RAM consumption ‘at runtime’. NB. Noticed that: UDP sockets can be nasty: suppose there is some kind of bombing of UDP broadcasts, and suppose that your application doesn’t have time to read the messages immediately. In that case a single UDP socket could grab all available network buffers in a short time and make your device become unreachable for a while. +TCP will protect against this, with this define:
#define ipconfigUDP_MAX_RX_PACKETS  8
This means that ‘by default’, no more than 8 messages will be stored per UDP socket. The 9th message will be ignored. Again this can be adapted at runtime and per socket, by using the sockopt ‘FREERTOSSOUDPMAXRX_PACKETS’. Its hard to be brief about +tcp… 🙂 Regards,

FreeRTOS TCP/IP stack vs lwIP

I’m working with STM32F2xx. I wrote my own netif driver for lwIP…
In that case it shouldn’t be hard to start working with +TCP. You could take some NetworkInterface.c as an example.
Your information makes me believe that using native stack would be more efficient compared to lwIP which has to stay compatible with arbitrary any OS.
Yes true, writing a TCP/IP stack for “any OS” is difficult. It is easier (more straight forward) to develop a TCP/IP stack for a single OS (running on +34 platforms). Except for the NIC/PHY, there is only one dependency on the platform: endianness! FreeRTOS makes everything else look the same.
#define ipconfigBYTE_ORDER FREERTOS_LITTLE_ENDIAN
#define ipconfigBYTE_ORDER FREERTOS_BIG_ENDIAN
One example: I saw that within lwIP a lot of code was handled from within an ISR (Xilinx port). Why is that? Because it can not be assumed that a task will become active soon after the interrupt has finished. Whereas within FreeRTOS there is this mechanism of waking up tasks from an ISR.
What is the current status of your development? Is it a time to start looking inside or better to wait for official release?
FreeRTOS+TCP is running in commercial products already. These products have streaming audio, and they run FTP- an HTTP-servers. So yes, I would trust it at this stage already.
My project is commercial and it is already in production, we can not take chances.
No please don’t take any risk 🙂 But if you have the time and capacity for testing the integrations, feel free. Regards, Hein

FreeRTOS TCP/IP stack vs lwIP

…we are offering free commercial type licenses for early adopters who are willing to work closely with us and provide feedback – and it would be great to have an STM32 driver. Regards.

FreeRTOS TCP/IP stack vs lwIP

Does it mean in future you are going to license it? What is “free commercial type licenses”? Sent from my iPhone
On Nov 22, 2014, at 1:44 PM, “Real Time Engineers ltd.” rtel@users.sf.net wrote: …we are offering free commercial type licenses for early adopters who are willing to work closely with us and provide feedback – and it would be great to have an STM32 driver. Regards. FreeRTOS TCP/IP stack vs lwIP Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/freertos/discussion/382005/ To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

FreeRTOS TCP/IP stack vs lwIP

Very broadly speaking various components are available under three license categories:
  • Open source,
FreeRTOS+TCP will always be provided under an open source license, but GPL rather than the modified GPL under which FreeRTOS itself is provided.
  • A light commercial license (what I was referring to),
This removes the requirement to open source, but does not provide a warranty. FreeRTOS+TCP will almost certainly be provided under this type of license too. It will be as per FreeRTOS+UDP license http://www.freertos.org/FreeRTOS-Plus/FreeRTOSPlusTCP/FreeRTOSPlusTCP_License.html Hopefully like other components we already provide, will provide free light commercial licenses for use on some partner’s parts too.
  • A full commercial license,
This is a standard commercial license, with warranty, etc, as per the OpenRTOS license compared to the FreeRTOS license (the two pieces of software being the same). We would not provide that ourselves, but may license other companies to do so – again as per OpenRTOS which is provided by a partner company not by us directly. Regards.