Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Real time embedded FreeRTOS mailing list 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT Training




Loading

FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 8, 2017

Hello,

I have great trouble with an FreeRTOS+TCP application I'm developing. It all worked fine and I communicated with the board over TCP untill suddenly no communication is possible anymore. I cannot connect to the board and not even ping to the board. Then I get: Destination host unreachable. Some background info:

Hardware platform: Xilinx Zynq on a Trenz TE0720 module Software: Xilinx SDK 2016.2 Network: Switch with router, pc and board connected to it Settings: Static IP

The first time I developed the application making use of the TCP stack for weeks when it suddenly didn't work anymore, changing nothing on the TCP settings. Then I built a new application from scratch to see what was going wrong, which was working fine again, so I imported my project and continued developing. Now after 4 days developing and testing, I just wanted to conduct a new test, it again cannot be connected to anymore, and the ping returns Destination host unreachable.

After IP_Init I set the config to call vApplicationNetworkHook if all is set. Strange thing is, this is still called, only no connection can be made.

Can someone please give some direction to look in for? Something on the Zynq itself? Something on the network?

Best regards, Giovanni


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 8, 2017

Hi Giovanni,

Lately there were a few changes to the network-interface software for Zynq. I'm not sure which release you're using now, but you might want to compare and try the new version that I attached. This is a non-official release, just for testing.

Attachments

Zynq_080317.zip (24112 bytes)

FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 8, 2017

Hello Hein,

Thank you for your answer. It seems the files I have are exactly the same as the files you sent. So unfortunately no progress there.

Thanks again, if you have more tips, please share!

Best regards, Giovanni


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 8, 2017

Sorry, my stupid. The latest /labs release does contain the latest network interface driver for Zynq.

Then I get: Destination host unreachable

In those cases, does it help to do a cold start of the device?

Something on the Zynq itself?

I'm afraid I'm not an expert on that

After IP_Init I set the config to call vApplicationNetworkHook if all is set. Strange thing is, this is still called, only no connection can be made

At first I thought of network buffers being exhausted. But FreeRTOS_IPInit() is being called after a reboot and everything should be OK.

Is your device OK? Do you have a second one to try? What about the LAN? Do you use a switch between laptop and device? Can other devices still be reached? Regards, Hein


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 8, 2017

Hello Hein,

  • No problem.

  • Every time I conduct a session I have to do a cold restart, or else SDK will not execute the session properly.

  • Indeed, IPInit() is only called after a reboot / restart / cold restart.

  • I have a second device, which does exactly the same.

  • I use a switch between pc and device and the rest of the network stays working as it did before. Also on Wireshark no messages with the devices IP address come through. Restart of the pc / switch or whatsoever make no difference.

For now, I'm able to keep on developing with an older version (from last friday), untill that one will stop working.. I am starting to get clueless..

Thanks again!

Best regards, Giovanni


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 8, 2017

Hi Giovanni, it is very difficult to get insight into the problem you encounter.

Do understand correctly that the symptom is that your device becomes unreachable? It doesn't respond to any of the network protocols?

But can you confirm that the CPU is still running? Are all tasks still running? Maybe you can check that with LED's, or a serial connection, or maybe in an Eclipse debug session?

The TCP/IP stack has been out for a long time and users consider is stable. You'll have to investigate. It is important to keep on checking the amount of available resources: stack per task, free memory for pvPortMalloc(), the number of free network buffers. And of course, if an exception occurs: let the device make it clear in some way. Usually an application will end in a for(;;) loop, becoming unreachable.

Now about the reboots/resets: we should be very clear: what do you do exactly to get your Zynq to life again?


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 8, 2017

Hello Hein,

I was hoping someone had this problem before and had a direction to search in.

CPU is still running, I was testing hardware, which had the highest priority. So I laid down the Ethernet problem and made another thread with a terminal application to test the hardware. Everything works as expected, apart from being reachable from the network. Also the network task is still running, but the listening function has no use, if the device is unreachable.

When it still worked, I could ping and connect to the devices after IPInit(). Also DHCP worked and I could track down the bootp messages through Wireshark. After the fault occured not anymore.

I'm afraid I have to check resources in memory etc, what you are saying, not my expertise. The application doesn't end in a endless loop, because it still calls the vApplicationNetworkHook in which I can still 'do things'.

Now about the reboots/resets: we should be very clear: what do you do exactly to get your Zynq to life again?

1- Shutdown the voltage of the board and power it back up 2- Disconnect and delete the debug session in SDK 3- Start a new debug session in SDK

This is the only way to start a proper debug session, else the application will not start properly and will hang somewhere during initialization of the processor. Not that strange, have noticed it with other processors also.

Thank you again for your effort!


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 10, 2017

If you look with WireShark, you will see that the device sends so-called gratuitous ARP-packets, for example:

~~~ 2 3.508662 Microchi_d4:6a:53 Broadcast ARP 60 Gratuitous ARP for 192.168.2.201 (Request) ~~~

Can you see these ARP-packets? And when the device becomes "unreachable", do you still see them?

About checking resources.

In many versions of NetworkInterface.c you will see the following 'checking' code:

~~~

/* Define in you FreeRTOSIPConfig.h : */

define ipconfigCHECKIPQUEUE_SPACE 1

/* Add to your prvEMACHandlerTask() : */

static void prvEMACHandlerTask( void *pvParameters ) { UBaseTypet uxLastMinBufferCount = 0; UBaseTypet uxLastMinQueueSpace = 0; UBaseTypet uxLastFreeHeapSpace = 0; UBaseTypet uxCurrentCount;

for( ;; )
{
    uxCurrentCount = uxGetMinimumFreeNetworkBuffers();
    if( uxLastMinBufferCount != uxCurrentCount )
    {
        /* The logging produced below may be helpful
        while tuning +TCP: see how many buffers are in use. */
        uxLastMinBufferCount = uxCurrentCount;
        FreeRTOS_printf( ( "Network buffers: %lu lowest %lu\n",
            uxGetNumberOfFreeNetworkBuffers(), uxCurrentCount ) );
    }

    #if( ipconfigCHECK_IP_QUEUE_SPACE != 0 )
    {
        uxCurrentCount = uxGetMinimumIPQueueSpace();
        if( uxLastMinQueueSpace != uxCurrentCount )
        {
            /* The logging produced below may be helpful
            while tuning +TCP: see how many buffers are in use. */
            uxLastMinQueueSpace = uxCurrentCount;
            FreeRTOS_printf( ( "Queue space: lowest %lu\n", uxCurrentCount ) );
        }
    }
    #endif /* ipconfigCHECK_IP_QUEUE_SPACE */
    {
        uxCurrentCount = xPortGetMinimumEverFreeHeapSize();
        if( uxLastFreeHeapSpace != uxCurrentCount )
        {
            /* The logging produced below may be helpful
            while tuning +TCP: see how many buffers are in use. */
            uxLastFreeHeapSpace = uxCurrentCount;
            FreeRTOS_printf( ( "Heap: lowest %lu\n", uxCurrentCount ) );
        }
    }
    ...
}

} ~~~

It will send out logging like "Network buffers: 24 lowest 20". Make sure that "lowest" never gets close to zero.

The IP-task has a message queue. You will see logging like "Queue space: lowest 19". This should remain far above zero.

Also the lowest amount of free heap will be logged as e.g. : "Heap: lowest 84904"

Can you make sure that this logging is sent to a terminal so you never miss it?

1- Shutdown the voltage of the board and power it back up

Is this power-cycle really necessary? Or would steps 2) en 3) be enough?

And a final request: could you attach the FreeRTOSIPConfig.h file that you use?

Regards.


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 10, 2017

Hello Hein,

Thanks again!

About ARP: When the device is reachable I can see the gratuitous ARP-packets. When the device is not reachable, I cannot see them. So it seems there is nothing coming out of the device (no arp request or something like it). I'm going to check that on hardware level.

About logging it gives the following: Network buffers: 64 lowest 64 (and later on it prints a second time and then lowest is 63) Queue Space: lowest 101 Heap lowest: 41820000 (later on, the second time it prints 4179032)

If I understand you correctly no problems here.

About power cycling: On my development board it was really needed, but now I tried with a full reset en it is not needed anymore. Maybe something on the development board with reset of power. Thanks for the note.

Attached: FreeRTOSIPConfig.h

Best regards

Attachments

FreeRTOSIPConfig.h (21105 bytes)

FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 10, 2017

Giovanni,

Ref your FreeRTOSIPCOnfig.h : I would definitely define:

~~~ /* Use a zero-copy EMAC driver in both directions. */ #define ipconfigZEROCOPYTXDRIVER 1 #define ipconfigZEROCOPYRXDRIVER 1 ~~~

Although the latter ipconfigZERO_COPY_RX_DRIVER is never mentioned for in the Zynq driver.

The Zynq Ethernet driver was a bit difficult to develop because of the cached memories. DMA see the physical memory, while the CPU reads and writes to memory caches. We've decided to reserve 1 MB of non-cached memory and dedicate that to the +TCP and +FAT drivers. It will be accessed by both DMA's.

The ipconfigZERO_COPY_TX_DRIVER variant of the Zynq driver was tested better than the memcpy version.

Did you have a particular reason to define ipconfigZERO_COPY_TX_DRIVER as 1 ?


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by giost on March 10, 2017

Hein,

Unfortunately it did not make a difference. And in your last sentence you meant a reason for being 0 instead of 1. Because it is (was) defined 0. No I did not had a reason and cannot remember changing that definition.

But I did some comparing (again and again) and I now got it back working. Not sure yet if it will stay working. In the IP settings nothing changed, also not in software. So I went back to the hardware definition in Xilinx Vivado and there was a clock setting for Ethernet changed. Of which I'm sure the firmware guy nor I have changed it. So I want to investigate if this is a bug in Vivado.

This setting influences the Vivado / SDK automatically generated driver causing to still properly 'set' the driver and the IP stack run through its initialization functions. But physically there was no connection to the network due to a clock not properly set or synchronized to.

As I said, I'm going to investigate it.

Though I've learned that the ZEROCOPYDRIVER suits better vor Zynq applications, so I will use that.

Thank you so much for helping me, I really appriciate it!


FreeRTOS+TCP does not connect to network anymore after a few days

Posted by heinbali01 on March 10, 2017

Glad to hear.

As I said, I'm going to investigate it.

When you've found it, please report about it so you can help others.

This setting influences the Vivado / SDK automatically generated driver causing to still properly 'set' the driver

When there is time, you may want to get rid of the automatic things of Eclipse/SDK and start using a Makefile project. It gives a very precise control of what happens during the make process. Regards, Hein


[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) 2004-2010 Richard Barry. Copyright (C) 2010-2016 Real Time Engineers Ltd.
Any and all data, files, source code, html content and documentation included in the FreeRTOSTM distribution or available on this site are the exclusive property of Real Time Engineers Ltd.. See the files license.txt (included in the distribution) and this copyright notice for more information. FreeRTOSTM and FreeRTOS.orgTM are trade marks of Real Time Engineers Ltd.

Latest News:

FreeRTOS V9.0.0 is now available for download.


Free TCP/IP and file system demos for the RTOS


Sponsored Links

⇓ Now With No Code Size Limit! ⇓
⇑ Free Download Without Registering ⇑


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

Renesas Electronics Gold Alliance RTOS Partner.jpg

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

Atmel RTOS partner supporting ARM Cortex-M3 and AVR32 microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Xilinx Microblaze and Zynq partner

Silicon Labs low power RTOS partner

Altera RTOS partner for Nios II and Cortex-A9 SoC

Freescale Alliance RTOS Member supporting ARM and ColdFire microcontrollers

Infineon ARM Cortex-M microcontrollers

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

Cypress RTOS partner supporting ARM Cortex-M3

Fujitsu RTOS partner supporting ARM Cortex-M3 and FM3

Microsemi (previously Actel) RTOS partner supporting ARM Cortex-M3

Atollic Partner

IAR Partner

Keil ARM Partner

Embedded Artists