Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Real time embedded FreeRTOS mailing list 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT Training




Loading

ftp get truncates file

Posted by rasty1 on July 14, 2016

Hi,

FTP RETR command truncates files on RAM disk. I managed to reproduce the same problem with CLI. It happens if I add "//" before file name as FTP server does. Beside this FTP hangs on get, dir.

Please see log below:

-->dir test.txt [writable file] [size=2048] <<< File is OK . [directory] [size=1024] -->type test.txt Hello word -->type //test.txt -->dir test.txt [writable file] [size=0] <<** file is truncated** . [directory] [size=1024]

target log:

-->type log.txt test log TCP socket on port 80 TCP socket on port 21 TPC-server: new FTP client USER

PORT a041418ip:56940 Client address a041418ip:5694 0 RETR test.txt FTP: sent: '//test.txt' 0 B ytes (0 bytes/sec)

Did anyone see anything like that?

Best regards Rasty


ftp get truncates file

Posted by rtel on July 14, 2016

I can try and replicate the behaviour here. Can you please provide more information on how to duplicate it. I understand you are sending a file that has // on the front, but how are you sending it? For example, is this done from a command line or from an IDE. If it is done from a command line please show the commands you are using.


ftp get truncates file

Posted by rasty1 on July 14, 2016

It is very easy to duplicate. Just create a file in RAM disk (say test.txt) and than in CLI use type command: type //test.txt


ftp get truncates file

Posted by rasty1 on July 14, 2016

Maybe small deviation from stock example:

define mainRAMDISKNAME "/"

and it does not allow to mak a RAM disk smaller than 3 MB.

pxRAMDisk = FFRAMDiskInit( mainRAMDISKNAME, ucRAMDisk, mainRAMDISKSECTORS, mainIOMANAGERCACHESIZE ); configASSERT( pxRAMDisk );

/* Print out information on the RAM disk. */ stat = FFRAMDiskShowPartition( pxRAMDisk ); pxFile = fffopen( "test.txt", "w" ); fffwrite( teststr, sizeof(teststr), 1, pxFile ); fffclose( pxFile );


ftp get truncates file

Posted by rtel on July 14, 2016

So this is not an FTP issue - from your first post I thought it was. Simply trying to view a file after adding // to the front of the file name. I will give that a try, although // is not a valid path to the file it should not result in the file being truncated.


ftp get truncates file

Posted by rasty1 on July 14, 2016

Actually I have an issues with FTP as well - it hangs on DIR, RETR command.

after DIR command it enters tight loop and I had to enter some delay in order to get some responce for( ;; ) { FreeRTOS_TCPServerWork( pxTCPServer, xInitialBlockTime ); vTaskDelay(10); <<-- added delay }

Truncated file is something solid that managed to see and reproduce.


ftp get truncates file

Posted by rtel on July 14, 2016

I just tried what I think you tried before, albeit with the head revision code rather than the release code. The output is below:

First, view a small file I FTPed onto the RAM disk, which is mounted in /

~~~ [Press ENTER to execute the previous command again] > dir

p.txt [writable file] [size=38] . [directory] [size=1024]

[Press ENTER to execute the previous command again] ~~~

Then I view the file without the // on the front.

~~~ type p.txt

11111111111111111111111111111111111111

[Press ENTER to execute the previous command again] ~~~

Performing a directory listing shows the file has the same length:

~~~ dir

p.txt [writable file] [size=38] . [directory] [size=1024]

[Press ENTER to execute the previous command again] ~~~

Then I do the same, but this time add the // to the front of the file name:

~~~ type //p.txt

11111111111111111111111111111111111111

[Press ENTER to execute the previous command again] ~~~

When I perform a directory listing again the file has not been truncated.

~~~ dir

p.txt [writable file] [size=38] . [directory] [size=1024]

[Press ENTER to execute the previous command again] > ~~~


ftp get truncates file

Posted by rasty1 on July 15, 2016

I see a difference in behavior of dir test.txt [writable file] [size=2048] On my system it laways reports 2048 no matter what size is.


ftp get truncates file

Posted by rtel on July 15, 2016

You mentioned you were using a RAM disk, but I didn't see a mention of the target. Are you using the Windows demo or something else?


ftp get truncates file

Posted by rasty1 on July 15, 2016

I'm using port to uBlaze. Based on Version from 2015. I can check later exact version. I managed to operate TCP/IP stack.

The difference from standard demo are 1. RAM disk mounted to "/" 2. no ssd 2. Ethernet MTU is set to 200 bytes at host side - I didn't find how to limit MTU at freertos (or it did not work) 3. I implemented TCP/IP CLI rather than UDP

Problem that I've faced down the road 1. Cannot make RAM disk smaller than 3mb - format fails. 2. FTP hangs at dir, send, get - enters end-less tight loop. I mentioned this in prev post. 3. DIR via CLI reports incorrect size of the file (maybe some sympthom of the problem)

If you didn't see anything like that before I'll try to update to the latest version. What version have you mentioned in your post?


ftp get truncates file

Posted by rtel on July 15, 2016

I'm using the head revision, which is not in a public repository, if you send me your files (not your own code, just the FreeRTOS, FreeRTOS+FAT and FreeRTOS+TCP files) then I can update those files to the latest version and send them back. You can contact me on r dot barry at freertos [dot] org.


ftp get truncates file

Posted by rasty1 on July 15, 2016

Thank you very much for your help! I sent files.


ftp get truncates file

Posted by rasty1 on July 15, 2016

Thank you very much for your help! I sent files.


ftp get truncates file

Posted by rtel on July 15, 2016

I have updated the +TCP and +FAT code to the head revision of the single interface TCP version - pay no attention to the version number in the header files as its not release code just a working copy. It looks like you were actually using quite an old version. I have not updated any of the other files - you may also like to use FreeRTOS V9.0.0, which you can download from SourceForge here.


ftp get truncates file

Posted by rasty1 on July 15, 2016

Thank you very much. I'll check it on Sunday and let you know the results.


ftp get truncates file

Posted by heinbali01 on July 17, 2016

Thanks for reportting this.

You wrote:

It happens if I add "//" before file name as FTP server does -->type //test.txt -->dir

Isn't //test.txt an invalid file name?

+FAT does have code to handle relative paths, it also recognises invalid characters, but it doesn't always detect invalid paths or file names. I think it is even possible to create a file with an empty string as a file name.

For some people, this is a short-coming. For others it is a blessing because they want a fast and small library.

We could add some more code checking path validity and make this dependent on some macro ffconfigCHECK_PATH_VALIDITY


ftp get truncates file

Posted by rasty1 on July 17, 2016

I tested a new version, result is the same. I think that something is wrong with ramdisk/filesystem configuration. DIR reports 2048 bytes, while in your example I see 28 bytes. Can you send me you setup of RAMDISK? -->dir **test.txt [writable file] [size=2048]** log.txt [writable file] [size=2048] . [directory] [size=1024] -->type test.txt Hello word -->type //test.txt

-->dir **test.txt [writable file] [size=0]** log.txt [writable file] [size=2048] . [directory] [size=1024]


ftp get truncates file

Posted by heinbali01 on July 17, 2016

Hi Rasty,

Do you have ffconfigPATH_CACHE enabled? It remembers the cluster number that belongs to a certain path. You could try the same test with:

#define ffconfigPATH_CACHE    0

But once again, //test.txt is an invalid file name and I think that any API called with such an entry should return an error. ( at this moment I don't have any hardware with a RAM-disk. Beginning of August I will replicate the problem and come back to it )


ftp get truncates file

Posted by rasty1 on July 17, 2016

ffconfigPATH_CACHE is not defined by default. I know that // is not valid, maybe it is symphom for other problem as well as reporting of wrong size of file. BTW. Truncation does not happen and size is reported correctly if file is in sub-folder


ftp get truncates file

Posted by heinbali01 on July 17, 2016

I do have a device for testing with a RAM disk: the Windows demo project!

I just tried what you've done with a 4MB RAM-disk mounted on:

#define mainRAM_DISK_NAME    "/"

But I'm afraid I can not repeat your finding. This is what I saw when using the CLI commands:

~~~~ >dir ff_test [directory] [size=0] test.txt [writable file] [size=12] . [directory] [size=1024]

>type test.txt
Hello world

>type //test.txt
Hello world

>dir
ff_test [directory] [size=0]
test.txt [writable file] [size=12]
. [directory] [size=1024]

~~~~

I tried both with and without the path caching (ffconfigPATH_CACHE).

Please find attached the file FreeRTOSFATConfig.h that I used for this test.

Attachments

FreeRTOSFATConfig.h (13808 bytes)

ftp get truncates file

Posted by rasty1 on July 17, 2016

I had to add 2 defines, otherwise it would not compile.

define ffconfigDEV_PATH "/"
define ffconfigDEV_SUPPORT 1

Still the same problems


ftp get truncates file

Posted by rasty1 on July 17, 2016

I'm afraid there is some basic problem with file system (please scroll to the end):

char teststr[]="Hello ------------------------------- wordnr"; /* Create the RAM disk. */ pxRAMDisk = FFRAMDiskInit( mainRAMDISKNAME, ucRAMDisk, mainRAMDISKSECTORS, mainIOMANAGERCACHE_SIZE ); configASSERT( pxRAMDisk );

/* Print out information on the RAM disk. */
stat = FF_RAMDiskShowPartition( pxRAMDisk );
pxFile = ff_fopen( "test.txt", "w" );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fclose( pxFile );

File looks like:

-->type test.txt Hello ------------------------------- word

------------------------------- word

--------------------------- word

---------------------- word

----------------- word

------------ word

------- word

-- word

rd

Hello ------------------------------- word

------------------------------- word

--------------------------- word

---------------------- word

----------------- word

------------ word

------- word

-- word

rd

Hello ------------------------------- word


ftp get truncates file

Posted by heinbali01 on July 17, 2016

Thanks, that was the difference between your and my project. Now it is reproducible for me. I'll have a detailed look at it.

Regards, Hein


ftp get truncates file

Posted by heinbali01 on July 17, 2016

You can either exclude ff_dev_support.c from your project and define:

#define ffconfigDEV_SUPPORT    0

... or you can define the following:

#define ffconfigDEV_SUPPORT    1
#define ffconfigDEV_PATH       "/dev"

The module was included in the +FAT distribution, but not yet documented. At this moment, the code is only a sketch. The idea is to open a virtual file and talk with a device ( just like e.g. /dev/ttyS0 under Linux ).


ftp get truncates file

Posted by heinbali01 on July 17, 2016

char teststr[]="Hello ------------------------------- wordnr"; fffwrite( teststr, sizeof(teststr), 1, pxFile );

Could you try the same test but without adding the terminating zero of the strings, like this:

ff_fwrite( test_str, sizeof(test_str) + 1, 1, pxFile );

Adding zeros is a bit confusing


ftp get truncates file

Posted by heinbali01 on July 17, 2016

Sorry, not +1 but -1 byte of course:

~~~~ -fffwrite( teststr, sizeof(teststr) + 1, 1, pxFile ); +fffwrite( teststr, sizeof(teststr) - 1, 1, pxFile ); ~~~~


ftp get truncates file

Posted by heinbali01 on July 17, 2016

Sorry, not +1 but -1 byte of course:

~~~~ -fffwrite( teststr, sizeof(teststr) + 1, 1, pxFile ); +fffwrite( teststr, sizeof(teststr) - 1, 1, pxFile ); ~~~~


ftp get truncates file

Posted by rasty1 on July 18, 2016

do you means "sizeof(test_str) - 1"? I'd expect some garbage at the end of the line. I did this test because my logs that I print to file also look incomplete.


ftp get truncates file

Posted by rasty1 on July 18, 2016

fffwrite( teststr, sizeof(test_str)-1, 1, pxFile );

-->type test.txt Hello ------------------------------- word Hello ------------------------------- word Hello ------ ------------------------- word Hello ------------ ------------------- word Hello ------------------ ------------- word Hello ------------------------ ------- word Hello ------------------------------ - word Hello ------------------------------- word

Hello ------------------------------- word Hell o ------------------------------- word Hello ---- --------------------------- word Hello ---------- --------------------- word Hello ---------------- --------------- word Hello ---------------------- --------- word Hello ---------------------------- --- word Hello ------------------------------- wo rd Hello ------------------------------- word He llo ------------------------------- word Hello -- ----------------------------- word Hello -------- ----------------------- word Hello -------------- ----------------- word


ftp get truncates file

Posted by heinbali01 on July 18, 2016

Right, that is what is expected.

I did this test because my logs that I print to file also look incomplete.

Have you found out why?


ftp get truncates file

Posted by rasty1 on July 18, 2016

I found the problem with typing file! unnessesary "strcat( pcWriteBuffer, cliNEW_LINE )" in prvTYPECommand inserts cr-lf every 50 chars. Now my logs look great! And I approach to the real problem. FTP and HTTP do not work.


ftp get truncates file

Posted by rasty1 on July 18, 2016

Short summary. 1. Patch (mentioned in https://sourceforge.net/p/freertos/discussion/382005/thread/afe1ae9f/#c48a/06af )did not help. 2. File size in sub-folder is reported correctly and files are not truncated. 2. Size of files in root is still reported as 2048 and truncated if opened for read with // before file name. 3. unnessesary "strcat( pcWriteBuffer, cliNEW_LINE )" in prvTYPECommand inserts cr-lf every 50 chars. Not severy but confusing.

Thank you very much dor the help.


ftp get truncates file

Posted by heinbali01 on July 18, 2016

What 'patch' are you referring to?

Do you mean this:

~~~~ You can either exclude ffdevsupport.c from your project and define: #define ffconfigDEV_SUPPORT 0 ~~~~

That should help for the problem that you reported as:

~~~~ -->type //test.txt ~~~~

And this should also solve it:

~~~~

define ffconfigDEV_SUPPORT 1
define ffconfigDEV_PATH "/dev"

~~~~

The mentioned defines must appear in your FreeRTOSFATConfig.h.

In your case, when ffconfigDEV_PATH is an empty string, the string "//test.txt" refers to a device and not a file.

A size of 2048 bytes is returned for a device and in your case "//test.txt" is still interpreted as a divice.

By the way, there is an easy way of sending out logging:

FreeRTOS-Plus/Demo/Common/Utilities/UDPLoggingPrintf.c

It has some buffering of the logging lines and it also precedes each line with a millisecond-time-stamp, such as:

2016-07-18 23:43:12.896 IP Address: 192.168.2.1

It is easy to add TCP logging to this module. I normally connect it to a telnet server. lUDPLoggingPrintf is not interrupt-proof :-(

~~~~ strcat( pcWriteBuffer, cliNEW_LINE ) ~~~~

Some telnet clients like this, because only complete lines (with a LF) will be flushed. For others (also for me), the extra LF can be confusing.

In UDPLoggingPrintf.c, no LF's will be added. Also it has an option to turn every LF into a LF/CR:

~~~~ #define configUDPLOGGINGNEEDSCRLF 1 ~~~~

"hello world\n" will be then sent as "hello world\r\n"

Regards.


[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) 2004-2010 Richard Barry. Copyright (C) 2010-2016 Real Time Engineers Ltd.
Any and all data, files, source code, html content and documentation included in the FreeRTOSTM distribution or available on this site are the exclusive property of Real Time Engineers Ltd.. See the files license.txt (included in the distribution) and this copyright notice for more information. FreeRTOSTM and FreeRTOS.orgTM are trade marks of Real Time Engineers Ltd.

Latest News:

FreeRTOS V9.0.0 is now available for download.


Free TCP/IP and file system demos for the RTOS


Sponsored Links

⇓ Now With No Code Size Limit! ⇓
⇑ Free Download Without Registering ⇑


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

Renesas Electronics Gold Alliance RTOS Partner.jpg

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

Atmel RTOS partner supporting ARM Cortex-M3 and AVR32 microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Xilinx Microblaze and Zynq partner

Silicon Labs low power RTOS partner

Altera RTOS partner for Nios II and Cortex-A9 SoC

Freescale Alliance RTOS Member supporting ARM and ColdFire microcontrollers

Infineon ARM Cortex-M microcontrollers

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

Cypress RTOS partner supporting ARM Cortex-M3

Fujitsu RTOS partner supporting ARM Cortex-M3 and FM3

Microsemi (previously Actel) RTOS partner supporting ARM Cortex-M3

Atollic Partner

IAR Partner

Keil ARM Partner

Embedded Artists