Quality RTOS & Embedded Software

 Real time embedded FreeRTOS RSS feed 
Real time embedded FreeRTOS mailing list 
Quick Start Supported MCUs PDF Books Trace Tools Ecosystem TCP & FAT Training




Loading

FAT acces time

Posted by hesimo on May 22, 2017

In a loop I open, write and close a file on SD-Card. Each time a file is opened, the access time is increased. How can I solve the problem?


FAT acces time

Posted by rtel on May 22, 2017

Have you tried a different SD card? There HUGE variations between cards. It is worth keeping a few premium brand cards to hand for comparison.


FAT acces time

Posted by heinbali01 on May 22, 2017

Beside the quality of the SD-cards, which indeed varies a lot:

How many files do you create within a single directory? The FAT system has a known problem with huge directories, I mean directories with thousands of files. We have tried to optimise the access in huge directories. There are some options that may increase the efficiency:

~~~ #define ffconfigHASHCACHE 1 /* Set to CRC8 or CRC16 to use 8-bit or 16-bit * HASH values respectively. */ #define ffconfigHASHFUNCTION CRC16 #define ffconfigHASHCACHEDEPTH 64 ~~~ Postpone flushing to disk :

~~~ #define ffconfigCACHEWRITETHROUGH 0 ~~~


FAT acces time

Posted by hesimo on May 23, 2017

I have tested different card.

The controller ist an ATSAM4E @ 120MHz. I use the HSMCI (own driver. not the asf stuff) to write 250 files (96Bytes/file) in a single directory. Hash function brings no improvement and needs too much ram.

Intenso SDHC 4GB class 4 1 16ms 100 36ms 250 74ms

LYNQ SDHC 4GB class 4 1 19ms 100 45ms 250 86ms

Transcend SDHC 4GB class 10 1 14ms 100 36ms 250 69ms

Toshiba SDHC 4GB class 4 1 14ms 100 37ms 250 71ms

I have determined that the time is used in the ff_open function.


FAT acces time

Posted by heinbali01 on May 23, 2017

Hi Hendrik, thanks for this overview! You're proving that the difference between qualities of the cards tested is not as spectacular as we assumed. Maybe the speed differences will show-up more clearly when writing tons of data.

I assume that the times show ( 16, 36, 74 ms ), are the times you need to create a single file? To me that looks as extremely fast!

How did you measure this? Is that the time it took to call ff_fopen()?

We have spend a lot of time on working with directories that contain like 65K files. The hashing of directory entries should really help to determine if a given short file name already exists or not.

I just took out my SAM4E Xplained board and created 250 files. The time it took to open/write/close was much longer: a minimum of 200 ms per file. The open itself took most of the time.

I attached a graph the actual times on my board. I used a cheap Transcend SDHC 4GB SD-card.

I wonder where our enormous difference in time comes from

Attachments


FAT acces time

Posted by heinbali01 on May 23, 2017

Another question that I should have asked you: do your file names confirm to the 8.3 notation? Or are those Long File Names ?


FAT acces time

Posted by hesimo on May 24, 2017

Hi, the files already exist on card. In a for loop I open overwrite and close each file. The times I measured, are the times for a single file. I simply measure the time by xTaskGetTickCount(). The file names confirm to the 8.3 notation. For all files I need about 10s.

I tested again (Transcend SDHC 4GB class 4) file 1: open 4ms, write 0ms, close 7ms file 100: open 25ms, write 2ms, close 6ms file 250: open 82ms, write 3ms, close 8ms


FAT acces time

Posted by heinbali01 on May 26, 2017

Hi Hendrik, I was measuring the time to create 250 new files, which is a lot more work.

ffconfigHASH_CACHE only optimises the creation of new files. It calculates a hash table for a directory so it knows quickly if a new short file name already exists or not.

On my SAM4E it also takes 10s to re-write 250 files of 96 bytes each.

These were the times that I measured:

~~~ # Open Write/Close 1 1 9 2 1 7 3 0 8

100 5 8 101 9 8 102 5 7

150 18 9 151 17 9 152 17 9

200 17 9 201 18 9 202 17 10

247 22 9 248 21 10 249 21 10 250 21 17 ~~~

And I see some strange peeks while writing/closing the files:

~~~ # Open Write/Close 130 14 9 131 14 154 132 15 305 133 15 316 134 15 151 135 15 9 136 15 9 ~~~

I'm not sure what is taking up to 300 ms. Sometimes an SD-card has some internal clean-up, it will flush cached sectors, which can take time. The card will remain 'busy' until the internal work is done.

The files were created from FILE001.TXT up to FILE250.TXT. So I think that when you upgrade them it is normal to see access times increasing along with the index of the file.

Regards.


FAT acces time

Posted by glenenglish on May 26, 2017

Hi Hendrick I have lots of SD card experience, You should NEVER assume the write performance will be deterministic. Hein is absolutely correct with regard to internal housekeeping.

However- stick to the highest class (and most expensive cards). AND there are MANY counterfeits . Lots of low quality rubbish with "Sandisk Extreme PRO " printed on them.. fake fake fake fake fake . only buy from reputable and listed retailers. Buying on ebay is trouble...

The cheap cards- well my mother does not care if the photo storage takes an extra second more than usual, but your application might !


[ Back to the top ]    [ About FreeRTOS ]    [ Sitemap ]    [ ]




Copyright (C) 2004-2010 Richard Barry. Copyright (C) 2010-2016 Real Time Engineers Ltd.
Any and all data, files, source code, html content and documentation included in the FreeRTOSTM distribution or available on this site are the exclusive property of Real Time Engineers Ltd.. See the files license.txt (included in the distribution) and this copyright notice for more information. FreeRTOSTM and FreeRTOS.orgTM are trade marks of Real Time Engineers Ltd.

Latest News:

FreeRTOS V9.0.0 is now available for download.


Free TCP/IP and file system demos for the RTOS


Sponsored Links

⇓ Now With No Code Size Limit! ⇓
⇑ Free Download Without Registering ⇑


FreeRTOS Partners

ARM Connected RTOS partner for all ARM microcontroller cores

Renesas Electronics Gold Alliance RTOS Partner.jpg

Microchip Premier RTOS Partner

RTOS partner of NXP for all NXP ARM microcontrollers

Atmel RTOS partner supporting ARM Cortex-M3 and AVR32 microcontrollers

STMicro RTOS partner supporting ARM7, ARM Cortex-M3, ARM Cortex-M4 and ARM Cortex-M0

Xilinx Microblaze and Zynq partner

Silicon Labs low power RTOS partner

Altera RTOS partner for Nios II and Cortex-A9 SoC

Freescale Alliance RTOS Member supporting ARM and ColdFire microcontrollers

Infineon ARM Cortex-M microcontrollers

Texas Instruments MCU Developer Network RTOS partner for ARM and MSP430 microcontrollers

Cypress RTOS partner supporting ARM Cortex-M3

Fujitsu RTOS partner supporting ARM Cortex-M3 and FM3

Microsemi (previously Actel) RTOS partner supporting ARM Cortex-M3

Atollic Partner

IAR Partner

Keil ARM Partner

Embedded Artists