Week 06 – Speeding Up The Read Speed, Recreate MBR to Reserve 1 MB of Space, Raw Reading & Writing

First Modification (29 June 2013)

Three text files are created with size of 35 KB, 175KB and 350 KB. The files were created by simply duplicating the whole 35 KB text file five times and 10 times respectively. By using the Timer1 module created last week, the read time for all three files using the original FSfread function are recorded. There are 2.2 s, 7.389 s and 24.123 s, as mentioned last week.  The reading speed are 15.909 KBps, 23.684 KBps and 24.123 KBps.

First modification (29 June 2013) is made by removing a counter to count the byte instructing the function to read. This removes the ability of the FSfread function to specify how many bytes to read and instead it reads 100 bytes no matter how many bytes were instructed to read in the arguments of the function. This saves an operation in the while loop, the speed improvement is minimal. The time recorded for the modified FSfread function are 1.4 s, 5.9 s and 11.9 s. The reading speed is improved to 25 KBps, 29.661 KBps and 29.412 KBps. Although it is just a single operation in the while loop, but due the large amount of looping the loop does, it amplifies the time that a single operation takes significantly.

The Process of PIC18 Reads From SD Card

Figure 1. Flowchart of PIC18 Reading From SD Card

The way of how FSfread works is it first reads the sector where the data is located (the data location is determined by another function, FSfopen). A sector size of a SD card is normally 512 bytes, but it may vary. The SD card used is formatted with 512 bytes sectors. Thus, the PIC18 will then copy the whole 512 bytes data in the sector into a 512 bytes RAM buffer, which is defined in the linker file. This process can be found in the function named  “PIC18_Optimized_SPI_Read_Packet” in SD-SPI.c, this is a optimized function made by Microchip and uses an inline assembly language to achieve maximum performance. There is a write variant of it called “PIC18_Optimized_SPI_Write_Packet“. The speed of this process is fast and is not the major cause of the slowdown.

The next process is that the FSfread will then find the exact location of the data requested in the 512 bytes RAM buffer, and copies the data from the RAM buffer to the destination pointer specified in the function argument. This process uses a while loop, and this is the exact same while which I removed an operation from during my first modification mentioned above. And it turns out that this process is taking up a lot of time. This is verified by commenting out the line which copies the data from the RAM buffer to the destination pointer, “*pointer = RAMread( dsk->buffer, pos++ );“. My guessing on this is because the buffer address is probably more than 8 bits, so it takes a lot of time and Microchip actually wrote a PIC18 optimized read and write in SD-SPI.c to speed up the process of reading and writing between the RAM buffer and the SD card.

Because of the stack frame limitation, I was only able read 100 bytes per FSfread function from the RAM buffer and output 100 bytes. It can actually read slightly more than 100 bytes but I chose 100 bytes due to convenience purposes.

Raw Reading Speed

I was then told by my supervisor to dump the part where the PIC reads from the SD card and stores into buffer because we are interested in how fast it can go and the buffer is not something that we might actually need at the end. By commenting out certain operations in the function codes, the raw reading is tested. The time recorded for raw reading a 35 KB file, 175 KB file and 350 KB file are 0.15 s, 0.70 s and 1.4 s. The reading speeds are 233.33 KBps, 250 KBps and 250 KBps. The speed improved drastically and is somewhat ‘logical’ compared to the previous speed.

Figure 2. FSfread Speeds

Raw Reading & Writing

After solving the speed issue, the ability of reading and writing raw must be tested. But before that, the PIC18 must be able reformat the SD card and reserved 1 MB right after the Master Boot Record (MBR), which is Sector 0,  for the usage of reading and writing raw data onto the SD card. A MBR is 512 bytes in size and contains a lot of information about the SD card. More information about MBR and related boot sectors can be found easily through the internet.

There is a function in FSIO.c which is somewhat similar to what I intended to do with MBR, the function is called FSCreateMBR. The function allows the user to create a master boot record for the device but the user are only able to state where the first boot sector is located and the number of sectors available in the memory. The function is only capable of formatting the SD card into FAT12 and FAT16 only. In other words, the function only can format a SD card up to 2 GB in size.

I created a new FSCreateMBR by modifying it and renaming it to AesteCreateMBR. It is a function which takes in no arguments, and the function will always reverse the first 2048 sectors, which is 1 MB, before it creates the first boot sector. I also modified the code to read from the SD card current size from the old MBR, and if the SD card size to larger than 2 GB, the function will format the card into a FAT16 with only 2 GB. This modification has to be made because the FSCreateMBR cannot format cards higher than 2 GB and I am using a 4 GB SD card. A remark on these two functions is that FSInit must be called before it and a FSformat function must be called after it to create a new first boot sector for the SD card. In the end, 2048 sectors is reserved can be verified by using fdisk, the whole SD card is wiped clean and the computer only recognizes the SD card as a 2 GB SD card instead of a 4 GB SD card.

Since the formatting is working, I created another 2 new functions to read and write raw to the reserved sectors. The functions prototype are “int RawSectorRead (DWORD sector_addr)” and “int RawSectorWrite (DWORD sector_addr)“. Both functions takes in an argument to state which sector to read/write, the functions return 0 if read/write is successful and return 1 if read/write is not successful. For testing purposes, RawSectorWrite will write a ASCII character ‘a’ to the whole sector. The ASCII character ‘a’ is intended to be read from the sector and displayed through the USART or Minicom. RawSectorRead will be reading from a sector and then saves the data into a 512 bytes RAM buffer. But, to display the data through USART, another function is created, “size_t testRead (void *ptr)“. testRead will read the first 100 bytes of data from the RAM buffer and returns the number of bytes read. I used this function because the RAM buffer is only defined in FSIO.c so I created another function to read the data from the RAM buffer. The RAM buffer, gDataBuffer, is declared globally in FSIO.c, thus, this three functions must be called together to prevent other FSIO.c functions to alter the data in the gDataBuffer. Finally after some minor correction in the first version of the codes, writing raw into sector and then reading it and display it through to USART is working nicely.