Detailed look at bitstreams, and a taste of Base64 and SD card CRC

Much of the week was spent trying to find out as much as possible about the Xilinx bitstream file formats (.bit). The actual configuration within a bitstream file is encrypted and is not available for public use. The FPGA usually contains the encryption key to be able to run the .bit file and configure itself. So the following information was gather regarding bitstreams, and some of the interesting features about the bitstreams.

What is a Xilinx Bitstream?

The full internal configuration state of SRAM-based pro-grammable logic devices is contained in a programming file called a bitstream.

The bitstream itself is actually microcode for a simple state machine inside the FPGA. It controls the many aspects of configuration data handling: integrity checking, initial and live loading, read-back. It also controls high-level boot-up operations.

Bitstream Composition

Bitstreams consist of a bitstream header followed by configuration packets which consist of a header and data words as described in the previous subsection. There are two types of bitstreams: full and partial. Full bitstreams configure the entire configuration memory in the FPGA while partial bitstreams can configure a subset of the configuration memory.

The bitstream header is optional depending upon the tool that generated it. The header contains information such as the Design Name (usually the name of the .ncd file), the Part Name, the Date and the Time and the total length of the bitstream. The following is a breakdown of the composition of a bitstream header file.

Bitstream header composition.

A bitstream header file is not necessarily constrained to a specified length and may vary depending on the information stored on it and the tool used to generate the bitream, i.e. iMPACT.

Example of a Bitstream header file:

The following is an example of a hex-dump from the header file of a bitstream.

Hex dump from beginning of bitstream file.

From the Hex-dump, some aspects of the composition table become clear. Each of the tokens (i.e. Design name token, Part name token) are separated using the letters a,b,c,d & e. The letter ‘a’ always corresponds to the data token for the Design name token, the letter ‘b’ always corresponds to the data token for the Part name token,  the letter ‘c’ always corresponds to the data token for the Date token, the letter ‘d‘ always corresponds to the data token for the Time token & the letter ‘e’ corresponds to the bitstream token or the bitstream length token which stores how big the bitstream actually is. This is of particular interest as harnessing this information from the bitstream header file itself could be useful in deducing the size of the bitstream to be read. Further details into this is still a bit fuzzy. The bitstream actually starts after the Synch Word, in this case after (aa 99 55 66).

Bitstream CRC 

Other than the header file, the bitstreams also comes with a CRC to check the integrity of the bitstream upon transfer to the FPGA. When enabled, a unique CRC value is calculated based on bitstream contents. If the calculated CRC value does not match the CRC value in the bitstream, the device will fail to configure. When CRC is disabled a constant value is inserted in the bitstream in place of the CRC, and the device does not calculate a CRC. The CRC check takes place right after the configuration data has been loaded to the FPGA. The CRC is enabled using the tool that generated the Bitstream.

Having gathered the above information about he Bitstreams, focus was shifted to helping one of my colleagues with on of his tasks that needed troubleshooting.

Base64 and SD card CRC

First of all, it was important to understand why Base64 encoding is used.

When there is some binary data that needs to be ship across a network, it is not generally done it by just streaming the bits and bytes over the wire in a raw format. Why? Because some media are made for streaming text. Some protocols may interpret the binary data as control characters (like a modem), or the binary data could be screwed up because the underlying protocol might think that a special character combination has been entered (like how FTP translates line endings).

So to get around this, people encode the binary data into characters. Base64 is one of these types of encodings. Why 64? Because you can generally rely on the same 64 characters being present in many character sets, and you can be reasonably confident that your data’s going to end up on the other side of the wire uncorrupted.

A bitstream is encoded to Base64 by converting the 3 bytes (24 bits) of data to 4 bytes of encoded character. The following is an illustration of how this works, as well as the index table for Base64.

Base64

My colleague was successfully able to develop a decoder to decode a Base64 file and store it into the raw sectors of the SD cards. Some important impacts of this process is that the overall size of the bitstreams becomes larger, i.e. since 3 bytes are converted to 4 bytes, a 512 byte file would be converted to 688 bytes. and a standard 350kB file would be converted to a 467kB file.

For the CRC on the SD card, as explained by my colleague on his blog, https://blog.aeste.my/index.php/archives/2867, the SD card uses a CRC 16 to check the validity and the integrity of the data. The CRC works by using a standard polynomial to find a divisor for the data. Before the data is sent, the CRC is calculated by dividing the data by the polynomial (in binary) and considering the remainder from the division to be the CRC value. Once the data is transferred to the SD card, the CRC check is done by dividing the data received by this CRC value and seeing if the end result is 0, which signifies that the data received is intact and uncorrupted. Otherwise, a 1 is returned and a CRC error is displayed.

With this knowledge, and the fact that CRC 16 on  the SD card means 2 bytes for the CRC value, the raw writing capabilities were tested with both the Base64 decoder and the CRC check for data integrity. Using, my colleagues code and on-line calculators to calculate the CRC value for a given data, the code was tested. It was discovered that the code was successfully able to write and check the raw writing for 1 sectors, but returned a CRC error whenever trying to attempt multi-block write. This was quite baffling as the coding seemed to make sense. Troubleshooting was done, but an adequate solution was not found. I updated my colleague about my findings about some of the troubleshooting and hopefully a resolution about why the CRC error is being received can be reached.