Converting the new Firefox cache2 files to an SQLite DB for investigating
Before I discuss the tool and the structure of the data it presents to the user of the Browser, I’ll look a little at the format of the data. For this exercise I’ll ignore the index file for the cache as this just provides a quick way of linking the chrome cache database with individually cached files, this is not important when working with both sets of data in SQLite as you’ll see.
But first a quick taster with a screenshot of a query on the resulting tables created by FirefoxCache2ToSQLite and viewed with the Forensic Browser for SQLite and – this will give you an idea of the sort of data you can see with a formatted report before I go on with the details.
Cache2 file format
Individual cached files are simply written to disk as a normal file with a name which is a sha1 hash of the URL of the file, simple. The file is followed immediately by the metadata followed by a 4 byte big endian integer (all integers are big endian) which is the length of the original file. These four bytes are the last four bytes of the file and equate to the offset of the start of the metadata.
So what is the metadata that follows the file content? The first few bytes are hashes of some of the data and are not really relevant for this article, we do however need to skip them and as the amount of data is variable we need to understand a little of what is there. Essentially there is a 4-byte hash followed by a further 2 bytes for every 262,144 bytes of the original file size. So we need to divide the file size by 262,144 and round up the result, multiply by 2 and add 4 and then add this to the offset value (the last four bytes of the cache file) to get to the start of the metadata itself.
The first section of metadata is 28 bytes split into 7 big-endian numbers, this is followed immediately by the URL of the cached page.
The screenshot below shows that the last four bytes of the cache file point to the end of the file proper, i.e. the beginning of the metadata at offset 0x0258. As this is a small file there follows the 4-byte hash 45C6AA5D followed in turn by the 2 byte hash 0339 on the first 262,144 bytes. This is then followed by 7 big-endian numbers and then the URL of the cached file.
|Last Fetched date||54DDEFE3||13/02/2015 12:36:51|
|Expiry Date||54E13bA3||16/02/2015 00:36:51|
The remaining bytes comprise a varying number of tuples made up of name value pairs, i.e. pairs of strings as seen in the screenshot below. These strings are written to the tuples table in the name and value columns as appropriate.
Examining the cache
FirefoxCache2ToSQlite creates 2 different SQLite tables. The first contains the fixes tables and has the following schema:
CREATE TABLE cache2 (
filename text primary key,
As determined by the primary key filed, there can be only one entry for each filename. The file size is the size of the cached file excluding any metadata, the file BLOB field contains the data for the cached file – The Forensic Browser for SQLite will allow you to view this as hex, the text of for images as a picture. The remaining fields are as above and the definitions are outside the scope of this article.
The tuples table has the following schema
CREATE TABLE tuples (
The filename field in this table is a not a primary key and there will be more than one entry for each corresponding entry on the cache2 table. Each entry will have a filename field so a join can be created in the main cache2 table. The following name and value columns contain the associated data for each tuple.
The following screenshot shows three of the tuples for the file with name ending in 7d62, these tuples are security-data, request-method and response-head along with the associated data for each of them.