Linux Storage Guide

© Dennis Leeuw dleeuw at made-it dot com
Last updated: 29 Feb 2012
License: GPL

1. Introduction

System usage / /home iSCSI  
Logical Volume(s) /dev/vg-1/lv-1 /dev/vg-1/lv-2 /dev/vg-1/lv-3 unused
Volume Group(s) /dev/vg-1
Physical Volume(s) /dev/md0 /dev/sdc1 /dev/sdd1
/dev/sda1 /dev/sdb1
iSCSI   iqn...-1 iqn...-2 iqn...-3
Hardware SCSI disk NAS

1.1 SAN vs. NAS

Let us first look at the terms SAN and NAS. SAN stands for Storage Attached Network and NAS for Network Attached Storage. A NAS is often refered to when storage is provided to a network like the Microsoft™ shares that are offered through SMB or CIFS, but also NFS shares.

SANs are disks that are offered from a network to a server. This is most often an iSCSI network or a Fibre Channel network.

The line between a SAN and a NAS was when the terms where invented more clear then they are now. An iSCSI share can be offered to the network and a CIFS share can be provided to a server. The most clear destinction between the two is still that a share offered by a NAS can be mounted by more then one host at the same time. Some kind of locking prevents access problems to files, while a share offered by a SAN can only be used by a single host in read/write mode.

2. The Filesystem Hierarchy


5. Some real file systems

5.1. A non-journaling filesystem example: Ext2

5.2. Commands

When formatting a device for an ext2 filesystem you can use the -b option to tell the system the blocksize:

mke2fs -b 4096 <device>

To see the blocksize of a certain filesystem use:

stat -f [<device>|<mount-point>]
dumpe2fs -h <device>

5.2. A journaling filesystem example: Ext3

10. Backup

It's worth nothing that you can store all your data on your server, when a simple crash of a disk destroys it all. So backups are needed. But there is more to back up then just backing up data so you can restore it in case of an emergency. Another scenario is that you might want to move data from your primary storage disks to tape, because you do not need them online, but can not throw the data away. So you store the data on cheaper storage, like tapes. The last backup scenario is to back up data where you want to be able to retrieve older versions of a single file. Meaning you do some version control on a document by making for example daily copies, so you can later on always go back to an earlier version.

All these different scenarios have an impact on how you do your back ups. Another important aspect is the safety of the backed up data. If for example your back up machine is next to the machine holding the data you have a copy, but when a fire breaks out it is of no use, because most likely both machines will be lost.

There are different solutions to this problem, like in the case that you do back ups to tape, you could move the tape with the back up off site, meaning to restore the data after a fire you might need to buy a new tapestreamer, but your data is safe. You could also decide to place the entire tapestreamer off site and do your back up across the network, or even across the Internet. It all depends on the amount of data, and how much security you need, and how much money you are will to spent.


10.1 Before you make backups

The design of your backup system is dependant on your local situation. The simplest form of backup is to copy your data to another harddisk. Before you you act it is always good to think first. This section discusses some points to think about before diving into a backup design.

What might go wrong

The first thing to realize is the possible failure scenarios, and probably more importantly how to avoid them. A couple of examples:

Of course there are always cases where you can not prevent disaster. There is, for example, no scenario to prevent user stupidity, where a user just accidentally removes some very important data that he needs now!

Recovering from disaster

If disaster strikes it is good to have a plan according to which you can recover. Important to note is that the plan should be accessable after the disaster, so an online version might not be the best solution. In this plan the following points should be addressed:

10.2 Backup Design

Notes before you start your design:

Something about filesystems

Windows users are used to the archive bit, which can be set if a backup is needed, or unset if the backup is done. GNU/Linux and Unix in general have another scheme for this. The ext2 and ext3 file systems provide you with a atime, ctime or mtime. The atime is the time the file was last accessed (read), the ctime is the time the inode was last changed and the mtime is the time the file was last modified. The last two probably need a little bit more explanation. When the mtime changes, the ctime changes. Meaning when a file is written to both change, but when you change the rights on a file, or change the ownership, which means you only adjust the information that is stored in the inode only the ctime changes. Which means for backing up files back up software will use the ctime. (see stat and touch)

The Emergency Backup

The emergency-backup consists of a full backup followed by a couple of incrementals, after which this cycle repeats itself.


Requirements are:

Questions that need to be answered:

Archive Backup

The archive-backup consists only of a full backup of the data after which the data can be removed from the primary disks. To make sure this data is redundant it needs to be written to two tapes, which should be stored separate from one another.



Question that need to be answered:

10.4 About tapes


If you have dropped a tape cardridge, restore its content and rebackup. Since dropped tapes have a shorter lifespan.

To maximize tape life, tape cartridges should be kept in an atmosphere free of contaminating dust particles and corrosive gases or chemicals. Cartridges should always be acclimated to the operating environment prior to mounting the cartridge on the drive. A minimum of 24 hours of acclimation time is generally recommended to make sure the cartridge is at the same humidity and temperature as the drive for newly received tapes.

The National Bureau of Standards publication, Care and Handling of Computer Magnetic Storage Media, recommends that magnetic tape be stored at 65 +/- 3 degrees Fahrenheit and 40% +/- 5% Relative Humidity.

National Media Laboratory = NML

Studies by the NML indicate that magnetic media, properly cared for, should have a lifetime which equals or exceeds that of the recording technology (10 to 20 years).


tar mt st

The shoeshine effect

Take notes from:,294698,sid181_gci1295968,00.html



10.5 About optical disks

common optical media formats have a storage life of 30 years or more

Store discs in a cool, dry environment away from direct light. Discs stored between 23 degrees F (-5 degrees C) and 86 degrees F (30 degrees C) can last up to 100 years

Do not leave the disc in direct sunlight or in a hot, humid environment--like your car on a summer day--as these conditions could warp and damage the disc

Do not allow moisture to condense on the disc.

Appendix A harddisks

A.1. IDE

hdparm -i gives you more information about the hardware, like disk manufacturer, serial number and disk geometry.

A.1.1 SATA

Serial ATA is ATA over serial lines. SATA uses smaller cables, then parallel ATA, which leaves more room and thus better cooling in the computer housing. SATA also does not use the master/slave setup anymore and is hotplugable.


A.2.1. SAS

Serial-attached SCSI has thinner cables, less bulky connectors and allows for longer cables. The hardware is cheaper and less prone to crosstalk.

Appendix D Fibre Channel


Appendix X References