It covers the main differences between ext3 and ext4 with a focus on filesystem consistency. This article was the initial motivation of this blog series, because many engineers are unaware that the standard option of ext4 (delalloc) is dangerous for their data!
This blog article details how a brutal interruption of a write operation can corrupt a whole filesystem and how a journal provides a protection against such incomplete writes.
This article aims to give a clear representation of the relation between filenames and inodes, to explain what is an inode and which differences exist between a hard link and a symbolic link.
2.1 Filename as Hardlink to Inode
On ext3 and ext4 (and some other filesystems) a file is stored
It explains how files are split into blocks and how the filesystem acts as an abstraction layer between files (seen by applications) and blocks (physically written on disks). It also says why disks are called block devices.
1.1 Splitting Up Files Into Blocks
A block device is a storage media, like a hard disk. You can use ‘blkid’ and ‘fdisk -l’ to show a list of block devices attached to your system.
In the following example, ‘sda’ is a disk with 2 partitions ‘sda1′ and ‘sda2′, and ‘sr0′ is a compact disc.
A filesystem can be created on a partition and also on a whole partitionless disk, as both partition and disk are valid block devices.
blkid /dev/sda1: UUID="5b0e4487-b5aa-4186-81fd-f561308f4cf1" TYPE="swap" /dev/sda2: UUID="ec851750-9854-4227-8898-d5695dbd297e" TYPE="ext3" /dev/sr0: LABEL="VBOXADDITIONS_4.2.16_86992" TYPE="iso9660" fdisk -l Disk /dev/sda: 12.9 GB, 12884901888 bytes 255 heads, 63 sectors/track, 1566 cylinders, total 25165824 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000be060 Device Boot Start End Blocks Id System /dev/sda1 2048 786431 392192 82 Linux swap / Solaris /dev/sda2 * 786432 25165823 12189696 83 Linux
In order to use the hardware efficiently and to increase the addressing limit the disk read and write operations are not processing the data byte by byte. Instead these operations are using a larger unit called a block (also called a sector), which is usually 512 Bytes.
This blog series aims to be a reference guideline by collecting useful knowledge and tutorials about Linux filesystems, mainly about ext3 and ext4.
Because, as systems engineers/administrators, when we need to manually perform disk maintenance on a server, we know that a tiny mistake may have large consequences.
“Trial-and-Error” is sometimes a good way to learn, and sometimes not. For sensitive topics like internet security or filesystem operations, it is often better to understand the theory based on reliable sources before trying; in this way we can avoid putting the data integrity at unnecessary risk.
Your Linux servers are running smoothly? Fine.
Now if something incidentally gets wrong you need also to prepare an emergency plan.
And even when everything is going fine, a backup that is directly stored on the same server can still be useful:
- for example when you need to see what was changed on a specific file,
- or if you want to find the list of files that were installed or modified after the installation of an application
This article is then intended to show you how you can set up an open-source solution, using the magic of the famous ‘rsync’ tool and some shell-scripting, to deploy a backup system without the need of investing into expensive proprietary software.
Another advantage of a shell-script is that you can easily adapt it according to your specific needs, for example for you DRP architecture.
The proposed shell-script is derivate from the great work of Mikes Handy’s rotating-filesystem-snapshot utility (cf. http://www.mikerubel.org/computers/rsync_snapshots).
It creates backups of your full filesystem (snapshots) with the combined advantages of full and incremental backups:
- It uses as little disk space as an incremental backup because all unchanged files are hard linked with existing files from previous backups; only the modified files require new inodes.
- Because of the transparency of hard links, all backups are directly available and always online for read access with your usual programs; there is no need to extract the files from a full archive and also no complicate replay of incremental archives is necessary.