Search

Howto – local and remote snapshot backup using rsync with hard links

Introduction

Your Linux servers are running smoothly? Fine.
Now if something incidentally gets wrong you need also to prepare an emergency plan.
And even when everything is going fine, a backup that is directly stored on the same server can still be useful:

  • for example when you need to see what was changed on a specific file,
  • or if you want to find the list of files that were installed or modified after the installation of an application

This article is then intended to show you how you can set up an open-source solution, using the magic of the famous ‘rsync’ tool and some shell-scripting, to deploy a backup system without the need of investing into expensive proprietary software.
Another advantage of a shell-script is that you can easily adapt it according to your specific needs, for example for you DRP architecture.

The proposed shell-script is derivate from the great work of Mikes Handy’s rotating-filesystem-snapshot utility (cf. http://www.mikerubel.org/computers/rsync_snapshots).
It creates backups of your full filesystem (snapshots) with the combined advantages of full and incremental backups:

  • It uses as little disk space as an incremental backup because all unchanged files are hard linked with existing files from previous backups; only the modified files require new inodes.
  • Because of the transparency of hard links, all backups are directly available and always online for read access with your usual programs; there is no need to extract the files from a full archive and also no complicate replay of incremental archives is necessary.

Read more

Git and GitHub: Tooling your Software Workflow and Development Life Cycle

History of Software Engineering

Since the 1940s when software engineering started close on hardware level until today on various technologies like embedded systems, web and mobile apps the demand on development process has increased tremendously. As a result in the 1980s the cost of owning and maintaining was twice as expensive as developing software.

Read more

Howto – bash audit / command logger

Introduction

Having a complete history of all typed commands can be very helpful in many scenarios:

  • when several administrators work together on the same server and need to know what was done previously
  • when someone need to redo an older sequence of commands or to understand an undocumented maintenance process
  • for troubleshooting or forensic analysis, by crosschecking the date of an event or of a file with the commands executed at that date

The standard ‘.bash_history’ file of the shell is unfortunately not written on disk in the case of a crash and it may be deleted by the user.
Another problem is that when many shell sessions are running concurrently, their logging will only occur when they are closed, therefore the commands of the history will not appear in their chronological order.
Furthermore, ‘.bash_history’ will not include essential information like the ‘working directory’ of the command; and by default the repetition or re-edition of commands will not be logged, too.

Read more