Sunday, January 22, 2012

Linux NBD Tutorial: Network Block Device Jumpstart Guide


Network block devices are used to access remote storage device that does not physically reside in the local machine. Using Network Block Device, we can access and use the remote storage devices in following three ways on the local machine:
  1. SWAP
  2. File System
  3. RAW
NBD presents a remote resource as local resource to the client. Also, NBD driver makes a remote resource look like a local device in Linux, allowing a cheap and safe real-time mirror to be constructed.
You can also use remote machine storage area as local machine swap area using NBD.

To setup the NBD based file system, we need a nbd-server (on remote machine, where we like to access/create the content) and nbd-client (on local machine, where we like to access the remote storage device locally).

I. NBD Server Side Configuration Steps

1. Install nbd-server

If you working on Debian flavor, get the nbd-server through apt-get.
# apt-get install nbd-server

2. Create a file content

Create a file using dd as shown below.
# dd if=/dev/zero of=/mnt/dhini bs=1024 count=36000
Use mke2fs to make the /mnt/dhini as a filesystem.
# mke2fs /mnt/dhini
When you try to make /mnt/dhini as ext2 file system, you may get a warning message as show below. Press y to continue.
/mnt/dhini is not a block special device.
Proceed anyway? (y,n) y

3. Start the NBD Server Daemon

Syntax: nbd-server  port-number   filename/filesystem

# nbd-server 1043 /mnt/dhini
You can also run the nbd-server on multiple ports as shown below.
# nbd-server 1043 1077 1076 /mnt/dhini
You can also specify the timeout to make the server to run N idle seconds

II. NBD Client Side Configuration Steps

Perform the following steps in the client machine, where you like to access the remote storage device.

1. Install nbd-client

If you working on debian flavor, get the nbd-client through apt-get.
# apt-get install nbd-client

2. Using nbd-client create a filesystem on client machine

Syntax: nbd-client  server-ip  server-port#  filename/filesystem

# nbd-client 192.168.1.11 1043 /mnt/dhini

Starting NBD client process: Connecting...Negotiation: ..size = 36000KB
bs=1024, sz=36000
connected /mnt/dhini
Activating...
fsck 1.39-WIP (31-Dec-2005)
/mnt/dhini: Superblock last write time is in the future.  FIXED.
/mnt/dhini has gone 49710 days without being checked, check forced.
/mnt/dhini: |===============================| 56.0%
/mnt/dhini: |===============================| 100.0%
Once it gets to 100%, you will get the block device on your local macine on the same path.
$ls -lh /mnt/dhini
brw-r--r-- 1 root root 43, 0 2009-02-05 17:31 /mnt/dhini
If you face any issues during the NBD configuration process, you may also configure the nbd-server and nbd-client through dpkg-reconfigure.

III. Mount the File System on Client-side

# mkdir /rem_space
# mount /mnt/dhini /rem_space
Once mounted, you may get the directory with “lost+found”. You can start accessing the files and directories properly from this point.

IV. Get Client Changes on Server-side

Mount the nbd filesystem locally
# mount  -o loop /mnt/dhini /client_changes
If you are not using “-o loop” option, you may get the following error:
mount: /mnt/dhini is not a block device (maybe try `-o loop'?)
When you list the /client_changes, You will get all the client created files and directories properly.

V. Access Remote Storage as Local Swap Memory Area

Configuration On Server side:

1. Create a file

# dd if=/dev/zero of=/mnt/dhini bs=1024 count=16000

2. Instead of create a file in ext2 filesystem create it as swap file, using mkswap

# mkswap /mnt/dhini

3. Run the server daemon

# nbd-server 1043 /mnt/dhini

Configuration On Client side:

1. Get the filesystem as swap area

# nbd-client 192.168.1.11 1043 -swap  /mnt/dhini

2. Cross check using “cat /proc/swaps “. This will list the swap areas

$ cat /proc/swaps
Filename    Type       Size      Used    Priority
/dev/hda4   partition  650624     57732  -1
/mnt/dhini  partition  15992    0        -4

Overview Of PoE – Power Over Ethernet Concepts and Devices List


1. What is PoE – Power Over Ethernet?

Power over Ethernet (PoE) does exactly what it says — it carries power over ethernet cables. So, an ethernet cables caries both data and current together to operate devices like wi-fi access points. This is very helpful to install wi-fi access points in ceilings, where it is hard to find power outlets. Only smaller devices can be powered using the PoE, as it carries limited power over the ethernet.
[PoE Setup]
Fig 1: A Typical PoE Setup (Image From: PowerOverEthernet.com)

2. How does PoE works?

Following are the terminologies used in the PoE network:
  • PD – Powered Devices: PDs are end devices that can accept the power transmitted over Ethernet Cat-5 cable.
  • PSE – Power Source Equipment: PSEs add power to the ethernet cable.
  • Midspan device: These are power injectors that stands between the standard ethernet switch and the end device.
  • Endspan devices (PoE Switches): These are Ethernet Switches that includes the circuit to inject power to the ethernet cable.

Cat5 ethernet cable has 4 twisted pairs. Out of 4 pairs:
  • 2 pairs are used for data transfer,
  • 2 pairs are spare and not used.

There are two approaches to transfer power over ethernet.

Approach 1 uses data pairs:

The same 2 pairs used for data transfer, is used to carry power. The PSE injects power to the 2 data pairs (pin 1,2 and pin 3,6) via the center tap of the transformers without affecting the data transfer.
PoE Approach2
Fig 2: PoE Approach 1 uses data pairs to send power (Image From: PowerOverEthernet.com)

Approach 2 uses the spare pairs:

The 2 spare pairs that are not used, is used to carry power. It applies +ve voltage to pins 4 and 5 of ethernet cable. It applies -ve voltage to pins 7 and 8.
PoE Approach1
Fig 3: PoE Approach 2 uses spare pairs to send power (Image From: PowerOverEthernet.com)

3. What are the Benefits of PoE?

  • Portability: Install end-devices where it is hard to get power.  For example, you can install a wireless access point in the ceiling, where you cannot find power outlet.
  • Cost Savings: For a large scale deployment of wireless access point, you don’t need to install power outlet for the access points, which can be a huge cost saving.
  • Simplicity: You need to run only ethernet cable to the end devices, which eliminates the cable clutter.
  • Safer: No AC power is involved, as you don’t need to install A/C power outlet for the end-devices.
  • Easy Maintenance: You can restart the end-device remotely. No need to press the power button to reset the equipment.  You can easily move the end-devices wherever you can lay a LAN cable.

4. Sample List of Equipments that uses PoE.

PoE Adapters:

PoE Wireless Access Point:

PoE Switch:

PoE Network Surveillance Camera:

PoE VoIP Phone:


PowerOverEthernet.com site maintains a Huge List of all PoE products, including a PoE Analog Clocks and PoE Gas Detector.

Additional PoE References

Safe Reboot Of Linux Using Magic SysRq Key


If you are working on kernel development, or device drivers, or running a code that could cause kernel panic, SysRq key will be very valuable. The magic SysRq key is a key combination in the Linux kernel which allows the user to perform various low level commands regardless of the system’s state.

It is often used to recover from freezes, or to reboot a computer without corrupting the filesystem. The key combination consists ofAlt+SysRq+commandkey. In many systems the SysRq key is the printscreen key.

First, you need to enable the SysRq key, as shown below.
echo "1" > /proc/sys/kernel/sysrq

List of SysRq Command Keys

Following are the command keys available for Alt+SysRq+commandkey.
  • ‘k’ – Kills all the process running on the current virtual console.
  • ‘s’ – This will attempt to sync all the mounted file system.
  • ‘b’ – Immediately reboot the system, without unmounting partitions or syncing.
  • ‘e’ – Sends SIGTERM to all process except init.
  • ‘m’ – Output current memory information to the console.
  • ‘i’ – Send the SIGKILL signal to all processes except init
  • ‘r’ – Switch the keyboard from raw mode (the mode used by programs such as X11), to XLATE mode.
  • ‘s’ – sync all mounted file system.
  • ‘t’ – Output a list of current tasks and their information to the console.
  • ‘u’ – Remount all mounted filesystems in readonly mode.
  • ‘o’ – Shutdown the system immediately.
  • ‘p’ – Print the current registers and flags to the console.
  • ’0-9′ – Sets the console log level, controlling which kernel messages will be printed to your console.
  • ‘f’ – Will call oom_kill to kill process which takes more memory.
  • ‘h’ – Used to display the help. But any other keys than the above listed will print help.
We can also do this by echoing the keys to the /proc/sysrq-trigger file. For example, to re-boot a system you can perform the following.
echo "b" > /proc/sysrq-trigger

Perform a Safe reboot of Linux using Magic SysRq Key

To perform a safe reboot of a Linux computer which hangs up, do the following. This will avoid the fsck during the next re-booting. i.e Press Alt+SysRq+letter highlighted below.
  • unRaw (take control of keyboard back from X11,
  • tErminate (send SIGTERM to all processes, allowing them to terminate gracefully),
  • kIll (send SIGILL to all processes, forcing them to terminate immediately),
  • Sync (flush data to disk),
  • Unmount (remount all filesystems read-only),
  • reBoot.

Tripwire Tutorial: Linux Host Based Intrusion Detection System


Tripwire is a host based Intrusion detection system for Linux. Tripwire monitors Linux system to detect and report any unauthorized changes to the files and directories. Once a baseline is created, tripwire monitors and detects, which file is added, which file is changed, what is changed, who changed it, and when it was changed. If the changes are legitimate, you can update the tripwire database to accept these changes.

Also, for monitoring solution, please refer to all our previous articles on Nagios

This step by step instruction guide explains how to install and configure open source version of tripwire.

1. Download Tripwire

Download the latest tripwire open source version from tripwire sourceforget project website. Extract the tripwire source code to the /usr/src directory as shown below.
# cd /usr/src
# wget http://internap.dl.sourceforge.net/sourceforge/tripwire/tripwire-2.4.1.2-src.tar.bz2
# bzip2 -d tripwire-2.4.1.2-src.tar.bz2
# tar xvf tripwire-2.4.1.2-src.tar

2. Install Tripwire

Use the prefix option as shown below to specify the installation directory. In this example, I’ve installed tripwire under /opt/tripwire. During make install, it will prompt you for various user inputs, that are highlighted in red below.
# cd tripwire-2.4.1.2-src

# ./configure --prefix=/opt/tripwire

# make

# make install

make[3]: Entering directory `/usr/src/tripwire-2.4.1.2-src'
prefix="/opt/tripwire" sysconfdir="/opt/tripwire/etc" \
        path_to_vi="/bin/vi" path_to_sendmail="/usr/sbin/sendmail" \
        ./install/install.sh
Installer program for: Tripwire(R) 2.4 Open Source
LICENSE AGREEMENT for Tripwire(R) 2.4 Open Source
Please read the following license agreement.  You must accept the
agreement to continue installing Tripwire.
Press ENTER to view the License Agreement.
[Note: Press enter key as instructed to view the license]

Please type "accept" to indicate your acceptance of this
license agreement. [do not accept] accept
[Note: Type accept to accept the license]

This program will copy Tripwire files to the following directories:
        TWBIN: /opt/tripwire/sbin
        TWMAN: /opt/tripwire/man
     TWPOLICY: /opt/tripwire/etc
     TWREPORT: /opt/tripwire/lib/tripwire/report
         TWDB: /opt/tripwire/lib/tripwire
 TWSITEKEYDIR: /opt/tripwire/etc
TWLOCALKEYDIR: /opt/tripwire/etc
CLOBBER is false.
Continue with installation? [y/n] y
[Note: Press y to continue the installation]

The Tripwire site and local passphrases are used to
sign a variety of files, such as the configuration,
policy, and database files.
(When selecting a passphrase, keep in mind that good passphrases typically
have upper and lower case letters, digits and punctuation marks, and are
at least 8 characters in length.)
Enter the site keyfile passphrase:
Verify the site keyfile passphrase:
[Note: Assign a passphrase for site keyfile.]

Generating key (this may take several minutes)...Key generation complete.
(When selecting a passphrase, keep in mind that good passphrases typically
have upper and lower case letters, digits and punctuation marks, and are
at least 8 characters in length.)
Enter the local keyfile passphrase:
Verify the local keyfile passphrase:
[Note: Assign a passphrase for local keyfile.]

Creating signed configuration file...
Please enter your site passphrase:
Wrote configuration file: /opt/tripwire/etc/tw.cfg
[Note: Enter the site passphrase.]

Creating signed policy file...
Please enter your site passphrase:
Wrote policy file: /opt/tripwire/etc/tw.pol
[Note: Enter the site passphrase]

The installation succeeded.
  • Site passphrase will secure the tw.cfg tripwire configuration file
    and tw.pol tripwire policy file. You have to assign a site passphrase
    even for a single instance tripwire.
  • Local passphrase will protect tripwire database and report files.

3. Initialize Tripwire Database

For the first time use, you should initialize the tripwire database as shown below.
# cd /opt/tripwire/sbin/

# ./tripwire --init

Please enter your local passphrase:
Parsing policy file: /opt/tripwire/etc/tw.pol
Generating the database...
*** Processing Unix File System ***
The object: "/sys" is on a different file system...ignoring.
### Warning: File system error.
### Filename: /cdrom
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /floppy
### No such file or directory
### Continuing...
### Warning: File system error.
### Filename: /initrd
### No such file or directory
### Continuing...
### Warning: File system error.
Wrote database file: /opt/tripwire/lib/tripwire/prod-db-srv.twd
The database was successfully generated.

4. Modify Tripwire Policy File

As shown above, during the tripwire database initialization, it may display “No such file or directory” error message for some of the default files mentioned in the tripwire policy file. If your system don’t have those files, edit the policy file and comment those entries.

For example, modify the /opt/tripwire/etc/twpol.txt tripwire policy file and comment out /cdrom and /floppy as shown below.
(
  rulename = "OS Boot Files and Mount Points",
)
{
  /boot                         -> $(ReadOnly) ;
#  /cdrom                        -> $(Dynamic) ;
#  /floppy                       -> $(Dynamic) ;
  /mnt                          -> $(Dynamic) ;
}

Using the tripwire policy files you can define the directories and files that needs to be monitored for the changes. You can also be more granular and specify the file attributes that should be either monitored or ignored.

Following are some of the UNIX system properties that are monitored by tripwire.

  • File addition, deletion and modification
  • File permissions and properties
  • Access timestamp
  • Modification timestamp
  • File type and file size
  • User id of owner and group id of owner
  • Hash checking: CRC-32, POSIX 1003.2 compliant 32-bit Cyclic Redundancy Check; MD5, the RSA Security Message Digest Algorithm; SHA, part of the SHS/SHA algorithm; HAVAL, a strong 128-bit signature algorithm

5. Update Tripwire Policy File

Once you’ve modified the policy file, it needs to be updated as shown below.
# ./tripwire --update-policy --secure-mode low ../etc/twpol.txt

Parsing policy file: /opt/tripwire/etc/twpol.txt
Please enter your local passphrase:
Please enter your site passphrase:
======== Policy Update: Processing section Unix File System.

======== Step 1: Gathering information for the new policy.
The object: "/sys" is on a different file system...ignoring.

======== Step 2: Updating the database with new objects.

======== Step 3: Pruning unneeded objects from the database.
Wrote policy file: /opt/tripwire/etc/tw.pol
Wrote database file: /opt/tripwire/lib/tripwire/prod-db-srv.twd
Note: if any files has been modified from the time you’ve done the tripwire initialization until the tripwire update policy, they will be listed under the “Step 1: Gathering information for the new policy” output of the above command.
### Warning: Policy Update Changed Object.
### An object has been changed since the database was last updated.

### Object name: Conflicting properties for object
### /u01/app/oracle/oradata/dbfiles/prod01.dbf
### > Modify Time
### > CRC32
### > MD5

6. Check for any changes to the files and update tripwire database.

Once the tripwire setup is completed, you should regularly perform checks to find out what files where added or modified from the last time the tripwire database was updated. You can perform this check interactively from command line as shown below.
# ./tripwire --check --interactive

Parsing policy file: /opt/tripwire/etc/tw.pol
*** Processing Unix File System ***

Performing integrity check...
Wrote report file:
/opt/tripwire/lib/tripwire/report/prod-db-srv-20081204-114336.twr

This will automatically open the following tripwire report file in the vi, where you can review all the files that has been added or modified to the system. As shown below, the “Added” and “Modified” files will have a check mark in front of them, indicating that you are accepting these changes to be updated to the tripwire database.
===============================================================================
Report Summary:
===============================================================================
Host name:                    prod-db-srv
Host IP address:              192.168.1.10
Host ID:                      None
Policy file used:             /opt/tripwire/etc/tw.pol
Configuration file used:      /opt/tripwire/etc/tw.cfg
Database file used:           /opt/tripwire/lib/tripwire/prod-db-srv.twd
Command line used:            ./tripwire --check --interactive

Remove the "x" from the adjacent box to prevent updating the database
with the new values for this object.

Added:
[x] "/u01/app/oracle/diag/rdbms/proddb/proddb/trace/proddb_m000_11376.trc"
[x] "/u01/app/oracle/diag/rdbms/proddb/proddb/trace/proddb_m000_11376.trm"

Modified:
[x] "/u01/app/oracle/diag/rdbms/proddb/proddb/metadata/INC_METER_CONFIG.ams"
[x] "/u01/app/oracle/diag/rdbms/proddb/proddb/metadata/INC_METER_INFO.ams"

Added object name:  /u01/app/oracle/diag/rdbms/proddb/proddb/trace/proddb_m000_11376.trc

  Property:            Expected                    Observed
  -------------        -----------                 -----------
* Object Type          ---                         Regular File
* Device Number        ---                         2049
* Inode Number         ---                         12026017
* Mode                 ---                         -rw-r-----
* Num Links            ---                         1
* UID                  ---                         oracle (1082)
* GID                  ---                         oinstall (1083)
* Size                 ---                         837
* Modify Time          ---                         Sat 06 Dec 2008 10:01:51 AM PST
* Blocks               ---                         8
* CRC32                ---                         AYxMeo
* MD5                  ---                         AXSkOul8R/np0fQP4q3QLv

Modified object name:  /u01/app/oracle/diag/tnslsnr/proddb/listener/trace/listener.log

  Property:            Expected                    Observed
  -------------        -----------                 -----------
  Object Type          Regular File                Regular File
  Device Number        2049                        2049
  Inode Number         2295281                     2295281
  Mode                 -rw-r-----                  -rw-r-----
  Num Links            1                           1
  UID                  oracle (1082)               oracle (1082)
  GID                  oinstall (1083)             oinstall (1083)
* Size                 5851880                     5858608
* Modify Time          Sat 06 Dec 2008 09:58:53 AM PST
                                                   Sat 06 Dec 2008 11:39:56 AM PST
* Blocks               11456                       11472
* CRC32                ANdM8R                      CK+bWM
* MD5                  DCW84lCuD2YJOhQd/EuVsn      CV8BMvZNJB9KQBXAf5yRDY

Please enter your local passphrase:
Incorrect local passphrase.
Please enter your local passphrase:
Wrote database file: /opt/tripwire/lib/tripwire/prod-db-srv.twd

7. How to view the twr report file?

All the tripwire report files with *.twr extension are stored under /opt/tripwire/lib/tripwire/report directory. tripwire report file *.twr is not a text file, which you can view directly. In order to view the report, use twprint and convert the *.twr file to a readable text format as shown below.
# ./twprint --print-report --twrfile \
/opt/tripwire/lib/tripwire/report/prod-db-srv-20081204-114336.twr  > \
/tmp/readable-output.txt

8. Monitor Linux System Integrity Regularly

Add tripwire checking as a cron job to monitor and report any changes on an on-going basis. For example, add the following line to your crontab to execute tripwire check daily at 4:00 a.m.
# Tripwire Monitor process
00 4 * * * /opt/tripwire/sbin/tripwire  --check

9. Tripwire Configuration and Policy File Locations

Use twadmin to view the current tripwire policy files. Only partial output is shown below.
#./twadmin --print-polfile
@@section GLOBAL
TWDOCS="/opt/tripwire/doc/tripwire";
TWBIN="/opt/tripwire/sbin";
TWPOL="/opt/tripwire/etc";
TWDB="/opt/tripwire/lib/tripwire";
TWSKEY="/opt/tripwire/etc";
TWLKEY="/opt/tripwire/etc";
TWREPORT="/opt/tripwire/lib/tripwire/report";
HOSTNAME=prod-db-srv;
Use twadmin to get information about all the tripwire configuration files as shown below.
# ./twadmin --print-cfgfile
ROOT          =/opt/tripwire/sbin
POLFILE       =/opt/tripwire/etc/tw.pol
DBFILE        =/opt/tripwire/lib/tripwire/$(HOSTNAME).twd
REPORTFILE    =/opt/tripwire/lib/tripwire/report/$(HOSTNAME)-$(DATE).twr
SITEKEYFILE   =/opt/tripwire/etc/site.key
LOCALKEYFILE  =/opt/tripwire/etc/prod-db-srv-local.key
EDITOR        =/bin/vi
LATEPROMPTING =false
LOOSEDIRECTORYCHECKING =false
MAILNOVIOLATIONS =true
EMAILREPORTLEVEL =3
REPORTLEVEL   =3
MAILMETHOD    =SENDMAIL
SYSLOGREPORTING =false
MAILPROGRAM   =/usr/sbin/sendmail -oi -t