Tuesday, September 13, 2011

Setting up snat and dnat with iptables on single NIC server

Objective: on a single NIC server, configure snat and dnat to totally change source ip and destination ip. When ssh-ing into ip at port 222,> then it will NAT the traffic as>
Environment: CentOS 5.4 server with single NIC, eth0:, eth0:0, eth0:1

1. Enable IP forwarding - most important, otherwise everything will fail. Even on single NIC server, as long as you are using snat and dnat
ehco 1 > /proc/sys/net/ipv4/ip_forward

2. /etc/sysconfig/iptables
-A PREROUTING -s -d -p tcp -j DNAT --dport 222 --to-destination
-A POSTROUTING -s -d -p tcp --dport 22 -j SNAT --to-source

:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp -s -d --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited

3. other usages for iptables NAT
If you are going to use tomcat to serve port 80, you can't do so without running tomcat as root, but you can use iptables to listen at port 80 while keeping tomcat to listen at unprivileged port 8080:

iptables -t nat -A PREROUTING -d -p tcp -j DNAT --dport 80 --to-destination
iptables -t nat -A PREROUTING -d -p tcp --dport 80 -j REDIRECT --to-ports 8080

4. Notes:

a. only after prerouting , it will be tested by forwarding rules.
b. iptables package check sequence:

How to use MRTG to generate web traffic for a router

Objective: use MRTG to generate web traffic for a router
Environment:  CentOS 4 and cisco 2501 router


1. Enable SNMP server for cisco router and set the bandwidth for the serial interface
ssh/telnlet into the router with pasword
conf t
snmp-server community public ro

interface serial 0/0
#bandwidth 2048

2. Make sure the udp 161 port is not blocked on cisco router

3. Install MRTG (either through rpm or manual install)

# yum install mrtg

4. Go to MRTG website documention - unix guide at 
use the first example command to generate mrtg.cfg

 cfgmaker --global "WorkDir: /home/tobi"           \
          --global "Options[_]: growright,bits"    \
          --ifref=ip                               \
          public@router.place.xyz > mrtg.cfg

note: replace workdir, remove --ifref=ip line.
You might use this command:
/usr/bin/cfgmaker --global 'WorkDir: /var/www/html/mrtg' --global 'Options[_]: bits,growright' --output /etc/mrtg/mrtg.cfg public@

5. setup cronjob (if you install MRTG through rpm , no need, because there's a file under /etc/cron.d/mrtg already, run rpm -qil mrtg to check which are the files installed by rpm)

6. Access the serial port MRTG graphic

note: DO NOT configure cronjob for mrtg again if you install mrtg through yum install rpm command, otherwise, it will conflict, you will always receive warning email saying 'cannot rename ....' or ' cannot find primary log file...' etc

General Linux server network performance guide

Environment: Linux web server serves browser client and Database server on the local LAN serves the application server which is also on the same LAN
Objective: maximum the network performance which is one of the 4 performance bottlenecks(CPU,Memory,Storage and Network I/O)


1. net.core.wmem_default(/proc/sys/net/core/wmem_default) and net.core.rmem_default(/proc/sys/net/core/rmem_default) (the following settings are also recommended by Oracle 11gR1 installation)

net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
For Oracle database, it's not recommended to configure net.ipv4.tcp_rmem and net.ipv4.tcp_wmem.
as stated on metalink.

2. net.core.netdev_max_backlog (/proc/sys/net/core/netdev_max_backlog), default is 1000 in Linux RHEL 5 kernel 2.6
set maximum number of incoming packets that will be queued for delivery to the device queue.
3. net.core.somaxconn(/proc/sys/net/core/somaxconn), default is 128
maximum accept queue backlog that can be specified via the listen() system call. or the number of pending connection requests.
4. optmem_max (/proc/sys/net/core/optmem_max)
maximum initialization size of socket buffers, expressed in bytes.
increase this  
TCP options:
5. net.ipv4.tcp_window_scaling (/proc/sys/net/ipv4/tcp_window_scaling), enable
6. disable net.ipv4.tcp_sack(/proc/sys/net/ipv4/tcp_sack)
on LAN, disabling this tcp_sack can actually improve performance.
when tcp_sack is disabled, you should also disable 7 and 8
7. net.ipv4.tcp_dsack(/proc/sys/net/ipv4/tcp_dsack)
8. net.ipv4.tcp_fack(/proc/sys/net/ipv4/tcp_fack)
9. net.ipv4.tcp_max_syn_backlog(/proc/sys/net/ipv4/tcp_max_syn_backlog)
controls the length of the tcp syn queue for each port. If client experience failures
connecting to busy servers, this value should be increased.
10. net.ipv4.tcp_synack_retries(/proc/sys/net/ipv4/tcp_synack_retries) set to 3
controls the number of times kernel tries to resend a response to an incoming syn/ack segments
11. net.ipv4.tcp_retries2 (/proc/sys/net/ipv4/tcp_retries2) set to 5
controls the number of times kernel tries to resend data to a remote host with which it has an established connection.
12. net.ipv4.tcp_max_tw_buckets (/proc/sys/net/ipv4/tcp_max_tw_buckets)
increase this to double value. 
13. net.ipv4.tcp_orphan_retries (/proc/sys/net/ipv4/tcp_orphan_retries)  set to 0
14. net.ipv4.tcp_fin_timeout set to 30
15. net.ipv4.tcp_keepalive
16. ip_local_port_range (net.ipv4.ip_local_port_range)
1024 65000
17. net.ipv4.tcp_window_scaling = 1
18. net.ipv4.tcp_timestamps = 1 
Partitions and File system performance:
1. database raid:
For oracle database server hard disk raid. Use raid1 for redo log, archivelog,
including flash recovery area archivelog, temporary tablespace.
use raid1+0 for database files

2. for partition on individual hard disk, the first partition for /boot, the second is 
for swap, the third is for /var, the fourth is for /usr, the last is for /home and /
for other partitions, the first partition is at the outer side of the hard disk which 
is much faster then the inside. outsider partition can be accessed faster than insider ones.
3. add 'noatime' for those often accessed partitions in /etc/fstab.
4. switch to another I/O scheduler 
root (hd0,0) kernel /vmlinuz-2.6.18-8.el5 ro root=/dev/sda2 elevator=deadline
5. swap partition size:
  RAM               Swap Space
  1 GB - 2 GB       1.5 times the size of RAM
  2 GB - 8 GB       Equal to the size of RAM
  more than 8GB     0.75 times the size of RAM
6. use hugepage and ramfs to improve performance.

Linux bash scripting comman usage guide

Objective: When you need to write bash script, you need to follow certain steps, I've summarized some tips which I've been using for the past. 

Parameter and print out script usage:
The first thing after #!/bin/sh line should be usage for your script, especially when your script needs the parameters:

if  [ $# -eq 1 ];then echo "$0 username";exit 1;fi

note: your script requires username as command parameter, so above line will force you to give one parameter after the command.

Path definition:
You should export PATH variables first like this:
export PATH=/usr/bin:/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin

Common tips in the script:
  • check command return code 
if [ $? -ne 0 ];then

  • while loop to read each line from /path/to/file and process it
while read line
echo $line
done < /path/to/file

or while IFS=\| read left right
echo $left $right
done < /path/to/filename

# another method
LINES=`cat /path/to/file | wc -l`
while [ $i -le $LINES ]
iLINE=`sed -n $i'p' /path/to/file`
echo $iLINE

  • specify return value
if [ $? -eq 0 ];then
return 0
return 2
  •  seq command usage in for loop 
 for i in `seq -w 1 10`;do echo $i;done
01 01 03 04 05 06 07 08 09 10

  • Always use "bash -n scriptname" to verify if your script got any syntax error.
  •  check if the file size is 0
if [ ! -s /path/to/filename ];then commands ;fi
note: if the /path/to/file filesize is not zero, then run commands

  • use mktemp and mkdtemp 
You can use mktemp and mkdtemp to create temporary file or directorie variables
# TMPFILE1=`mktemp`
# TMPDIR1=`mktempdir`
  • how to let ls function as cd
ls () { builtin cd ; }

note: how to let ccd function as cd;pwd;foo

alias foo=`echo testing builtin`
ccd () { builtin cd "$1";pwd;foo; }

so ccd /tmp command will cd to /tmp, then print current working directory and print string 'testing builtin' 

common tips:
  1. batch rename file
rename 1.txt and 2.txt to 1.bat and 2.bat 

ls *.txt | sed 's#\(.*\).txt#mv \1.txt \1.bat#'  | sh 
    2.  use bc to calculate from CLI

# echo "scale=2;34359730176/1024/1024/1024" | bc
# echo "ibase=10;obase=16;10" | bc
# echo "ibase=16;obase=A;A" | bc

  3.  force root user to run script
if [ "$(id -u)" != "0" ]; then
echo "Only root can run this script"
exit 1

How to let users to change OpenLDAP password themselves through Linux CLI

Environment: OpenLDAP on Linux (CentOS, Fedora, Redhat or OEL), already configured the userpassword attribute.
Objective:  to let users to change their LDAP userpassword attribute themselves.

1.  configure access control part in slapd.conf
access to attr=userPassword
        by self write
        by anonymous auth
        by dn="cn=Manager,dc=dev,dc=domain,dc=com" write
        by * none

# note: above 'by self write' and 'by anonymous auth' attibutes
are very important, otherwise the users cannot change password by
access to attr=proxyAccess
        by self read
        by dn="cn=Manager,dc=dev,dc=domain,dc=com" write
        by * none

access to *
        by dn="cn=Manager,dc=dev,dc=domain,dc=com" write
        by users read
2.  user ldapmodify to change it.
  • Method 1:  use ldapmodify with Manager DN
ldapmodify -x -H ldap://  -D 'cn=Manager,dc=dev,dc=domain,dc=com' -W -f jephe.ldapmodify

[root@mars openldap]# more jephe.ldapmodify
dn: uid=jephe,ou=people,dc=dev,dc=domain,dc=com
replace: userpassword
userpassword: {MD5}risfylFZSeXVT7IrjtlVdQ==

You can use command 'slappasswd -h {MD5}' to generate userpassword line above
New password: testing
Re-enter new password: testing

  • Method 2 :  use ldapmodify with user own DN
ldapmodify -x -H ldap:// -D 'uid=jephe,ou=people,dc=dev,dc=domain,dc=com' -W -f jephe.ldapmodify
Enter LDAP Password:
modifying entry "uid=jephe,ou=people,dc=dev,dc=domain,dc=com"

3. use ldappasswd to change it
  • Method 3:  use ldappasswd with Manager DN
ldappasswd -x  -D cn=Manager,dc=dev,dc=domain,dc=com -w password  -s password uid=jephe,ou=People,dc=dev,dc=domain,dc=com
Result: Success (0)

  • Method 4: use ldappasswd with user own DN
ldappasswd -x  -D uid=jephe,ou=People,dc=dev,dc=domain,dc=com -w abcd1234  -s 12345 uid=jephe,ou=People,dc=dev,dc=domain,dc=com
Result: Success (0)