MySQL Storage Engines – an overview, their limitations and an attempt for comparison

Most of us are using Mysql database and majority don’t know how to choose the data base engines, what are the different types of storage engines available in mysql and how they differ from each other. In this article let me give you a brief idea about the Storage Engines and what are the limitations and where to use these various storage engines.

One of the greatest things about MySQL, other than being free, widely supported and fast, is the flexibility of choosing different storage engines for different tables. These storage engines act as handlers for different table types. Thus MySQL storage engines include both those that handle transaction-safe tables and those that handle non-transaction-safe tables along with many others. MySQL does this through their Pluggable Storage Engine Architecture.

MySQL Storage Engines Overview

To determine which storage engines your server supports, we use the SHOW ENGINES statement. The value in the Support column indicates whether an engine can be used. A value of YES, NO, or DEFAULT indicates that an engine is available, not available, or available and currently set as the default storage engine.

mysql> SHOW ENGINES\G
*************************** 1. row ***************************
Engine: MyISAM
Support: DEFAULT
Comment: Default engine as of MySQL 3.23 with great performance
*************************** 2. row ***************************
Engine: MEMORY
Support: YES
Comment: Hash based, stored in memory, useful for temporary tables
*************************** 3. row ***************************
Engine: InnoDB
Support: YES
Comment: Supports transactions, row-level locking, and foreign keys
*************************** 4. row ***************************
Engine: BerkeleyDB
Support: NO
Comment: Supports transactions and page-level locking
*************************** 5. row ***************************
Engine: BLACKHOLE
Support: YES
Comment: /dev/null storage engine (anything you write to it disappears)
*************************** 6. row ***************************
Engine: EXAMPLE
Support: YES
Comment: Example storage engine
*************************** 7. row ***************************
Engine: ARCHIVE
Support: YES
Comment: Archive storage engine
*************************** 8. row ***************************
Engine: CSV
Support: YES
Comment: CSV storage engine
*************************** 9. row ***************************
Engine: ndbcluster
Support: NO
Comment: Clustered, fault-tolerant, memory-based tables
*************************** 10. row ***************************
Engine: FEDERATED
Support: YES
Comment: Federated MySQL storage engine
*************************** 11. row ***************************
Engine: MRG_MYISAM
Support: YES
Comment: Collection of identical MyISAM tables
*************************** 12. row ***************************
Engine: ISAM
Support: NO
Comment: Obsolete storage engine

The listing shows the full list of available database engines.

There are a number of ways you can specify the storage engine to use. The simplest method, if you have prefer a particular engine type that fits most of your database needs then you can set the default engine type within the MySQL configuration file using the following commands(using the option storage_engine or when starting the database server (by supplying the –default-storage-engine or –default-table-type options on the command line).

More flexibility is offered by allowing you to specify the default storage engine to be used MySQL, the most obvious is to specify the engine type when creating the table:

CREATE TABLE mytable (id int, title char(20)) ENGINE = INNODB

You can also alter the storage engine used in an existing table:

ALTER TABLE mytable ENGINE = MyISAM

However, you should be careful when altering table types in this way, as making a modification to a type that does not support the same indexes, field types or sizes may mean that you lose data. If you specify a storage engine that doesn’t exist in the current database then a table of type MyISAM (the default) is created instead.

Before taking any decision about which engine we need to choose, first we need to think about the different core functionality provided by each engine that allow us to differentiate between them. We can divide up the core functionality into four areas; the supported field and data types, locking types, indexing and transactions. Some engines have unique functionality that can also drive your decision.

Field and Data Types

Although all of the engines support the common data types, i.e., integers, reals and character based storage, not all engines support other field types, particularly the BLOB (binary large object) or TEXT types. Other engines may support only limited character widths and data sizes.

These limitations, while directly affecting the information you store may also have a related effect to the types of searches you perform, or the indexes you create on that information. In turn, these differences can affect the performance and functionality of your application as you may have to make decisions about functionality based on the storage engine choice you make for the type of data you are storing.

Locking

Locking within database engines defines how access and updates to information are controlled. When an object in the database is locked for updating, other processes cannot modify (or in some cases read) the data until the update has completed.

Locking not only affects how many different applications can update the information in the database, it can also affect queries on that data. The reason for this is that the queries may be accessing data that may be being altered or updated. In general, such delays are minimal. The bulk of the locking mechanism is devoted to preventing multiple processes updating the same data. Since both additions (INSERT statements) and alterations (UPDATE statements) to the data require locking, you can imagine that multiple applications using the ame database can have a significant impact.

Locks are supported by different storage engines at different object levels, and these levels affect the concurrency of access to the information. Three different levels are supported, table locking, block locking and row locking. Table locking is most commonly supported and is the locking provided in MyISAM. It locks an entire table during an update. This will limit the number of applications that are updating a specific table to just one, and this can affect heavily used multi-user databases because it introduces delays into the update process.

Page level locking is used by the Berkeley DB storage engine and locks data according to the page (8Kb) of information that is being uploaded. When performing updates across a range of locations within the database, the locking is not a problem, but because adding rows involves locking the final 8Kb of the data structure, adding large numbers of rows, particularly of small data, can be a problem.

Row level locking provides the best concurrency; only individual rows within a table are locked, which means that many applications can be updating different rows of the same table without causing a lock situation. Only the InnoDB storage engine supports row level locking.

Indexing

Indexing can dramatically increase the performance when searching and recovering data from the database. Different storage engines provide different indexing techniques and some may be better suited for the type of data you are storing.
Some storage engines simply do not support indexing at all either because they use the indexing of the underlying tables (in the MERGE engine for example) or because the data storage method does not allow indexing (FEDERATED or BLACKHOLE engines).

Transactions

Transactions provide data reliability during the update or insert of information by enabling you to add data to the database, but only to commit that data when other conditions and stages in the application execution have completed successfully. For example, when transferring information from one account to another you would use transactions to ensure that both the debit from one account and the credit to the other completed successfully. If either process failed, you could cancel the transaction and the changes would be lost. If the process completed,then we would confirm it by committing the changes.

Storage Engines:

MyISAM

The MyISAM engine is the default engine in most MySQL installations and is a derivative of the original ISAM engine type supported in the early versions of the MySQL system. The engine provides the best combination of performance and functionality, although it lacks transaction capabilities (use the InnoDB or BDB engines) and uses table-level locking.

Unless you need transactions, there are few databases and applications that cannot effectively be stored using the MyISAM engine. However, very high-performance applications where there are large numbers of data inserts/updates compared to the number of reads can cause performance problem for the MyISAM engine. It was originally designed with the idea that more than 90% of the database access to a MyISAM table would be reads, rather than writes.

With table-level locking, a database with a high number of row inserts or updates becomes a performance bottleneck as the table is locked while data is added. Luckily this limitation also works well within the restrictions of a non-transaction database.

Limitations

1. Crash recovery can be a time-consuming process owing to MyISAM’s lack of a transaction log. Expect to have to perform such a recovery during your peak-usage period. (If it can happen, it almost certainly will.)
2. MyISAM does not support or enforce foreign key constraints.
3. All UPDATE queries to the same table are serialized — that is to say they carried out one at a time — and block all other queries, including SELECTs, from executing. This effect is pronounced on most busy multi-user applications.
4. MyISAM supports concurrent INSERTs only in certain cases.
5. Maximum of 64 indexes per row,

When to use MyISAM

1. Your application demands full-text search capabilities. Rather than push all your data into MyISAM tables to gain full-text searching, it may be viable to split your dataset into data that must be indexed for full-text searching — and stored using MyISAM — and data that should be stored using a transactional engine, such as InnoDB. A scheduled background job may then asynchronously update your MyISAM full-text indexes and provide links from the InnoDB data as appropriate. This is a common example of how to gain the best from all worlds.
2. Your application is effectively single-user — there are very few concurrent queries hitting the MySQL server.
3. You are performing limited testing or development where performance is not under scrutiny.

MERGE

The MERGE storage engine, also known as the MRG_MyISAM engine, is a collection of identical MyISAM tables that can be used as one. You can then execute queries that return the results from multiple tables as if they were just one table. Each table merged must have the same table definition.

The MERGE table is particularly effective if you are logging data directly or indirectly into a MySQL database and create an individual table per day, week or month and want to be able to produce aggregate queries from multiple tables. There are limitations to this however, you can only merge MyISAM tables and the identical table definition restriction is strictly enforced. Although this seems like a major issue, if you had used one of the other table types (for example InnoDB) then the merge probably wouldn’t be required.

Limitations

1. You can use only identical MyISAM tables for a MERGE table.
2. You cannot use a number of MyISAM features in MERGE tables. For example, you cannot create FULLTEXT indexes on MERGE tables. (You can, of course, create FULLTEXT indexes on the underlying MyISAM tables, but you cannot search the MERGE table with a full-text search.)
3. If the MERGE table is non-temporary, all underlying MyISAM tables must be non-temporary, too. If the MERGE table is temporary, the MyISAM tables can be any mix of temporary and non-temporary.
4. MERGE tables use more file descriptors. If 10 clients are using a MERGE table that maps to 10 tables, the server uses (10 × 10) + 10 file descriptors. (10 data file descriptors for each of the 10 clients, and 10 index file descriptors shared among the clients.)
5. Key reads are slower.Key reads are slower. When you read a key, the MERGE storage engine needs to issue a read on all underlying tables to check which one most closely matches the given key. To read the next key, the MERGE storage engine needs to search the read buffers to find the next key. Only when one key buffer is used up does the storage engine need to read the next key block.

When to Use MERGE

1. Easily manage a set of log tables. For example, you can put data from different months into separate tables, compress some of them with myisampack, and then create a MERGE table to use them as one.
2. Obtain more speed. You can split a big read-only table based on some criteria, and then put individual tables on different disks. A MERGE table on this could be much faster than using the big table.
3. Perform more efficient searches. If you know exactly what you are looking for, you can search in just one of the split tables for some queries and use a MERGE table for others. You can even have many different MERGE tables that use overlapping sets of tables.
4. Perform more efficient repairs. It is easier to repair individual tables that are mapped to a MERGE table than to repair a single large table.
5. Instantly map many tables as one. A MERGE table need not maintain an index of its own because it uses the indexes of the individual tables. As a result, MERGE table collections are very fast to create or remap. (Note that you must still specify the index definitions when you create a MERGE table, even though no indexes are created.
6. If you have a set of tables from which you create a large table on demand, you should instead create a MERGE table on them on demand. This is much faster and saves a lot of disk space.
7. Exceed the file size limit for the operating system. Each MyISAM table is bound by this limit, but a collection of MyISAM tables is not.
8. You can create an alias or synonym for a MyISAM table by defining a MERGE table that maps to that single table. There should be no really notable performance impact from doing this (only a couple of indirect calls and memcpy() calls for each read).

MEMORY

The MEMORY storage engine (previously known as the HEAP storage engine) stores all data in memory; once the MySQL server has been shut down any information stored in a MEMORY database will have been lost. However, the format of the individual tables is kept and this enables you to create temporary tables that can be used to store information for quick access without having to recreate the tables each time the database server is started.

Long term use of the MEMORY storage engine is not generally a good idea, because the data could so easily be lost. However, providing you have the RAM to support the databases you are working on, use of MEMORY based tables is an efficient way of running complex queries on large data sets and benefitting from the performance gains.

The best way to use MEMORY tables is to use a SELECT statement to select a larger data set from your original, disk-based, tables and then sub-analyse that information for the specific elements you want.

FEDERATED

The FEDERATED storage engine (added in MySQL 5.03) enables you to access data from remote MySQL database (other databases may be supported in the future) as if it were a local database. In effect, the MySQL server acts as a proxy to the remote server, using the MySQL client access library to connect to the remote host, execute queries and then reformat the data into the localized format.

In essence, it is a way for a server, rather than a client, to access a remote database and can be an effective way of combining data from multiple hosts or of copying specific data from remote databases into local tables without the use of data exports and imports.

ARCHIVE

The ARCHIVE storage engine supports only the INSERT and SELECT statements, but does support most of the MySQL field types. Information stored in an ARCHIVE storage engine table is compressed and cannot be modified and so ARCHIVE tables are perfect for storing log data (which you don’t want to be able to change) or information that is no longer in active use (for example, old invoicing or sales data).

While the information is stored very efficient, care should be taken when accessing data stored in the ARCHIVE tables. Because the information is compressed, selects have to read the entire table, and that also means decompressing the information. This can obviously increase the time taken to perform complex searches and retrievals. If you are performing a large number of queries on the information in these tables it may be easier to temporarily copy your data to another, uncompressed, data type such as MyISAM.

CSV

The CSV storage engine stores data not in a binary format, but in the form a CSV (Command Separated Values) file. Because of this, there are limitations to the data stored. It is not an efficient method for storing large volumes of data, or larger data types like BLOB, although such types are supported. There is also no indexing. However, because the data is stored in the CSV format it is exceedingly portable; these CSV files generated can easily be imported into many different software packages, including Excel, OpenOffice and database systems like Access or FileMaker.

In general, the CSV engine is impractical as a general database engine. It is, however, probably the most effective and easiest method for data exchange. What makes it so convenient is that we can use SELECT and INSERT statements to create the database, which in turn means that we can easily produce CSV files based on queries of other data.

With some careful work, the CSV storage engine can also be used as an effective way of getting information into MySQL. Here, you can create the tables first, shutdown the MySQL server, copy over CSV files that you have exported from Excel, Access or another database, and you can then import the data and copy it over to MyISAM or InnoDB tables.

BLACKHOLE

The Blackhole storage engine accepts but does not store data and retrievals always return an empty set. The functionality can be used in distributed database design where data is automatically replicated, but not stored locally. Although you can create tables and indexes, all SQL statements that would add or update information to the database are executed without actually writing any data. The database structure is retained, however, and you can create any indexes on the (non-existent) information that you want.

Although this seems like a futile exercise, it does allow you to test out database structures and play with table definitions without actually creating any data. Even more useful, however, is that SQL statements on BLACKHOLE databases are written to the binary log, and therefore are replicated to slave databases.

You can use this functionality to update one or more slaves directly without writing any local data. There are a number of potential uses for this functionality.

ISAM

The ISAM storage engine was the original engine type available with versions of MySQL up until MySQL 3.23, when the MyISAM storage engine was introduced. ISAM has a number of different limitations that make it impractical as a database engine. These include the storage format, which is native to the platform (and therefore not portable between systems), a maximum table size of just 4GB and limited text searching facilities. Indexes are also more limited. Since MyISAM is supported on the same platforms as ISAM, and provides better compatibility, portability and performance.

ISAM is included for backwards compatibility, you certainly shouldn’t use ISAM for new databases, use MyISAM instead.

Berkeley DB (BDB)

The Berkeley DB (or BDB) engine is based on the technology provided by the Berkeley DB storage system developed by SleepyCat software. BDB is a hash based storage mechanism, and the keys to the hash values are stored very efficiently. This makes the recovery of information–especially when accessed directly using a unique key incredibly quick, and by far the quickest of the available database types. Recovering full records is even quicker if you the data is short enough to be stored with the unique key (i.e., under 1024 bytes long). BDB is also one of only two types of storage engine that support transactions.

BDB is, however, limited in other ways. Although it uses page locking, locking only 8192 bytes of a table, rathter than the entire table, during an update this can cause problems if you are performing a large number of updates in the same page (for example, inserting many rows). There is unfortunately no way round this. Sequential data access–for example a large quantity of rows matching non-indexed data–can be a lot slower because the data needs to be scanned row by row.

Recovery of information with BDB tables can also be a problem. Data in BDB is stored in a combination of the key index, the data file and binary data logs. A loss of data in any of these sections, even just one of the data logs, can make the data in the database totally unrecoverable.

Where BDB shines therefore is in locations where you can access specific blocks of data by a unique key that does not frequently change.

InnoDB

The InnoDB Engine is provided by Innobase Oy and supports all of the database functionality (and more) of MyISAM engine and also adds full transaction capabilities (with full ACID (Atomicity, Consistency, Isolation, and Durability) compliance) and row level locking of data.

The key to the InnoDB system is a database, caching and indexing structure where both indexes and data are cached in memory as well as being stored on disk. This enables very fast recovery, and works even on very large data sets. By supporting row level locking, you can add data to an InnoDB table without the engine locking the table with each insert and this speeds up both the recovery and storage of information in the database.

As with MyISAM, there are few data types that cannot effectively be stored in an InnoDB database. In fact, there are no significant reasons why you shouldn’t always use an InnoDB database. The management overhead for InnoDB is slightly more onerous, and getting the optimization right for the sizes of in-memory and on disk caches and database files can be complex at first. However, it also means that you get more flexibility over these values and once set, the performance benefits can easily outweigh the initial time spent. Alternatively, you can let MySQL manage this automatically for you.

If you are willing (and able) to configure the InnoDB settings for your server, then I would recommend that you spend the time to optimize your server configuration and then use the InnoDB engine as the default.

Limitations
1. Queries that result in large scans of the tablespace are often slower when using InnoDB.
2. Consistency is only maintained if the underlying operating system and hardware can guarantee buffer flushes. This limitation is inherent in all transactional database management systems.
3. InnoDB tables consume a greater amount of space on-disk than their MyISAM equivalents. This is now largely irrelevant given the ubiquity of large (multiple hundreds of gigabytes) hard disk drives.

When to use InnoDB

1. You are developing an application that requires ACID compliance. At the very least, your application demands the storage layer support the notion of transactions.
2. You require expedient crash recovery. Almost all production sites fall into this category, however MyISAM table recovery times will obviously vary from one usage pattern to the next. To estimate an accurate figure for your environment, try running myisamchk over a many-gigabyte table from your application’s backups on hardware similar to what you have in production. While recovery times of MyISAM tables increase with growth of the table, InnoDB table recovery times remain largely constant throughout the life of the table.
3. Your web site or application is mostly multi-user. The database is having to deal with frequent UPDATEs to a single table and you would like to make better use of your multi-processing hardware.

NDB Cluster

Another well-known storage engine. It allows one to cluster tables. That means you have multiple masters, which all can do inserts, updates and deletes on the same table. NDB has row-level locking, but not full multi-version concurrency control. Due to some architectural limitations of how the MySQL server executes joins, they perform rather poorly with NDB storage engine. For that reason, it is best used for single table primary key lookups. So, you would not likely port your entire web-facing database to NDB Cluster for example.

Summary

As you may have been able to conclude from the above summary of the different storage engines available, there are few reasons not to use either the MyISAM or InnoDB engine types. MyISAM will do in most situations, but if you have a high number of updates or inserts compared to your searches and selects then you will get better performance out of the InnoDB engine. To get the best performance out of InnoDB you need to tweak the parameters for your server, otherwise there is no reason not to use it.
The MERGE engine is an exceedingly effective way of querying data from multiple, identically defined, tables. The MEMORY engine is the best way to perform a large number of complex queries on data that would be inefficient to search on a disk based engine. The CSV engine is a great way to export data that could be used in other applications. BDB is excellent for data that has a unique key that is frequently accessed. The following table provides an overview of some storage engines provided with MySQL:

Recompile PHP for Litespeed webserver

LSAPI is LiteSpeed’s open-source API between external applications and LiteSpeed Web Server. This how-to is for compiling and installing PHP + LSAPI on Linux, OS X, FreeBSD, Solaris, and so on.  LiteSpeed  comes with PHP 4.4.x compiled with LSAPI. To change the LightSpeed php to the latest stable one, we need to recompile the php with the LSAPI for LiteSpeed.

I  am trying to provide some info for doing this task.

Instructions:

a)  Download the required php  from php.net

wget  http://in2.php.net/get/php-5.2.11.tar.bz2/from/this/mirror

tar -xzvf  php-5.2.11.tar.bz2

cd php-5.2.11/sapi

b) Download and  the  latest LSAPI for PHP from http://www.litespeedtech.com/ into the “sapi” folder of php source:

wget http://www.litespeedtech.com/packages/lsapi/php-litespeed-4.10.tgz

tar  -xvf php-litespeed-4.10.tgz

c) Change directory to root PHP source directory and run commands:

cd ..

touch ac*

./buldconf –force

d) Configure/Compiling

If you have an  apache compilation already , remove the part   option “–with-apxs” and use    ” –with-litespeed”  . Also  you may need to change the “prefix” option too . So a basic  configure command would be like  this

./configure  ‘–prefix=/lsphp5′ ‘–with-litespeed’ ‘with-mysql’

make

make install

Note: You must compile PCRE support inorder for the default auto-index php script to work correctly

Post Install Configurations

1. Replace the existing lsphp binary with the new one.

Change the directory to the current installation of “lsws” ( it varies on different machines)

cd  /usr/local/lsws/fcgi-bin

mv lsphp lsphp.old

cp /php-5.2.11/sapi/litespeed/php lsphp-5.2.11

ln -sf lsphp5 lsphp-5.2.11

Check Installation

/usr/local/lsws/fcgi-bin/lsphp5 -v

It should return something like:

PHP 5.2.11 (litespeed) (built: Sep26 2008 14:09:09)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies

Notice litespeed in parenthesis. This means that the PHP binary has litespeed (LSAPI) support builtin.

2. php.ini

The php.ini file will be located at /usr/local/lsws/php/php.ini

If we want to use the old PHP.ini just copy it here.

# cd /usr/local/lsws/php
# mv php.ini php.ini.old
# cp /usr/local/ZEND/etc/php.ini .

3. Restart Litespeed Webserver

Finally restart LSWS and use our new PHP binary.

/usr/local/lsws/bin/lswsctrl restart

How to recompile Kernel?

Kernel Recompilation

Compiling custom kernel has its own advantages and disadvantages.  It helps to optimize the kernel to your environment (hardware and usage patterns).  I shall try to guide you through Kernel recompilation process.

Step 1:

Download the kernel source

cd /usr/local/src

wget  http://www.kernel.org/pub/linux/kernel/v2.6/linux-x.y.z.tar.bz2

Note: Replace x.y.z with actual version number.

Step 2:

Extract the  source file

tar -xjvf linux-x.y.z.tar.bz2

Step 3:

Patching the Kernel

If you are requested to apply any patches , follow these steps

a) Move the downloaded kernel patch to the /usr/local/src directory.

b)  Extract the patch file

c)  Patch the kernel  source using the extracted  patch file

cd /usr/local/src/linux-x.y.z

patch -p1 < patchfile-2.2.x

Now the  Kernel Source is patched against known  vulnerabilities.

Step 4:

Configuration

If you are trying to upgrade the Kernel of already running server , it is always better use the existing configuration. To do this follow these steps

#uname -a
Linux  Server1  2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

# cd /boot

There you can see different configuration files as given below

#ls

config-2.6.18-128.el5    initrd-2.6.18-128.el5.img  message
config-2.6.18-164.el5     initrd-2.6.18-164.el5.img  quota.user*

identify the configuration file corresponding to the version of OS installed , In our case it is config-2.6.18-164.el5 . We are  copying this file to the downloaded kernel source  to use it during configuration.

#cp -p config-2.6.18-164.el5 /usr/local/src/linux-x.y.z/.config

# make clean

# make mrproper

# make menuconfig

You have to select different options as per your need.   If you  intended to use the existing configuration ,specify the path to the file  ( .config in this case) by selecting the option

“Load an Alternative configuration file”

Step 5: Compilation

Compile the Kernel using the following commands

Compile to create a compressed kernel image

# make

Compile kernel modules:

# make modules

Install kernel modules

# make modules_install

Step 6: Install Kernel

If the above steps completed without any errors , now its the time to Install the new Kernel

# make install

It will install three files into /boot directory as well as modification to your kernel grub configuration file:

System.map-x.y.z

config-x.y.z

vmlinuz-x.y.z

Step 7:  Create the Initrd image

Type the following command :

# cd /boot

# mkinitrd -o initrd.img-x.y.x  x.y.z

initrd images contains device driver which needed to load rest of the operating system later on. Not all computer requires it, but it is  better  to create one

Step 8: Boot Loader Modification

Mofdify the boot loader to boot the new OS as default . Check the documentaions corresponding to your boot loader

Step 9: The last step

execute the following command

#reboot

Wait a few minutes and once it is up , you can see that the new Kernel is loaded 

RootKits and anti rootkits

A rootkit is a collection of programs that enable an attacker to get the same privilage as the root user in a linux or unix system. The word is composed of two portions: ‘root’ – meaning the application will provide the highest access level of the root/administrator in the system and ‘kit’ – meaning it has a number of tools.

Attackers after getting access to a server, will install a rootkit to hide their identity and run desired scripts anywhere within the server. It makes the life of a hacker easy once installed. Rootkits are not easily detectable. Sometimes, if the rootkit is one of the latest ones without a diagnosis, the server will have to be rebuild from scratch.

A rootkit will have multiple applications for cracking the entire server, some of them are:

Server Access Applications (Back door application)
These applications will create a backdoor to log in to the hacked system without using the exploit again.

Log clearing Applications
These applications clear the logs of the events performed by the hacker or the applications used. They all the associated log files in the server.

Packet sniffing Applications
These applications monitor the data through the various interfaces in the server at particular ports.

Malicious Scripts
Many scripts will be installed like IRC bots, ddos daemons, spam servers, trojans, worms etc.

There are mainly two kinds of root kits. The application rootkit and the kernel rootkit.

Application rootkits
These rootkits mimic a particular application and will hide the attackers files/processes from being revealed by the original application. To illustrate, a rootkit ls application will perform all the task of a normal ls but will not display any of the files of the attacker. Other application rootkits will create backdoors for unauthorised access, packet sniffers etc which go undetected or are hidden by renaming. Application rootkits are the most common.

Kernel rootkits
Kernel rootkits modify the kernel and apply patches to the kernel and device drivers. They also hide the applications and files of the attacker. As antivirus and other applications run beneath the kernel, they are the most undetectable rootkits.

‘Prevention is better than cure’ – as this saying goes, it is always better to keep the system secure and updated when ever possible to stop these installations. There are some applications which help detect any known rootkits running in the system. One such is the chkrootkit.

chkrootkit is one of the popular rootkit detectors (an anti-rootkit) and it is know to detect common rootkits on unix/linux servers. chkrootkit relies on basic string processing techniques to determine the presence of rootkits. It scans specific sytem files and binaries targeted by rootkits for known signatures.

The following are the instructions to install chkrootkit version 0.49 in a server.

cd /usr/local/

wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz

wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.md5

md5sum -c chkrootkit.md5 # to check if the downloaded file is intact

tar -xzf chkrootkit.tar.gz

cd chkrootkit-0.49/

make sense

./chkrootkit

chkroootkit will check all the files and display the status of the files analysed. This information may be logged for future reference. For this a cron job may be setup to be run at least once a month.

Inserting an entry like the one below into the systems cron tasks (executed atleast once a month) will send the report of the chkrootkit vulnerabilities to the administrator conserned.

/usr/local/chkrootkit-0.49/chkrootkit | mail -s "chkrootkit report $(date +%d/%m/%y)" "admin@domain.com"

All about SSL

SSL or Secure Sockets Layer (Notice the last s in sockets) is used to secure the communication over the internet. This technique was introduced by Netscape. It uses the RSA public key cryptography for encryption/decryption.

In the protocol stack used in the internet. The SSL protocol runs above TCP/IP and below higher-level protocols such as HTTP or IMAP.

The SSL protocol includes two sub-protocols:
1) SSL record protocol
2) SSL handshake protocol

The SSL record protocol defines the format used to transmit data. The SSL handshake protocol involves using the SSL record protocol to exchange a series of messages between an SSL-enabled server and an SSL-enabled client when they first establish an SSL connection.

Now SSL for the layman
SSL basically creates an encrypted communication channel between the two parties involved in the communication. For a third person involved in the middle of this communication channel, the data seems to be garbled.

Suppose Alice (A, the browser) wishes to communicate with Bob (B, the server) then the exact steps that takes place inorder to begin the encrypted communication are:

1) A -> B hello
Alice contacts Bob and requests for a private communication (request for an https link at port 443)

2) B -> A Hi, I’m Bob, bobs-certificate
Bob send back to Alice his certificate. A certificate authenticates that it is Bob who is actually communicating with Alice. It is like a unique ID card displayed.

3) A -> B prove it
Alice requests Bob to prove his identity.

4) B -> A Alice, This Is bob { digest[Alice, This Is Bob] } bobs-private-key
Bob sends back a message and its digest encrypted with his private key. This step can also be like sending a document with a digital signature (when you have Alice’s public key).

5) A -> B ok bob, here is a secret {secret} bobs-public-key
Alice sends back to Bob some secret. Usually a session key encrypted using Bob’s public key obtained from his certificate

6) B -> A {some message,MAC}secret-key
Next Bob generates a secret key from Alice’s secret (earlier step) and sends back to Alice the real message and its MAC encrypted with this secret key. This is actually the encrypted website.

Terminologies

Certificate
This is actually bobs public key containing document which is digitally signed by a certificate issuer’s private key (like Verisign). In this process Verisign gets all the necessary documents to verify that Bob’s identity is correct and it gets Bob’s public key (and some other data like certificate expiry period, Bobs identity) and encrypts it with its own private key. Now Verisign’s public key comes built-in along with every browser (so that the browser can get bobs public key from within it).

Digest
Digest or more appropriately Message Digest is like a summary of the actual message or a portion of the message. The digest of a message is is unique for every unique message, it is a one way function such that obtaining the digest, it is never possible to recover the original message (This does not involve using any key in the process). Message Digest always appears with the original message. Upon reception of this Message and its digest at the receiver’s end, the receiver can once again calculate the digest from the original message and verify the integrity of the message.

Digital signature
Let Bob send a document to Alice which is digitally signed. For this Bob must have Alice’s public key and Alice must have Bob’s public key.Bob takes the document, encrypts it first with Alice’s public key and next with its own private key(Bob’s)

B -> A [{message}alices-public-key ]bobs-public-key

Session Key
The only secret which is communicated using public key encryption is a session key. Now the session key is chosen from the ‘secret’ that the parties accept. the session key could be the secret itself or a portion of the secret or the result when the secret is passed through a previously agreed algorithm. The SSL encrypted communication does’t necessary have to be created using a public key encryption technique (This uses a lot of overhead, i.e. processing and time), it may be simple symmetric cypher(less overhead) using this session key once agreed upon. There are a variety of cypher suites available (IDEA Blow-fish RSA DES MD5 KEA) and both the parties may choose some encryption technique based on the protocol used (SSL1.0 SSL2.0 TLS etc)

MAC
MAC or Message Authentication Code is similar to the Message Digest we have discussed. It is used to verify the integrity of the Message.

MAC := Digest[ some message, secret ]

Files associated with SSL

CSR
CSR or Certificate Signing Request is a string of text generated by the server. This file is sent to the SSL vendor while purchasing an SSL. In the process of generating your CSR, you provide a number of details regarding the domain being registered. Excerpts of text from all these are taken to generate your private key. This private key is present only within the server and nowhere else. The content of the CSR basically contains the public key along with all the details you have used. You get this as domain.com.csr or domain_com.csr.

CA bundle
CA (Certificate Authority) bundle file is one which contains the public key of the Certificate Issuer (Like Verisign’s public key). Usually this is not required while installing the SSL and most browsers will have this detail in advance to decrypt the SSL certificate (the CRT file) from the server. You get this as domain.com.cabundle or domain_com.ca-bundle.

CRT
This is the actuall SSL certificate as obtained from the SSL vendor. It is a file (containing the public key of the domain secured with SSL and other details like the expiry date, owner information, address etc of the SSL) which is encrypted with the private key of the SSL vendor (Digitaly signed by the SSL vendor). You get this as domain.com.crt or domain_com.crt .

Key file
This is the file which holds your private key (strictly confidential material). The file will have the RSA private key as generated by your server software. You get this as customcardsplus.com.key or customcardsplus_com.key. This file is not usually send to your SSL vendor unlike the CSR. You get this as domain.com.key or domain_com.key .

SSL in a cPanel server
Any service can be secured in a communication channel which is encrypted with SSL. Each of this service on the encrypted channel will be on a different port. Some of them are as follows:

service	normal	ssl
http	80	443
telnet	23	992
imap	143/220	993
pop	109/110	995
smtp	25	465

A domain served as a secure webpage will require a dedicated IP (in a shared environment). SSL protocol is designed to use IP-based mapping. SSL does not support host headers. Therefore, you should have a unique IP address assigned to your secure site. These pages are served from the port 443. Let us examine the configuration of such a website in the apache’s config file /usr/local/apache/conf/httpd.conf.

Every website (in our example domain.com with username: doma) enabled with SSL has a unique set of directives in the VirtualHost section for the 443 port as:

 Dedicated IP of the domain

ServerName domain.com #Domain name secured with SSL
ServerAlias www.domain.com

DocumentRoot /home/doma/public_html

ServerAdmin webmaster@domain.com
UseCanonicalName off
CustomLog /usr/local/apache/domlogs/domain.com combined

CustomLog /usr/local/apache/domlogs/domain.com-bytes_log "%{%s}t %I .\n%{%s}t %O ."
ScriptAlias /cgi-bin/ /home/doma/public_html/cgi-bin/
SSLEngine on #This directive enables the SSL on this domain
SSLCertificateFile /etc/ssl/certs/www.domain.com.crt #Location of CRT file
SSLCertificateKeyFile /etc/ssl/private/www.doma.com.key #Location of Private key
SSLCACertificateFile /etc/ssl/certs/www.domain.com.cabundle #Location of CAbundle file
CustomLog /usr/local/apache/domlogs/domain.com-ssl_log combined #Log specific for the SSL served webpage
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown


SSLOptions +StdEnvVars #This directive will pass mod_ssl environment variables to the server scripts.

Some times the directive SSLCertificateChainFile is used in place of SSLCACertificateFile. The minimal addition you will have to make to enable SSL in your httpd.conf file is:

DocumentRoot /var/www/html
ServerName www.yourdomain.com
SSLEngine on
SSLCertificateFile /path/to/your_domain_name.crt
SSLCertificateKeyFile /path/to/your_private.key
SSLCertificateChainFile /path/to/DigiCertCA.crt

There are two locations where you are likely to find the SSL related files in your cPanel server. Usually the crt, key and the ca bundle are present in the home directory of the user in /home/username/ssl/, if it was installed using the client’s cpanel. However if the WHM was used instead to install the same, you will find it in /etc/ssl/. In either of these locations you will find two directories: certs/ and private/. certs contain the crt and cabundle while the private contains the keys.

Now you know how ssl works in your server, Any more questions? just comment!

Detailed explanation of Domain registration and it’s complete process

Websites have become a integral part of an organization in today’s world. The only medium to introduce oneself in today’s online business is by his website. Nowadays everyone are smart enough on spending some money to build an attractive website to enhance the business. Along with the appearance the name of the site too must be a catchy one. Here we discuss everything about the domain registration, transfer of domains and their status.

Domain Name

A domain name is essentially your website’s name (without www). Every domain name ends with a top level domain(TLD) name, which is always either one of a small list of generic names (three or more characters).Some instances are .com, .info, .in, .net, .org, .biz etc. For eg : “www.google.com”—->”google.com” makes the domain name and “.com” the top level domain and “www” the lower level domain. As simple as that.

Domain registration

Internet domain registration involves three entities:

Registrant who wishes to register the domain name.

Registrar who provide services to registrant.

Registry that provides services to registrar while serving as an authorative body of all information to resolve names registered in the registry’s TLD.

How to register a domain

An end-user cannot directly register and manage their domain name information with ICANN. A designated registrar must be chosen. We can register our domain with the domain registrars like Mark Monitor and Godaddy. A domain name registrar is a company, accredited by the Internet Corporation for Assigned Names and Numbers (ICANN), to register Internet domain names.

Here are some quickies to clear with the registration

1. Effective name for an effective growth
It’s important that your website name is being found in the major search engines like Google and yahoo. For that you need to chose a domain name that suits the business as well as includes the keywords that visitors would search for in the main search engines. A short, descriptive and catchy domain name may just be money in the bank.

2. Renewal
The registrars assign the domain name on a lease for a particular period of time (maximum is 10 years which varies with TLD). After the period the domain name becomes free and anyone in the internet can register with the same. So it’s neccessary to renew the lease period.Also make sure that your domain registrar will send a remainder when your domain name is about to expire.

3. Prevention is better than worry

It would be advisable to protect your domain name by registering with multiple extensions. For example if you have registered with .com then you can also register with other extensions like .org, .net etc. You don’t need to build a site on all domains and can reserve them from others registering it.

Choosing a domain name

Choosing an effective domain name is very essential in the online business. The domain name can be of any length upto 67 characters, which includes the 26 letters of the English alphabet, numbers 0-9, and hyphens – provided that it can not begin with hyphen. More than 128 million domain names are registered and thousands are added daily. So it is a herculean task to choose a right and apt one. It would be preferable if you select a name that is related to the content in your website. Make a list of these words and try different combinations. Also try to make a name that is short, memorable, not easily confused and related to the content in your website. Make a list of these words and try different combinations. Also try to make a name that is short, memorable, not easily confused and related to the business.

Choosing a registrar

Some of the important aspects that should be considered while choosing a registrar for domain name:

1. ICANN accreditation: The registrar should be accredited by ICANN(Internet Corporation for Assigned Names and Numbers). 2. Reliabilty: The registrar must be reliable as they are the ones who protect the domain of domain registrants from losing their domains. 3. Contract Agreement: It’s not like that once you have registered the domain names,is yours forever. The contract that you sign with the registrars may affect in many ways.They have the right to cancel the domain name for for certain reasons, generally when you use the domain for illegal or spamming purposes. So make sure that you have thoroughly gone through the contract. 4. Price: Prices differs from one registrar to other. While comparing prices, you must also look for factors that are provided by the registrar, that are customer support, testimonies etc. 5. 24×7 Support: Make sure that registrar is providing the 24×7 support. 6. Contact Information: One must look for the contact information of the registrar where you can easily get to them without any effort. You might want to test them to confirm whether they are available before you buy from them. 7. Management: Domain management is most important factor to be considered. It’s important that how they manages the domain. Many of them provides a control panel which allows you to simply log in on the web and update the information

Domain Name Registry

The domain name after the registration will be stored in a registry. It is a database of all domain names registered in top-level domain. They actually store the information about your domain name and it’s location so that it is accessible all around the world. A registry operator (also caled as Network Information Center) is the part of the Domain Name System (DNS) of the Internet that keeps the database of domain names, and generates the zone files which convert domain names to IP addresses. Each NIC is an organization that manages the registration of Domain names within the top-level domains for which it is responsible, controls the policies of domain name allocation, and technically operates its top-level domain. Domain names are managed by an hierarchy named IANA (Internet Assigned Numbers Authority).

Whois Record

It is a query/response protocol used for querying an official database to determine

Registrant: The person who holds the domain or registered the domain name.
Administrative Contact: The person/organization responsible for all administrative issues related to the domain name and the registrant information.
Billing Contact: The person/organization responsible for all the billing issues related to the domain name.
Technical Contact: The person/organization responsible for all the technical issues related to the domain.

The other information that is retrieved by the whois service is

Registrar of record: The domain name registration that registered/approved the domain name.
Record last updated: The last update made to the domain’s whois record.
Domain Creation date: The date the domain was initially registered.
Domain Expiry date: The date on which the domain name expires.

How to transfer a domain name

Domain names can be transferred from one registrar to another. You could save money on domain name renewals by transferring them to a different registrar. There are certain steps to be followed for transferring a domain from one registrar to another.

For domain transfer you need to obtain an auth code from the old registrar. Auth code is a 6 to 26 bit character code (password)assigned by the registrars to the owners as a security measure . Auth code basically is a password for the domain name. This authorised code must be submitted to the new registrar for the transfer. After receiving the auth code the new registrar will initiate the transfer. They will send an email confirmation to the holder which must be approved within 5 days otherwise will lead to failure of transfer. This is the first approval which is done by the holder. Even if you have approved the email and your domain name is locked or suspended for non-payment at the loosing registrar end then the transfer fails.

As a second approval loosing registrar sends you a mail to cancel the transfer request,ignoring the mail will complete the transfer within 5-7 days according to the ICANN. If you cancel the request the transfer will fail.Even on his step if the loosing registrar lock or unlock your domain name then the transfer fails.

Furthermore please make sure that

1. If you recently registered or transferred your domain name, please wait 60 days before applying for transfer.
2. Domain name is not locked by the loosing registrar.
3. Domain name has not been deleted.

The Administrative Contact and the Registered Name Holder, as listed in the Losing Registrar’s publicly accessible Whois service are the only parties that have the authority to approve or deny a transfer request to the gaining registrar. Registrar’s may use whois data from either the Registrar of Record or the relevant Registry for the purpose of verifying the authenticity of a transfer request.

New Registrar’s Requirements

The transfer request must be done by the registered name holder. An authorised code must be submitted by the holder and the authorisation must be done via a valid Standardized Form of Authorisation (FOA). The FOA labeled “Initial Authorisation for Registrar Transfer” must be used by the gaining Registrar to request an authorisation for a registrar transfer from the administrative contact. Another FOA labeled Confirmation of standard authorisation may be used by the registrar for the confirmation of the transfer request which is done through email as explained earlier.

Transfer Disputes

Disputes between registrars over alleged violations of the transfer policy may be initiated by any ICANN-accredited registrar. If you believe that your domain name was transferred to a new registrar without your authorization or consent, please contact the original registrar. If you believe that your transfer request was inappropriately denied by your registrar, please contact the registrar to whom you wish to transfer for assistance.

Who manages your domain names & your ISPs IP Addresses

IANA & ICANN – That’s the shortest answer! Now here comes the boring explanation 

The assignment of identifiers such as addresses and names, to ensure that they are created and allocated in a way that is acceptable to all is the main factor for the success of the Internet. So some sort of centralized organization is required. The organization originally responsible for this task was Internet Assigned Names and Numbers (IANA). IANA was originally charged with the task of managing which IP address blocks had been assigned to different companies and groups, and maintaining periodically-published lists of Internet parameters such as TCP and UDP Port Numbers. It also was in charge of DNS registrations. As the Internet grew, there was the requirement of a additional authority to manage the growing load. So by the mid 90s the Internet Corporation for Assigned names and Numbers (ICANN) came into existence.

ICANN is now officially in charge of all of the centralized registration tasks including IP address assignment, DNS domain name assignment, and protocol parameters management.

This development would have meant that IANA would have been completely replaced by ICAAN. But that did not happen. Instead, IANA was put under ICANN and is now in charge of IANA. Both organizations are responsible for IP addresses and parameters. Thus there are basically no differences between the two. These two together are at the top level of the Internet’s Name and Addresses registration and their delegation process. They also maintain the 13 root servers in the world which are at the top of the DNS tree.

For the functioning of the whole DNS system, 2 factors are to be maintained :

1. NAMES (Domain Names)

2. NUMBERS ( IP & TCP-UDP protocol numbers)

NAMES or DOMAIN NAME SYSTEM (DNS)

The domains at their top level are classified as :

1. gTLD (generic Top Level Domain)
2. ccTLD (country code Top Level Domain)

Generic Top Level Domian (gTLD)

The initial gTLDs and their original intended organization types were:

.ARPA : A temporary domain used many years ago for transition from hosts (flat file) to DNS. Its name refers to the ARPAnet, the precursor of the modern Internet. Today this domain is used for reverse DNS resolution.

.COM : Corporations and businesses.

.EDU : Universities and other educational organizations.

.GOV : Government agencies.

.MIL : Military organizations.

.NET : Organizations that implement, deal with or manage networking technologies

.ORG : Other organizations that don’t fit into any of the classifications above.

The .ARPA domain is the “Address and Routing Parameter Area” domain and is designated to be used exclusively for Internet-infrastructure purposes. ( Refer:http://encyclopedia.thefreedictionary.com/Address+and+Routing+Parameter+Area ) It is administered by the IANA in cooperation with the Internet technical community under the guidance of the Internet Architecture Board.The .arpa domain currently includes the following second-level domains: ARPA, IN-ADDR.ARPA, IN-ADDR.ARPA, IRIS.ARPA, IP6.ARPA, URI.ARPA, URN.ARPA . So the ARPA domain was not for commercial registration purposes. This left only six categories for all other organizations. Also, the TLDs weren’t all used as was originally foreseen; for example, the .GOV and .MIL domains were not used for all types of government and military organizations, but primarily for the United States federal government and military. .EDU ended up being used only for universities, again in the United States. This left only three common top-level domains – .COM, .NET and .ORG – for almost all other groups and companies that wanted to use the organizational hierarchy. Since there were only three such TLDs, they quickly became very “crowded”, especially the .COM domain. A new fourth domain, .INT for international organizations, was added fairly soon to the original seven, but it too was only for a small number of organizations, such as international standards bodies.These TLDs are intended to provide a place for all companies and organizations to be named based on their organization type. There were originally six such domains, but this has been expanded so that there are now fifteen to meet the growing needs. Please refer to the below link for the complete list :

http://www.iana.org/domains/root/db/#

Country Code Top Level Domain (ccTLD)

In theory, the gTLDs would have been sufficient to meet the needs of all the individuals, companies and groups in the world. This is especially true since .ORG by definition is a “catch all” that can include anyone or anything. However, back at the beginning of DNS, its creators recognized that the generic TLDs might not meet the needs of everyone around the world. There are several reasons for this, chief among them:

American Monopoly of the Generic Domains : The United States organizations and companies dominate the generic TLDs. This is not surprising, given that the Internet was first developed in the U.S.A., but it still presents a problem for certain groups. For example, if the United States military controls the .MIL domain where does, say, India’s military fit into the name space?

Language : Most of the generic domains are populated by organizations that primarily do business in English. There are hundreds of languages in the world, however, and it’s easier for the speakers of those tongues if they can more readily locate resources they can understand.

Local Control : Countries around the world rarely agree on much, and they certainly differ on how organizations within their nations should have their Internet presence arranged. There was a desire on the parts of many to allow nations to have the ability to set up subsets of the name space for their own use.

For these and other reasons, the Internet’s name space was set up with a set of country code top-level paralleling the generic ones, sometimes called / ccTLD / or geopolitical TLDs since they are based on geopolitical divisions of the world. In this hierarchy, every country of the world is assigned a particular two-letter code as a top-level domain, with a specific authority put in charge of administering the domain. For example, the ccTLD for Great Britain is “.UK”, the one for Canada “.CA” and the one for Japan is “.JP”. The codes often are more meaningful in the local language than in English, incidentally; Germany’s is “.DE” and Switzerland’s “.CH”. Refer to the following link for the complete list :

http://www.iana.org/domains/root/db/#

Each country has the authority to set up its TLD with whatever internal substructure it chooses; again, this is the power of a hierarchical structure. Some countries enforce a further geographical substructure at the lower levels. For example, the .US domain for the United States was originally set up so that all second-level domains were two-letter state abbreviations (this was later changed). Below  is the reason

Disadvantage of strict ccTLD implementation :

For eg: We need to know about a company which is located in Germany, say BMW (wow!). As per the ccTLD basis the company site should be somewhat www.bmw.de . The question is, what if we never knew the location of company ? We will obviously not sit and try suffixing those 200 ccTLDs out there. The most obvious URL that we Internet users would type into the browser would be www.bmw.com since we know it is a commercial organization. So this is where the popularity of gTLDs  are exhibited. (Ofcourse with today’s search engines like google, we can manage to find that out. But what if it is the domain of a small store in an unknown country and we do not have the time to google it out ?)

Another fine eg would be this : In the U.S , the authority in charge of this domain chose to make it follow a strict geographical hierarchy, so every domain must be of the form “organization.city.state-code.US”. So, to use this part of the name space, a company “xyz”in Boston must be within the “xyz.boston.ma.us” domain. This format has made the name more longer and harder to guess. Further,  if you weren’t aware of the city in which the company is located,  it would have added to the trouble finding it out. Finally, the .US authority eventually abandoned the strict geographical hierarchy due to its non-acceptance.

IANA is responsible for management of the DNS root zone. The role is in assigning the operators of top-level domains, such as .UK and .COM, and maintaining their technical and administrative details.

Root Zone Database : IANA’s Root Zone Database contains the authoritative record of the operators of various top-level domains. The Root Zone Database represents the delegation details of top-level domains, including gTLDs such as “.COM”, and country-code TLDs such as “.UK”. As the manager of the DNS root zone, IANA is responsible for coordinating these delegations in accordance with its policies and procedures.

DOMAIN NAME REGISTRY, DOMAIN NAME REGISTRAR & DOMAIN NAME REGISTRANT

or  simply

REGISTRY, REGISTRAR  &  REGISTRANT

A domain name REGISTRY, is a database of all domain names registered in a top-level domain. A registry operator, also called a Network Information Center (NIC), is the part of the Domain Name System (DNS) of the Internet that keeps the database of domain names, and generates the zone files which convert domain names to IP addresses. Each NIC is an organisation that manages the registration of Domain names within the top-level domains for which it is responsible, controls the policies of domain name allocation, and technically operates its top-level domain. It is potentially distinct from a domain name registrar.

A domain name REGISTRAR is an organization or commercial entity, accredited by the Internet Corporation for Assigned Names and Numbers (ICANN) or by a national country code top-level domain (ccTLD) authority, to manage the reservation of Internet domain names in accordance with the guidelines of the designated domain name registries and offer such services to the public. Such a registrar is know as “Accredited Registrar” or “Designated Registrar”.

A domain name REGISTRANT is a person/organization who/which  owns a domain name in the webspace ( i.e. in the world of Internet) so that he /it can create a website and start sharing information on the Internet. Or going by the name, a registrant is the one which approached a registrar and has registered a domain name in his name and is the owner of it. Once became a registrant of a domain name, he is the sole owner of it and no other person on the planet can request for the same domain name in the Internet’s webspace or namespace until the domain name gets expired and is deleted from the registry thereby becoming publically available.

Did you know a few hosts out there, register the domain name in their own names and not yours ? Get the domains registered in your name.

Explanation

Please refer to the links for the list of gTLDs & ccTLDs. In the tables for gTLDs and ccTLDs, Sponsoring Organisation mentioned, is the “Domain Name Registry” for the respective domain. These organisations have been directly authorized by ICAAN to hold the Root Zone Database for the domains they are handling i.e. a  domain registry comes just  below the  ICANN/IANA  in the  DNS  authority hierarchy. One of the famous registry is “Verisign” which handles .COM and .NET domains, NeuStar Inc. for .BIZ etc. This means that they are the ultimate authority (excluding ICANN/IANA) for matters pertaining to the TLDs they handle.

In the young age of the DNS, they (Sponsoring Organisation) handled all the name registrations of the domain under their authority. Later on as the Internet became more crowded, the load on them increased. Further more, these organisations increased the charges for registration (Grreeed is human nature  ). So with the aim to increase competition in this field and decrease the rates, ICANN made the domain name registration more public i.e. they started lending out the registration right to other private firms. These firms will now have the power to register a domain name into the world of Internet. For this, they will have to register with ICANN for the TLD domain they are interested in. Once they get registered, they will be an ICANN “Accredited Registrar” or “Designated Registrar” or simply a “Domain Name Registrar” (eg: goDaddy). For becoming an accredited registrar, one need not contact ICANN directly. They will have to find out which is the “Domain Registry” for the TLD they are interested in and just register at the particular registry’s website. This is one of the main differences between a “registry” and a “registrar”. A particular TLD’s registry has the power to authorize a 3rd party as that TLD’s accredited registrar and people who wish to start a domain(or website) can buy a domain name from this registrar. One can become an accredited registrar for more than 1 TLDs. For eg : if a company needs to become the accredited registrar for the TLDs – .com , .biz  and  .coop , they will have to individually register with the : VeriSign Global Registry Services, DotAsia Organisation Ltd. and DotCooperation LLC respectively. Once they get registered  they attain the “Accredited Registrar” status for the TLDs .COM, .BIZ & .COOP. Their company name will automatically be entered into the “Accredited Registrar” list of ICANN. The company can then go onto provide domain names under .COM, .BIZ & .COOP to clients.

So the IANA/ICANN is responsible for  maintaining the DNS ROOT which is the upper-most part of the DNS hierarchy, and involves delegating administrative responsibility of “top-level domains”, which are the last segment of a domain name, such as .com, .uk and .nz. Part of this task includes evaluating requests to change the operators of country code domains, as well as day-to-day maintenance of the details of the existing operators.

NUMBER SYSTEM

IANA is responsible for global coordination of the Internet Protocol addressing systems, as well as the Autonomous System Numbers (ASN) used for routing Internet traffic. Just like maintaining the Name system, IANA has its subsidiaries for looking after the Number system.

The IP address is a Number resource that IANA manages in addition to many others. The task of assigning IPv4 and IPv6 to the end user in Internet is done in a 2 level hierarchy :

Level 1  : RIR – Regional Internet Registry( there is no such technical term for this hierarchy separation as “level 1″ & “level 2″…just mentioned for clear understanding)

The RIRs manage the allocation of IP addresses on a continent basis. These RIRs have the authority to re-allocate them within their respective geographical areas (of continental scope). There  are accordingly  5  RIRs covering the whole globe. The RIRs are the ones which are directly below the IANA in hierarchy. They are :

AFRINIC (for African Continent) : AfriNIC is a non-government, not-for-profit, membership based organization, based in Mauritius that serves the African Internet Community. AfriNIC is the Regional Registry for Internet Number Resources for Africa.   (http://www.afrinic.net)

APNIC ( for Asia Pacific region) : APNIC is a not-for-profit organization providing Internet addressing services to the Asia Pacific. It includes India , China , Japan, Aus etc..http://www.apnic.net/)

ARIN (North America Region) : American Registry for Internet Numbers (ARIN). It covers USA, Canada etc…(https://www.arin.net)

LACNIC (Latin America and some Caribbean Islands) : It is a Latin American and Caribbean Islands Internet Registry. (http://lacnic.net/)

RIPE NCC (for Europe, Middle East and parts of Central Asia) : Réseaux IP Européens Network Coordination Centre. (http://www.ripe.net/)

Level 2 : NIR – National Internet Registry

( this is an intermediate registry only for APNIC. for other RIRs it will have another name. )

The NIR is an organization directly under the umbrella of a RIR with the task of coordinating IP address allocations and other Internet resource management functions at a national level within a country.

The following NIRs are currently operating in the APNIC region:

* CNNIC, China Internet Network Information Center
* JPNIC, Japan Network Information Center

Level 2 : Local Internet Registry or Internet Service Provider

( this is again level 2 since it is for RIRs other than APNIC )

An Internet Service Provider(ISP) , also sometimes referred to as an Internet Access Provider (IAP), is a company that offers its customers access to the Internet. The ISP connects to its customers using a data transmission technology appropriate for delivering IP datagrams as dial-up, DSL, wireless or dedicated high-speed interconnects. In India we have the following ISPs : BSNL, Reliance, TATA etc..

And finally from the organisations in the Level 2 we the end users get the connection.

Thus in the paragraphs above we saw the authority hierarchy in the management of Internet’s NAMES & NUMBERS.

In addition to this IANA also directly manages  :

1) .INT : designed for the sole use of cross-national organisations, such as treaty organisations, that do not naturally fit into a specific country’s top-level domain. For example, the World Health Organisation uses who.int for its Internet presence, whilst NATO uses nato.int

2) .ARPA : The .arpa domain is used internally by Internet protocols, such as for reverse mapping of IP addresses

3) IDN Practices Repository : Internationalized domain names are domain names represented by native language characters. The native language domain name will be followed by .com or .net. IANA maintains a collection of “IDN tables”, which represent permitted code points (letters) allowed for Internationalised Domain Name registrations in particular registries

4) Protocol Assignments :  IANA is responsible for maintaining many of the codes and numbers contained in a variety of Internet protocols.

Note : Having understood all these one might still wonder the difference between IANA & ICANN. IANA is one of the Internet’s oldest institutions, with its activities dating back to the 1970s. Today it is operated by ICANN, an internationally-organized non-profit organization set up by the Internet community in Sept. 30 1998 to help coordinate IANA’s areas of responsibilities. Thus basically there is no difference between them. So their names are used interchangeably in many contexts.