Saturday, January 14, 2012

How to Check and Repair MySQL Tables Using Mysqlcheck


When your mysql table gets corrupted, use mysqlcheck command to repair it.
Mysqlcheck command checks, repairs, optimizes and analyzes the tables.

1. Check a Specific Table in a Database

If your application gives an error message saying that a specific table is corrupted, execute the mysqlcheck command to check that one table.
The following example checks employee table in sureshstuff database.
# mysqlcheck -c sureshstuff employee -u root -p
Enter password:
sureshstuff.employee    OK
You should pass the username/password to the mysqlcheck command. If not, you’ll get the following error message.
# mysqlcheck -c sureshstuff employee
mysqlcheck: Got error: 1045: Access denied for user 'root'@'localhost' (using password: NO) when trying to connect
Please note that myisamchk command that we discussed a while back works similar to the mysqlcheck command. However, the advantage of mysqlcheck command is that it can be executed when the mysql daemon is running. So, using mysqlcheck command you can check and repair corrupted table while the database is still running.

2. Check All Tables in a Database

To check all the tables in a particular database, don’t specify the table name. Just specify the database name.
The following example checks all the tables in the alfresco database.
# mysqlcheck -c alfresco  -u root -p
Enter password:
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
alfresco.JBPM_COMMENT                              OK
alfresco.JBPM_DECISIONCONDITIONS                   OK
alfresco.JBPM_DELEGATION                           OK
alfresco.JBPM_EVENT                                OK
..

3. Check All Tables and All Databases

To check all the tables and all the databases use the “–all-databases” along with -c option as shown below.
# mysqlcheck -c  -u root -p --all-databases
Enter password:
sureshstuff.employee                                 OK
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
..
..
mysql.help_category
error    : Table upgrade required. Please do "REPAIR TABLE `help_category`" or dump/reload to fix it!
mysql.help_keyword
error    : Table upgrade required. Please do "REPAIR TABLE `help_keyword`" or dump/reload to fix it!
..
If you want to check all tables of few databases, specify the database names using “–databases”.
The following example checks all the tables in sureshstuff and alfresco database.
# mysqlcheck -c  -u root -p --databases sureshstuff alfresco
Enter password:
sureshstuff.employee                              OK
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
..

4. Analyze Tables using Mysqlcheck

The following analyzes employee table that is located in sureshstuff database.
# mysqlcheck -a sureshstuff employee -u root -p
Enter password:
sureshstuff.employee   Table is already up to date
Internally mysqlcheck command uses “ANALYZE TABLE” command. While mysqlcheck is executing the analyze command the table is locked and available for other process only in the read mode.

5. Optimize Tables using Mysqlcheck

The following optimizes employee table that is located in sureshstuff database.
# mysqlcheck -o sureshstuff employee -u root -p
Enter password:
sureshstuff.employee         OK
Internally mysqlcheck command uses “OPTIMIZE TABLE” command. When you delete lot of rows from a table, optimizing it helps to get the unused space and defragment the data file. This might improve performance on huge tables that has gone through several updates.

6. Repair Tables using Mysqlcheck

The following repairs employee table that is located in sureshstuff database.
# mysqlcheck -r sureshstuff employee -u root -p
Enter password:
sureshstuff.employee        OK
Internally mysqlcheck command uses “REPAIR TABLE” command. This will repair and fix a corrupted MyISAM and archive tables.

7. Combine Check, Optimize, and Repair Tables

Instead of checking and repairing separately. You can combine check, optimize and repair functionality together using “–auto-repair” as shown below.
The following checks, optimizes and repairs all the corrupted table insureshstuff database.
# mysqlcheck -u root -p --auto-repair -c -o sureshstuff
You an also check, optimize and repair all the tables across all your databases using the following command.
# mysqlcheck -u root -p --auto-repair -c -o --all-databases
If you want to know what the command is doing while it is checking, add the –debug-info as shown below. This is helpful while you are checking a huge table.
# mysqlcheck --debug-info -u root -p --auto-repair -c -o sureshstuff employee
Enter password:
sureshstuff.employee  Table is already up to date

User time 0.00, System time 0.00
Maximum resident set size 0, Integral resident set size 0
Non-physical pagefaults 344, Physical pagefaults 0, Swaps 0
Blocks in 0 out 0, Messages in 0 out 0, Signals 0
Voluntary context switches 12, Involuntary context switches 9

8. Additional Useful Mysqlcheck Options

The following are some of the key options that you can use along with mysqlcheck.
  • -A, –all-databases Consider all the databases
  • -a, –analyze Analyze tables
  • -1, –all-in-1 Use one query per database with tables listed in a comma separated way
  • –auto-repair Repair the table automatically it if is corrupted
  • -c, –check Check table errors
  • -C, –check-only-changed Check tables that are changed since last check
  • -g, –check-upgrade Check for version dependent changes in the tables
  • -B, –databases Check more than one databases
  • -F, –fast Check tables that are not closed properly
  • –fix-db-names Fix DB names
  • –fix-table-names Fix table names
  • -f, –force Continue even when there is an error
  • -e, –extended Perform extended check on a table. This will take a long time to execute.
  • -m, –medium-check Faster than extended check option, but does most checks
  • -o, –optimize Optimize tables
  • -q, –quick Faster than medium check option
  • -r, –repair Fix the table corruption.

Things to learn

Advance administrative tasks on OS linux
  Linux Server, Installation and configuration of Apache, FTP, DNS, DHCP, SAMBA Server,       networking, RAID configuration on LINUX platforms
  Troubleshooting issues of linux Operating System.
  Snapshot based backup technology.
  Build and support complex linux systems of physical and virtual(XEN based).
  experience on another Open Source Hypervisor such as VBox, KVM with experience on P2V,  V2P, BMR
  Backup solutions Veritas / Arc serve / NTBackup. 

  large server cluster
  systems automation 
  Amazon EC2, SQS, S3
  tools like Puppet, MemCached, Varnish, TokyoCabinet, HAProxy, Nagios, Cacti, Ganglia, and MySQL 
  C and Kernel programming, multithreading and synchronization
  Linux kernel, driver development and troubleshooting
  OS concepts, multiprocessing, File Systems, Virtual Memory
  processor architecture, SMP systems, memory hierarchy
  high availability database driven environments.
  Apache HA, High Availability, SAN, DRBD, Load balancing, TCP/IP, database driven environment
  Load Balancing, DRBD, Apache HA,  IP tables,  Web application 
  VMWare ESX / vSphere
  RedHat Cluster
  Sun Cluster
  EMC and Hitachi SAN storage and zoning
  Fault Tolerant networking, High Availability
  VMware : Clustering, High Availablity, Dynamic Resource Scheduling
           Storage distribution
           Switches, physical networking
           VMWare Certified Professional
           FibreChannel storage

  Loadbalancers, Packet Filtering, IPFilter, Security, Penetration testing, Break/Fix   
  install, tune, troubleshoot, apply patches and support Web servers and integrate third party applications Responsible for uptime of the Web severs and improving performance 

  knowledge in conducting VAPT and providing solutions
  knowledge of source control management.
  Knowledge in ethical hacking and information security management systems
  implementation and maintenance of web analytics
  Apache, Tomcat and IIS web servers administration
  nmap, snmpwalk, nessus, dnswalk, libwhisker, strace
  HP Openview, Nagios, ZenOSS
  Automated Web Application Stress Testing
  Tomcat, and Bea Weblogic java application servers
  openldap, slurpd to syncrepl migration, ldapsearch

  Advanced usage of sniffers for network troubleshooing and optimization
  tcpdump, ethereal, and ngrep all covered indepth 
  Linux and Unix proactive system hardening
  tcpdump, ntop, ethereal / wireshark
  Apache Web Server, Bind DNS,

  Vulnerability scanning and network penetration testing of customer facing applications
  Customized and branded Perl open source web metrics reporting software for intranet usage statistics

Useful Commands

I use the following ps commands in order to check for performance probelms:

1) Displaying top CPU_consuming processes:

ps aux | head -1; ps aux | sort -rn +2 | head -10

2) Displaying top 10 memory-consuming processes:

ps aux | head -1; ps aux | sort -rn +3 | head

3) Displaying process in order of being penalized:

ps -eakl | head -1; ps -eakl | sort -rn +5

4) Displaying process in order of priority:

ps -eakl | sort -n +6 | head

5) Displaying process in order of nice value

ps -eakl | sort -n +7

6) Displaying the process in order of time

ps vx | head -1;ps vx | grep -v PID | sort -rn +3 | head -10

7) Displaying the process in order of real memory use

ps vx | head -1; ps vx | grep -v PID | sort -rn +6 | head -10

8) Displaying the process in order of I/O

ps vx | head -1; ps vx | grep -v PID | sort -rn +4 | head -10

9) Displaying WLM classes

ps -a -o pid, user, class, pcpu, pmem, args

10) Determinimg process ID of wait processes:

ps vg | head -1; ps vg | grep -w wait

11) Wait process bound to CPU

ps -mo THREAD -p

lsof:

       To list all open files, use:
     
             # lsof

       To list all open Internet, x.25 (HP-UX), and UNIX domain files, use:

        # lsof -i -U

       To list all open IPv4 network files in use by the process whose PID is 1234, use:

             # lsof -i 4 -a -p 1234

       To list all files using any protocol on ports 513, 514, or 515 of host wonderland.cc.purdue.edu, use:

             # lsof -i @wonderland.cc.purdue.edu:513-515

       To list all files using any protocol on any port of mace.cc.purdue.edu (cc.purdue.edu is the default domain), use:

             # lsof -i @mace

       To list all open files for login name ââabeââ, or user ID 1234, or process 456, or process 123, or process 789, use:

             # lsof -p 456,123,789 -u 1234,abe

       To list all open files on device /dev/hd4, use:

             # lsof /dev/hd4


     To find the process that has /u/abe/foo open, use:

             # lsof /u/abe/foo

       To send a SIGHUP to the processes that have /u/abe/bar open, use:

             # kill -HUP âlsof -t /u/abe/barâ

       To find any open file, including an open UNIX domain socket file, with the name /dev/log, use:

             # lsof /dev/log

       To find processes with open files on the NFS file system named /nfs/mount/point whose server  is  inaccessible,  and presuming your mount table supplies the device number for /nfs/mount/point, use:

             # lsof -b /nfs/mount/point

       To do the preceding search with warning messages suppressed, use:

             # lsof -bw /nfs/mount/point

       To ignore the device cache file, use:

             # lsof -Di

       To  obtain  PID  and command name field output for each process, file descriptor, file device number, and file inode number for each file of each process, use:

             # lsof -FpcfDi

       To list the files at descriptors 1 and 3 of every process running the lsof command for login  ID  ââabeââ  every  10  seconds, use:

             # lsof -c lsof -a -d 1 -d 3 -u abe -r10

       To list the current working directory of processes running a command that is exactly four characters long and has an  âoâ or âOâ in character three, use this regular expression form of the -c c option:

             # lsof -c /^..o.$/i -a -d cwd

       To find an IP version 4 socket file by its associated numeric dot-form address, use:

             # lsof -i@128.210.15.17  

fuser:

      # fuser -km /home

kills all processes accessing the file system /home in any way.

      # if fuser -s /dev/ttyS1; then :; else something; fi

invokes something if no other process is using /dev/ttyS1.

      # fuser telnet/tcp shows all processes at the (local) TELNET port.

Some Important Command to find DDOS Attack

netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

netstat -ntu | grep -v TIME_WAIT | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

netstat -an | grep :80 | awk '{print $5}' | cut -f1 -d":" | sort | uniq -c | sort -n




netstat Command Example

# netstat –listen

Display open ports and established TCP connections:

# netstat -vatn

For UDP port try following command:

# netstat -vaun

If you want to see FQDN then remove -n flag:

# netstat -vat

lsof Command Examples

Display list of open ports

# lsof -i

To display all open files, use:

# lsof

To display all open IPv4 network files in use by the process whose PID is 9255, use:

# lsof -i 4 -a -p 9255


list information about TCP sessions on your server (specifically SSH in this example)
# lsof -i tcp@`hostname`:22

COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
sshd2   7585 root    5u  IPv4  16105       TCP localbox:ssh->your.src.ip.here:5897 (ESTABLISHED)
sshd2   7653 root    5u  IPv4  16188       TCP localbox:ssh->your.src.ip.here:2262 (ESTABLISHED)

list information about all TCP session
# lsof -i tcp@`hostname`

list information about all sockets using port 53 (will display named information on UDP/TCP)
# lsof -i @`hostname`:53

list information about all UDP sessions
# lsof -i udp@`hostname`

will list all open files with "ssh" in them
# lsof -c ssh

list everything but with UIDs insted of the UID name from /etc/passwd
# lsof -l

list all open files with "ssh" and only the UIDs
# lsof -l -c ssh

list all open files for the /tmp dir (very slow), but good for finding that nasty process that's holding a file open (although:  fuser -m /tmp, will do the same thing)
# lsof +D /tmp

Tuesday, December 20, 2011

Android OS Versions Comparision

Android OS chart: Comparing the major versions

Android-OS-Comparison-Guide-Nov

Ice Cream Sandwich: What to expect from Android 4.0

Why is it called Ice Cream Sandwich?
Google has named each version of Android after a dessert item, in alphabetical order. There was once a version of Android named Cupcake and later versions named Donut and Eclair. A few phones sold on the market today are still running Froyo, and Gingerbread is currently used on the majority of new Android phones. The version known as Honeycomb was designed for and is used on many new Android tablets.
Why would I want Ice Cream Sandwich?
Ice Cream Sandwich is a big deal, and it differs from every Android operating system before it. Ice Cream Sandwich blends parts of Gingerbread and parts of Honeycomb into a single version of Android that can be used on both tablets and smartphones.

Android owners have long been plagued by something called fragmentation. With fragmentation, software that is written for one version of Android might not look so hot or work the way you expect it to on a device running another version of the OS. With different manufacturers releasing phones with a wide range of specs and custom software, it can be hard for developers to create apps that work for all phones and for phone makers to create phones that can be easily updated to new versions of Android.
Ice Cream Sandwich is expected to unify Android as a whole and set a few norms that should make the updating and development processes a little easier. The OS also offers quite a few new innovative features that are well worth a look.
Visual updates: If you're already an Android user, then Ice Cream Sandwich is going to look a lot different than what you've become accustomed to over the years. Google has completely overhauled the OS to make it more user-friendly, giving the entire operating system a new modern font and a magazine-like feel.
Improved camera: The camera app in Ice Cream Sandwich comes with a number of built-in photo editing features, so you can adjust your pictures directly on your phone. Improvements include basic things like red-eye reduction, cropping and straightening, and Hipster filters that add Instagram-like effects to your photos. The camera also has a single-motion panoramic mode, allowing you to take panoramic shots quickly and easily.
Voice typing: Android has had voice recognition capabilities for a while now, but Ice Cream Sandwich takes things a bit further by allowing you to say what you want to type. The feature will likely come in handy when you need to send a message while driving or when you need to keep your hands elsewhere.
Improved contacts and voicemail: Ice Cream Sandwich improves how Android displays your contacts with the People app. The app lets you see at a glance how you're connected to someone, as well as check out that person's latest moves on social networking sites. The OS also integrates visual voicemail, which lets you view and play each of your voicemails individually rather than having to listen to all of your messages at once. Voicemail can be sped up or slowed down while you're listening.
Face unlock: More a parlor trick than an actual security feature, phones and tablets running Ice Cream Sandwich can be unlocked using your face rather than a secret code. Early tests of the feature have shown that a phone can be unlocked with a picture of you rather than your actual face, and your phone may have trouble recognizing you depending on the lighting where you are and whether or not you're wearing makeup. Still, it's a clever little trick to have up your smartphone sleeve.
Android Beam: Android Beam lets you "beam" content back and forth between two phones by holding them together. For instance, if you had pulled up directions to a restaurant and wanted to share them with a friend, you can hold your phone up to your friend's handset and use Android Beam to load the exact same map on your friend's phone.
Data usage graph: With the rising cost of data plans and the extinction of the unlimited plan, monitoring the amount of data you use on your phone has become more important. The data usage graph in Ice Cream Sandwich helps you track your data usage, as well as see what apps are using the biggest share of your monthly plan, so you can zero in on any problems you have. Set up alerts to let you know when you're reaching your monthly data allotment and disable the data hogs.
Other improvements: Ice Cream Sandwich adds a number of other small tweaks and improvements such as the ability to take screenshots, save web pages to view offline, and create folders for apps. There's also a new calendar app with pinch-to-zoom functionality.
Will all phones have Ice Cream Sandwich?
The first phone to be released running Ice Cream Sandwich was the Galaxy Nexus. Most Android phones and tablets released in 2012 will likely be running the operating system; however, we're still likely to see at least a few phones running Gingerbread and tablets running Honeycomb come to stores as well.
In order for a manufacturer to use Ice Cream Sandwich on a phone, the phone maker has to test the OS on the handset and make sure it works with any of its own software it wants to put on the phone. (In the phone world, the software a manufacturer puts on top of the operating system on its phones is called a skin.)
The Galaxy Nexus is pure Android in that it doesn't have any type of skin on top of Ice Cream Sandwich. Those phones that do have skins, such as handsets made by HTC, Motorola, and others by Samsung, will need to be tested running the OS and their software customized before they can be released. Depending how that process goes, some new devices may be released running an older version of Android, and your existing Android device may or may not wind up getting the update.