http://linoxide.com/guide/linux-cheat-sheet.pdf
http://www.nixtutor.com/linux/all-the-best-linux-cheat-sheets/
http://www.nixtutor.com/linux/all-the-best-linux-cheat-sheets/
Showing posts with label Commands. Show all posts
Showing posts with label Commands. Show all posts
Thursday, March 27, 2014
Sunday, February 12, 2012
sed and awk usage
# # # # # # # # # # # # # # # # # #
A description of the basic special characters
# # # # # # # # # # # # # # # # # #
. (Dot)
Newline character (\ n, newline) would match any character except:
However, awk on the line (\ n) does not specify only the newline character (\ n) is the match come.
* (Asterisk)
General characters that match the regular expression does not have a limit on the number of characters (0 to infinity).
[...] (Brackets)
All characters in parentheses find a character that matches. For example, [aE] if he, a look or E character.
- (Hyphen) when using a specific character to a certain point between all the characters are.
In parentheses, the function of all meta-characters lose their will. ^ Just as the character with the exception of the features there are also.
^ (Caret) inside the parentheses! (Not) have the capability. For example, [^] is the 'non-whitespace character is any character point.
^ (Caret)
The beginning of each line indicates. This is not the first character is also important to know.
Only the beginning of each line will be called. The following is not a newline character in awk.
$ (Dollar)
Marks the end of each line. ^ (Caret) as well as the transfer is not a newline character in awk.
\ {N, m \} (middle brackets)
Single characters, meta characters shall specify the area that matches the number.
\ {N \}, saying that the dog is exactly n,
\ {N, \} n-2, or
\ {, M \} refers to the m or less,
\ {N, m \} is at least n, m more or less says.
\ (Back slash)
Next character is a special character (Special Character) Indicate that you are.
# # # # # # # # # # # # # # # # # #
# # # # # # # # # # # # # # # # # #
Replace
:% S / boy / girl / / / edit mode the letters in the file, the entire row of the boy and girl with the task of replacing the letters.
: 5,10 s / boy / girl / g / / 5 through 10 letters to change the row, you could come out in a line more than once to change the option g.
:., 20s/boy/girl / / / current row. Up to 20 rows.
:., $ S / boy / girl / / / current row. In the last row.
:., / Man / s / boy / girl / / / current row. In the next row that contains the letters of the man.
# # # # # # # # # # # # # # # # # #
# # # # # # # # # # # # # # # # # #
sed [-e script] [-f script-file] [file ...]
Basic functions have been adopted from the ed, sed all these features are applied to. However, the interactive editor ed is
sed is a stream editor. Streaming, interactive editor and editor of the difference between the input and output is an interactive editor
One is made, a stream editor resulted in the input is the output of one.
\ N for newline character is used as a streaming editor.
Find (search), output (print),
sed-n '/ abd / p' list.txt: list.txt file while reading line by line (-n: Do not print that reads) abd character finds the line output (p) is.
Replacement (substitute),
sed 's / addrass / address /' list.txt: addrass a change in address. However, the original file without changing the output is reversed.
sed 's / addrass / address /' list.txt> list2.txt
sed 's / \ t / \ /' list.txt: Convert Enterprise taepmunjareul
sed 's / □ □ * / □ /' list.txt: (* display: □ indicates a space character.) one or more white space the above syntax string and turns it into one space.
Add (insert)
scriptfile - s /
sed-f scriptfile 1.html:
Delete (delete)
sed '/ TD / d' 1.html: TD Delete the line containing the character to the output.
sed '/ Src /! d' 1.html: Src characters do not delete the line.
sed '1, 2d '1.html: the first one line, two lines, delete.
sed '/ ^ $ / d 1.html: is the command to delete the blank line
Regular expressions to extract only the file name
s / ^. * \ / \ ([a-zA-Z0-9.] * \) ". * $ / \ 1 /: ^ at the beginning of the line. * any string, \ (, \) the regular expressions, grouping, and the top end of the line $.
(S; ^. * \ / \ ([A-zA-Z0-9.] * \) ". * $; \ 1 ;) \ 1 refers to the first element of the grouped.
[A-zA-Z0-9.], And the letters and numbers. (Comma) character to represent (character) says.
That is the same string as the first group GF02.jpg and I grouped all of the following line is to change the contents.
/ G: global meaning when there are multiple characters on each line target is to handle.
who | sed-e 's;. * $;;': the first space of each line deleted from the final make up.
who | sed-e 's; ^. *;;': from the beginning of each line, delete the last to make space.
who | sed-e 's; ^. *:;;': from the beginning of each line: where there is a character (:) character) to make deleted.
The-n option
sed on the input received is always stdout found that each line represents. However, sometimes a few lines from a file to extract a sed there are times when you want to use. In this case, use the-n option. This option is exactly what you tell me if you need to print any line you do not want to tell sed. Therefore p is used as the command. Line number and line number by showing a range of lines of text using sed optionally be able to print. As you can see in the next, the first two lines from a file was printed.
$ Sed-n '1, 2p 'intro Just print the first 2 lines from intro file.
Surrounded by a forward slash instead of the line number sign p command if used with a string of characters that includes sed them through the lines of standard input will be printed. Thus, the first two lines from one file to print can be used as follows.
$ Sed-n '/ UNIX / p' intro Just print lines containing UNIX
sed '5 d ': line 5, delete
sed '/ [Tt] est / d': Test or delete any line that contains the test
sed-n '20, 25p 'text: text only print from the 20-25 to the line
sed '1, 10s/unix/UNIX/g 'intro: intro of the first 10 lines of changes to the UNIX unix
sed '/ jan/s/-1/-5': jan all the lines that contain the first change of -1 to -5
sed 's / ... / /' data: each data line, delete them from the first three characters
sed 'S / ... $ / /' data: each data line is deleted from the final 3 characters
sed-n '1 'text: a non-printing characters \ nn as (where nn is the character of the octal value),
And tab characters> to print all lines from each text that represents
# # # # # # # # # # # # # # # # # #
awk command
# # # # # # # # # # # # # # # # # #
awk '/ west /' datafile: west line output in the article
awk '/ ^ north /' datafile: north of the line that begins with the output
awk '/ ^ (no | so) /' datafile: no, or so the line that starts with the output
awk '{print $ 3, $ 2}' datafile: datafile list, and the second field of the third to the clearance space output
awk '{print $ 3 $ 2}' datafile: datafile third and second fields of the list, just attach the output
awk '{print "Number of fields:" NF}' datafile: datafile number of fields each of which returns julmadaui.
awk '$ 5 ~ / \. [7-9] + /' datafile: fifth field is a number between 7 and 9 periods daeumet the record that at least one output
awk '$ 2! ~ / E / {print $ 1, $ 2}' datafile: E pattern in the second field of the record is not the first and second field output
awk '$ 3 ~ / ^ Joel / {print $ 3 "is a nice guy."}' datafile: Joel Starting with the third field "is a nice guy" with the output
awk '$ 8 ~ / [0-9] [0-9] $ / {print $ 8}' datafile: If two numbers, the eighth field is the field output
awk '$ 4 ~ / Chin $ / {print "The price is $" $ 8 "." } 'Datafile: After the fourth field in Chine "The price is $" 8 times the output field, and the period
awk-F: '{print $ 1}' datafile:-F option to enter the field ':' as a distinction.
awk-F "[:]" '{print $ 1, $ 2}' datafile: input field space and ':' as the field delimiter
awk-f awk_script.file datafile:-f option when using the awk script file this content.
awk '$ 7 == 5' datafile: 7 times the field is equal to 5, the output
awk '$ 2 == "CT" {print $ 1, $ 2}' datafile: 2 times the field "CT" character is equal to 1 and 2 field output
awk '$ 7 <5 {print $ 4, $ 7}' datafile: 7 times a field is less than 5, 4, 7 field output
awk '$ 6> .9 {print $ 1, $ 6}' datafile: 6 times a field is greater than 0.9 times, six times the output
awk '$ 8> 10 && $ 8 <17' datafile
awk '$ 2 == "NW" | | $ 1 ~ / south / {print $ 1, $ 2}' datafile
Redirecting the output / input
File descriptor 0: standard input
File descriptor 1: The standard output
The file descriptor 2: The standard error output
$ Kill -1 1234> killout.txt 2> & 1: killout.txt as standard output and error output means saying.
$ Kill -1 1234 1> & 2: the standard error output to the output means saying.
Friday, February 10, 2012
What is strace?
Strace is quite simply a tool that traces the execution of system calls. In its simplest form it can trace the execution of a binary from start to end, and output a line of text with the name of the system call, the arguments and the return value for every system call over the lifetime of the process.
But it can do a lot more:
- It can filter based on the specific system call or groups of system calls
- It can profile the use of system calls by tallying up the number of times a specific system call is used, and the time taken, and the number of successes and errors.
- It traces signals sent to the process.
- It can attach to any running process by pid.
If you've used other Unix systems, this is similar to "truss". Another (much more comprehensive) is Sun's Dtrace.
How to use it
This is just scratching the surface, and in no particular order of importance:
1) Find out which config files a program reads on startup
Ever tried figuring out why some program doesn't read the config file you thought it should? Had to wrestle with custom compiled or distro-specific binaries that read their config from what you consider the "wrong" location?
The naive approach:
$ strace php 2>&1 | grep php.ini
open("/usr/local/bin/php.ini", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/local/lib/php.ini", O_RDONLY) = 4
lstat64("/usr/local/lib/php.ini", {st_mode=S_IFLNK|0777, st_size=27, ...}) = 0
readlink("/usr/local/lib/php.ini", "/usr/local/Zend/etc/php.ini", 4096) = 27
lstat64("/usr/local/Zend/etc/php.ini", {st_mode=S_IFREG|0664, st_size=40971, ...}) = 0
So this version of PHP reads php.ini from /usr/local/lib/php.ini (but it tries /usr/local/bin first).
The more sophisticated approach if I only care about a specific syscall:
$ strace -e open php 2>&1 | grep php.ini
open("/usr/local/bin/php.ini", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/local/lib/php.ini", O_RDONLY) = 4
The same approach work for a lot of other things. Have multiple versions of a library installed at different paths and wonder exactly which actually gets loaded? etc.
2) Why does this program not open my file?
Ever run into a program that silently refuse to read a file it doesn't have read access to, but you only figured out after swearing for ages because you thought it didn't actually find the file? Well, you already know what to do:
$ strace -e open,access 2>&1 | grep your-filename
Look for an open() or access() syscall that fails
3) What is that process doing RIGHT NOW?
Ever had a process suddenly hog lots of CPU? Or had a process seem to be hanging?
Then you find the pid, and do this:
root@dev:~# strace -p 15427 Process 15427 attached - interrupt to quit futex(0x402f4900, FUTEX_WAIT, 2, NULLProcess 15427 detached
Ah. So in this case it's hanging in a call to futex(). Incidentally in this case it doesn't tell us all that much - hanging on a futex can be caused by a lot of things (a futex is a locking mechanism in the Linux kernel). The above is from a normally working but idle Apache child process that's just waiting to be handed a request.
But "strace -p" is highly useful because it removes a lot of guesswork, and often removes the need for restarting an app with more extensive logging (or even recompile it).
4) What is taking time?
You can always recompile an app with profiling turned on, and for accurate information, especially about what parts of your own code that is taking time that is what you should do. But often it is tremendously useful to be able to just quickly attach strace to a process to see what it's currently spending time on, especially to diagnose problems. Is that 90% CPU use because it's actually doing real work, or is something spinning out of control.
Here's what you do:
root@dev:~# strace -c -p 11084 Process 11084 attached - interrupt to quit Process 11084 detached % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 94.59 0.001014 48 21 select 2.89 0.000031 1 21 getppid 2.52 0.000027 1 21 time ------ ----------- ----------- --------- --------- ---------------- 100.00 0.001072 63 total root@dev:~#
After you've started strace with -c -p you just wait for as long as you care to, and then exit with ctrl-c. Strace will spit out profiling data as above.
In this case, it's an idle Postgres "postmaster" process that's spending most of it's time quietly waiting in select(). In this case it's calling getppid() and time() in between each select() call, which is a fairly standard event loop.
You can also run this "start to finish", here with "ls":
root@dev:~# strace -c >/dev/null ls % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 23.62 0.000205 103 2 getdents64 18.78 0.000163 15 11 1 open 15.09 0.000131 19 7 read 12.79 0.000111 7 16 old_mmap 7.03 0.000061 6 11 close 4.84 0.000042 11 4 munmap 4.84 0.000042 11 4 mmap2 4.03 0.000035 6 6 6 access 3.80 0.000033 3 11 fstat64 1.38 0.000012 3 4 brk 0.92 0.000008 3 3 3 ioctl 0.69 0.000006 6 1 uname 0.58 0.000005 5 1 set_thread_area 0.35 0.000003 3 1 write 0.35 0.000003 3 1 rt_sigaction 0.35 0.000003 3 1 fcntl64 0.23 0.000002 2 1 getrlimit 0.23 0.000002 2 1 set_tid_address 0.12 0.000001 1 1 rt_sigprocmask ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000868 87 10 total
Pretty much what you'd expect, it spents most of it's time in two calls to read the directory entries (only two since it was run on a small directory).
5) Why the **** can't I connect to that server?
Debugging why some process isn't connecting to a remote server can be exceedingly frustrating. DNS can fail, connect can hang, the server might send something unexpected back etc. You can use tcpdump to analyze a lot of that, and that too is a very nice tool, but a lot of the time strace will give you less chatter, simply because it will only ever return data related to the syscalls generated by "your" process. If you're trying to figure out what one of hundreds of running processes connecting to the same database server does for example (where picking out the right connection with tcpdump is a nightmare), strace makes life a lot easier.
This is an example of a trace of "nc" connecting to www.news.com on port 80 without any problems:
$ strace -e poll,select,connect,recvfrom,sendto nc www.news.com 80
sendto(3, "\24\0\0\0\26\0\1\3\255\373NH\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, 28) = 0
poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
sendto(3, "\213\321\1\0\0\1\0\0\0\0\0\0\3www\4news\3com\0\0\34\0\1", 30, MSG_NOSIGNAL, NULL, 0) = 30
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
recvfrom(3, "\213\321\201\200\0\1\0\1\0\1\0\0\3www\4news\3com\0\0\34\0\1\300\f"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, [16]) = 153
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, 28) = 0
poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
sendto(3, "k\374\1\0\0\1\0\0\0\0\0\0\3www\4news\3com\0\0\1\0\1", 30, MSG_NOSIGNAL, NULL, 0) = 30
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
recvfrom(3, "k\374\201\200\0\1\0\2\0\0\0\0\3www\4news\3com\0\0\1\0\1\300\f"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, [16]) = 106
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, 28) = 0
poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
sendto(3, "\\\2\1\0\0\1\0\0\0\0\0\0\3www\4news\3com\0\0\1\0\1", 30, MSG_NOSIGNAL, NULL, 0) = 30
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
recvfrom(3, "\\\2\201\200\0\1\0\2\0\0\0\0\3www\4news\3com\0\0\1\0\1\300\f"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("62.30.112.39")}, [16]) = 106
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("216.239.122.102")}, 16) = -1 EINPROGRESS (Operation now in progress)
select(4, NULL, [3], NULL, NULL) = 1 (out [3])
So what happens here?
Notice the connection attempts to /var/run/nscd/socket? They mean nc first tries to connect to NSCD - the Name Service Cache Daemon - which is usually used in setups that rely on NIS, YP, LDAP or similar directory protocols for name lookups. In this case the connects fails.
It then moves on to DNS (DNS is port 53, hence the "sin_port=htons(53)" in the following connect. You can see it then does a "sendto()" call, sending a DNS packet that contains www.news.com. It then reads back a packet. For whatever reason it tries three times, the last with a slightly different request. My best guess why in this case is that www.news.com is a CNAME (an "alias"), and the multiple requests may just be an artifact of how nc deals with that.
Then in the end, it finally issues a connect() to the IP it found. Notice it returns EINPROGRESS. That means the connect was non-blocking - nc wants to go on processing. It then calls select(), which succeeds when the connection was successful.
Try adding "read" and "write" to the list of syscalls given to strace and enter a string when connected, and you'll get something like this:
read(0, "test\n", 1024) = 5
write(3, "test\n", 5) = 5
poll([{fd=3, events=POLLIN, revents=POLLIN}, {fd=0, events=POLLIN}], 2, -1) = 1
read(3, "
This shows it reading "test" + linefeed from standard in, and writing it back out to the network connection, then calling poll() to wait for a reply, reading the reply from the network connection and writing it to standard out. Everything seems to be working right.
Friday, January 27, 2012
7 Linux chkconfig Command Examples – Add, Remove, View, Change Services
Chkconfig command is used to setup, view, or change services that are configured to start automatically during the system startup.
This article contains 7 practical examples that explains how to use the chkconfig command.
1. Check Service Startup status from Shell Script
When you execute chkconfig command only with the service name, it returns true if the service is configured for startup. The following code snippet shows how to check whether a service is configured for startup or not from a shell script.
# vi check.sh chkconfig network && echo "Network service is configured" chkconfig junk && echo "Junk service is configured" # ./check.sh Network service is configured
You can also specifically check whether it is configured for a particular run level or not.
# vi check1.sh chkconfig network --level 3 && echo "Network service is configured for level 3" chkconfig network --level 1 && echo "Network service is configured for level 1" # ./check1.sh Network service is configured for level 3
2. View Current Status of Startup Services
The –list option displays all the services with the current startup configuration status.
# chkconfig --list abrtd 0:off 1:off 2:off 3:on 4:off 5:on 6:off acpid 0:off 1:off 2:off 3:off 4:off 5:off 6:off atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off ...
To view only the services that are configured to be started during system startup, do the following. Please note that this assumes that your system startup level is 3.
chkconfig --list | grep 3:on
To view the startup configuration of a particular service, grep the output of ‘chkconfig –list’ for that service.
chkconfig --list | grep network
3. Add a new Service to the Startup
Use –add option to add a specific service to the list of services that will be started during system reboot.
The following example shows how to add a new service (for example, iptables) to the list of services that needs to be started. The ‘chkconfig –add’ command will also turn on level 2, 3, 4 and 5 automatically as shown below.
# chkconfig --list | grep iptables # chkconfig --add iptables # chkconfig --list | grep iptables iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Note: “chkconfig –add” only adds an existing service to the list of startup. If the service doesn’t exist, you should first install it before adding it to the system startup list. While this is pretty obvious, it is worth to mention it, as a newbie might make this mistake.
4. Remove a Service From Startup List
The following example shows that ip6tables services is configured for startup.
# chkconfig --list | grep ip6tables ip6tables 0:off 1:off 2:off 3:on 4:off 5:off 6:off
To remove it from the startup list, use the –del option as shown below.
# chkconfig --del ip6tables # chkconfig --list | grep ip6tables
5. Turn-on or Turn-off a Service for a Selected Run Level
Sometimes you might not want to delete the whole service. Instead, you might just want to turn the flag on or off for a particular run level (for a particular service).
The following example will turn off nfserver service for level 5
# chkconfig --level 5 nfsserver off
You can also combine multiple levels. The following example will turn off nfsserver for both level 3 and 5.
# chkconfig --level 35 nfsserver off
6. Script Files under rc.d Subdirectories
Whenever you add or remove a service from chkconfig control, it does the following under the /etc/rc.d sub-directories.
- When chkconfig –add command is executed, it creates a symbolic link file to start and stop the service under corresponding rc directory.
- When chkconfig –del command is executed, it removes the symbolic link file from the corresponding rc directory.
The following example shows that xinetd is enabled for both run level 3 and 5.
So, xinetd will have two files under rc3.d directory, and two files under rc5.d directory. The file that starts with K is used during shutdown (K stands for kill). The file that starts with S is used during startup (S stands for start).
# chkconfig --list | grep xinetd xinetd 0:off 1:off 2:off 3:on 4:off 5:on 6:off xinetd based services: # cd /etc/rc.d/rc3.d # ls | grep xinetd K08xinetd S14xinetd # cd /etc/rc.d/rc5.d # ls | grep xinetd K08xinetd S14xinetd
7. rcx.d Directory Changes for Add Operation
When you add a new service to chkconfig control, the default run levels for that service will be turned on automatically, and files will be created under the corresponding rcx directories.
For example, if the nfsserver service doesn’t exist in the chkconfig control, no nfsserver service startup files would be present under /etc/rc.d/rc*.d directories as shown below.
# chkconfig --list | grep nfsserver nfsserver 0:off 1:off 2:off 3:off 4:off 5:off 6:off # ls /etc/rc.d/rc3.d | grep nfsserver # ls /etc/rc.d/rc5.d | grep nfsserver
After you add the nfsserver service, you’ll see the symbolic links under these directories.
# chkconfig --add nfsserver nfsserver 0:off 1:off 2:off 3:on 4:off 5:on 6:off # cd /etc/rc.d/rc3.d # ls -l | grep nfsserver lrwxrwxrwx 1 root root 12 2011-06-18 00:52 K08nfsserver -> ../nfsserver lrwxrwxrwx 1 root root 12 2011-06-18 00:52 S14nfsserver -> ../nfsserver # cd /etc/rc.d/rc5.d # ls -l | grep nfsserver lrwxrwxrwx 1 root root 12 2011-06-18 00:52 K08nfsserver -> ../nfsserver lrwxrwxrwx 1 root root 12 2011-06-18 00:52 S14nfsserver -> ../nfsserver
When you turn off the service either using –del option or –level option, the corresponding symbolic link file under rcx.d directory will be deleted as shown below.
# chkconfig --level 5 nfsserver off # ls /etc/rc.d/rc5.d | grep nfsserver
Subscribe to:
Comments (Atom)