Bandwidth vs Latency

Latency versus Bandwidth - What is it?

One of the most commonly misunderstood concepts in networking is speed and capacity. Most people believe that capacity and speed are the same thing. For example, it's common to hear "How fast is your connection?" Invariably, the answer will be "640K", "1.5M" or something similar. These answers are actually referring to the bandwidth or capacity of the service, not speed.

Speed and bandwidth are interdependent. The combination of latency and bandwidth gives users the perception of how quickly a webpage loads or a file is transferred. It doesn't help that broadband providers keep saying "get high speed access" when they probably should be saying "get high capacity access". Notice the term "Broadband" - it refers to how wide the pipe is, not how fast.

Latency:

Latency is delay.

For our purposes, it is the amount of time it takes a packet to travel from source to destination. Together, latency and bandwidth define the speed and capacity of a network.

Latency is normally expressed in milliseconds. One of the most common methods to measure latency is the utility ping. A small packet of data, typically 32 bytes, is sent to a host and the RTT (round-trip time, time it takes for the packet to leave the source host, travel to the destination host and return back to the source host) is measured.

The following are typical latencies as reported by others of popular circuits type to the first hop. Please remember however that latency on the Internet is also affected by routing that an ISP may perform (ie, if your data packet has to travel further, latencies increase).

Ethernet                  .3ms
Analog Modem              100-200ms
  ISDN                      15-30ms
DSL/Cable                 10-20ms
Stationary Satellite      >500ms, mostly due to high orbital elevation
  
DS1/T1                    2-5ms

Bandwidth:

Bandwidth is normally expressed in bits per second. It's the amount of data that can be transferred during a second.

Solving bandwidth is easier than solving latency. To solve bandwidth, more pipes are added. For example, in early analog modems it was possible to increase bandwidth by bonding two or more modems. In fact, ISDN achieves 128K of bandwidth by bonding two 64K channels using a datalink protocol called multilink-ppp.

Bandwidth and latency are connected. If the bandwidth is saturated then congestion occurs and latency is increased. However, if the bandwidth of a circuit is not at peak, the latency will not decrease. Bandwidth can always be increased but latency cannot be decreased. Latency is the function of the electrical characteristics of the circuit.

Bandwidth vs. Latency

There are two factors in online gaming when it comes to your internet connection: bandwidth and latency.  Let's review on what each of them means:

    * Bandwidth – The average rate of successful data transfer through a communications path (usually measured in KB/s or Mb/s – that's megabits, not megabytes!  See below) 

    * Latency – The time it takes for data to be passed from the source to a server and back again (usually measured in milliseconds)

It's the Latency, Stupid

Stuart Cheshire, May 1996.
(Revised periodically)
Copyright © Stuart Cheshire 1996-2001
Years ago David Cheriton at Stanford taught me something that seemed very obvious at the time -- that if you have a network link with low bandwidth then it's an easy matter of putting several in parallel to make a combined link with higher bandwidth, but if you have a network link with bad latency then no amount of money can turn any number of them into a link with good latency.
It's now many years later, and this obvious fact seems lost on the most companies making networking hardware and software for the home. I think it's time it was explained again in writing.

Fact One: Making more bandwidth is easy.

Imagine you live in a world where the only network connection you can get to your house is a 33kbit/sec modem running over a telephone line. Imagine that this is not enough for your needs. You have a problem.
The solution is easy. You can get two telephone lines, and use them together in parallel, giving you a total of 66kbit/sec. If you need even more you can get ten telephone lines, giving you 330kbit/sec. Sure, it's expensive, and having ten modems in a pile is inconvenient, and you may have to write your own networking software to share the data evenly between the ten lines, but if it was important enough to you, you could get it done.
It may not be cheap, but at least it's possible.
People with ISDN lines can already do this. It's called "bonding" and it uses two 56 (or 64) kbit/sec ISDN channels in parallel to give you a combined throughput of 112 (or 128) kbit/sec.

Fact Two: Once you have bad latency you're stuck with it.

If you want to transfer a large file over your modem it might take several seconds, or even minutes. The less data you send, the less time it takes, but there's a limit. No matter how small the amount of data, for any particular network device there's always a minimum time that you can never beat. That's called the latency of the device. For a typical Ethernet connection the latency is usually about 0.3ms (milliseconds -- thousandths of a second). For a typical modem link the latency is usually about 100ms, about 300 times worse than Ethernet.
If you wanted to send ten characters over your 33kbit/sec modem link you might think the total transmission time would be:

80 bits / 33000 bits per second = 2.4ms.

but it doesn't. It takes 102.4ms because of the 100ms latency introduced by the modems at each end of the link.
If you want to send a large amount of data, say 100K, then that takes 25 seconds, and the 100ms latency isn't very noticable, but if you want send a smaller amount of data, say 100bytes, then the latency is more than the transmission time.
Why would you care about this? Why do small pieces of data matter? For most end-users it's the time it takes to transfer big files that annoys them, not small files, so they don't even think about latency when buying products. In fact if you look at the boxes modems come in, they proudly proclaim "14.4 kbps", "28.8 kbps" and "33.6 kbps", but they don't mention the latency anywhere. What most end-users don't know is that in the process of transferring those big files their computers have to send back and forth hundreds of little control messages, so the performance of small data packets directly affects the performance of everything else they do on the network.
Now, imagine the same scenario as before. You live in a world where the only network connection you can get to your house is a modem running over a telephone line. Your modem has a latency of 100ms, but you're doing something that needs lower latency. Maybe you're trying to do computer audio over the net. 100ms may not sound like very much, but it's enough to cause a noticable delay and echo in voice communications, which makes conversation difficult. Maybe you're trying to play an interactive game over the net. The game only sends tiny amounts of data, but that 100ms delay is making the interactivity of the game decidedly sluggish.
What can you do about this?
Nothing.
You can compress the data, but it doesn't help. It was already small to start with, and that 100ms latency is still there.
You can get 80 phone lines in parallel, and send one single bit over each phone line, but that 100ms latency is still there.
Once you've got yourself a device with bad latency there's absolutely nothing you can do about it (except throw out the device and get something else).

Fact Three: Current consumer devices have appallingly bad latency.

A typical Ethernet card has a latency less than 1ms. The Internet backbone as a whole also has very good latency. Here's a real-world example:

• The distance from Stanford to Boston is 4320km.
• The speed of light in vacuum is 300 x 10^6 m/s.
• The speed of light in fibre is roughly 66% of the speed of light in vacuum.
• The speed of light in fibre is 300 x 10^6 m/s * 0.66 = 200 x 10^6 m/s.
• The one-way delay to Boston is 4320 km / 200 x 10^6 m/s = 21.6ms.
• The round-trip time to Boston and back is 43.2ms.
• The current ping time from Stanford to Boston over today's Internet is about 85ms:

  [cheshire@nitro]$ ping -c 1 lcs.mit.edu
  PING lcs.mit.edu (18.26.0.36): 56 data bytes
  64 bytes from 18.26.0.36: icmp_seq=0 ttl=238 time=84.5 ms

• So: the hardware of the Internet can currently achieve within a factor of two of the speed of light.

So the Internet is doing pretty well. It may get better with time, but we know it can never beat the speed of light. In other words, that 85ms round-trip time to Boston might reduce a bit, but it's never going to beat 43ms. The speed's going to get a bit better, but it's not going to double. We're already within a factor of two of the theoretical optimum. I think that's pretty good. Not many technologies can make that claim.
Compare this with a modem. Suppose you're 18km from your ISP (Internet Service Provider). At the speed of light in fibre (or the speed of electricity in copper, which is about the same) the latency should be:

18000 / (180 x 10^6 m/s) = 0.1ms

The latency over your modem is actually over 100ms. Modems are currently operating at level that's 1000 times worse than the speed of light. They have a long way to go before they get close to what the rest of the Internet is achieving.
Of course no modem link is ever going to have a latency of 0.1ms. I'm not expecting that. The important issue is the total end-to-end transmission delay for a packet -- the time from the moment the software makes the call to send the packet to the moment the last bit of the packet arrives the destination and the packet delivered to the software at the receiving end. The total end-to-end transmission delay is made up of fixed latency (including the speed-of-light propagation delay), plus the transmission time. For a 36 byte packet the transmission time is 10ms (the time it takes to send 288 bits at a rate of 28800 bits per second). When the actual transmission time is about 10ms, working to make the latency 0.1ms would be silly. All that's needed is that the latency should not be so huge that it completely overshadows the transmission time. For a modem that has a transmission rate of 28.8kb/s, a sensible latency target to aim for is about 5ms.

Fact Four: Making limited bandwidth go further is easy.

If you know you have limited bandwidth, there are many techniques you can use to reduce the problem.

Compression

If you know you have limited bandwidth, compression is one easy solution.
You can apply general purpose compression, such as gzip, to the data.
Even better, you can apply data-specific compression, because that gets much higher compression ratios. For example, still pictures can be compressed with JPEG, Wavelet compression, or GIF. Moving pictures can be compressed with MPEG, Motion JPEG, Cinepak, or one of the other QuickTime codecs. Audio can be compressed with uLaw, and English text files can be compressed with dictionary-based compression algorithms.
All of these compression techniques trade off use of CPU power in exchange for lower bandwidth requirements. There's no equivalent way to trade off use of extra CPU power to make up for poor latency.
All modern modems have compression algorithms built-in. Unfortunately, having your modem do compression is nowhere near as good as having your computer do it. Your computer has a powerful, expensive, fast processor in it. Your modem has a feeble, cheap, slow processor in it. There's no way your modem can compress data as well or as quickly as your computer can. In addition, in order to compress data, your modem has to hold on to the data until it has a block that's big enough to compress effectively. That adds latency, and once added, there's no way for you to get rid of latency. Also, the modem doesn't know what kind of data you are sending, so it can't use the superior data-specific compression algorithms. In fact, since most images and sounds on Web pages are compressed already, the modem's attempts to compress the data a second time is futile, and just adds more latency without giving any benefit.
This is not to say that having a modem do compression never helps. In the case where the host software at the endpoints is not very smart, and doesn't compress its data appropriately, then the modem's own compression can compensate somewhat for that deficiency and improve throughput. The point is that modem compression only helps dumb software, and it actually hurts smart software by adding extra delay. For someone planning to write dumb software this is no problem. For anyone planning to write smart software this should be a big cause for concern.

Bandwidth-conscious code

Another way to cope with limited bandwidth is to write programs that take care not to waste bandwidth.
For example, to reduce packet size, wherever possible Bolo uses bytes instead of 16-bit or 32-bit words.
For many kinds of interactive software like games, it's not important to carry a lot of data. What's important is that when the little bits of data are delivered, they are delivered quickly. Bolo was originally developed running over serial ports at 4800 bps and could support 8 players that way. Over 28.8 modems it can barely support 2 players with acceptable response time. Why? A direct-connect serial port at 4800 bps has a transmission delay of 2ms per byte, and a latency that is also 2ms. To deliver a typical ten byte Bolo packet takes 22ms. A 28800 bps modem has transmission delay of 0.28ms per byte, but a latency of 100ms, 50 times worse than the 4800 bps serial connection. Over the 28.8 modem, it takes 103ms to deliver a ten byte packet.

Send less data

A third way to cope with limited bandwidth is simply to send less data.
If you don't have enough bandwidth to send high resolution pictures, you can use lower resolution.
If you don't have enough bandwidth to send colour images, you can send black and white images, or send images with dramatically reduced colour detail (which is actually what NTSC television does).
If you don't have enough bandwidth to send 30 frames per second, you can send 15fps, or 5fps, or fewer.
Of course these tradeoffs are not pleasant, but they are possible. You can either choose to pay more money to run multiple circuits in parallel for more bandwidth, or you can choose to send less data to stay within the limited bandwidth you have available.
If the latency is not good enough to meet your needs you don't have the same option. Running multiple circuits in parallel won't make your latency any better, and sending less data won't improve it either.

Caching

One of the most effective techniques throughout all areas of computer science is caching, and that is just as true in networking.
If visit a web site, your Web browser can keep a copy of the text and images on your computer's hard disk. If you visit the same site again, all your Web browser has to do check that the copies it has stored are up to date -- i.e. check that the copies on the Web server haven't been changed since the date and time the previous copies were downloaded and cached on the local disk.
Checking the date and time a file was last modified is a tiny request to send across the network. This kind of request is so small that the throughput of your modem makes no difference -- latency is all that matters.
Recently companies have started providing CDROMs of entire Web sites to speed Web browsing. When browsing these Web sites, all the Web browser has to do is check the modification date of each file it accesses to make sure that the copy on the CDROM is up to date. It only has to download files that have changed since the CDROM was made. Since most of the large files on a Web site are images, and since images on a Web site change far less frequently than the HTML text files, in most cases very little data has to be transferred.
Since for the most part the Web browser is only doing small modification date queries to the Web server, the performance the user experiences is entirely dominated by the latency of the connection, and the throughput is virtually irrelevant.

Another analogy

Even smart people have trouble fully grasping the implications of these latency issues. It's subtle stuff.
The Cable TV industry is hyping "cable modems" right now, claiming that they're "1000 times 'faster' than a telephone modem." Given the lack of public awareness of the importance of latency, I wouldn't be in the least surprised if many of them have latency that is just as bad, or maybe even worse, than telephone modems. (The results from some early prototype cable modems, however, look quite promising. Lets hope the production ones are as good.)
Another device in a similar position is the DirecPC satellite dish, which is supposed to be "14 times faster than a 28.8 modem". Is it really? Here are some excerpts of what Lawrence Magid had to say about it in his article in the San Jose Mercury News (2nd February 1997):

The system is expensive, requires a relatively elaborate installation and configuration and, in the end, doesn't necessarily speed up your access to the World Wide Web.
I set up two nearly identical PCs side by side. One was connected to the Net at 28.8kbps and the other with DirecPC. In most cases the satellite system displayed Web pages a bit faster than the one with a modem, but not by much.
In some cases, the modem-equipped PC was faster, especially with sites that don't have a great deal of graphics.
Alluring as its promise may be, DirecPC for now doesn't offer spectacular advantages for normal Web surfing, even though it does carry a spectacular price.

Do we see a pattern starting to emerge yet?
Part of the problem here is misleading use of the word "faster".
Would you say that a Boeing 747 is three times "faster" than a Boeing 737? Of course not. They both cruise at around 500 miles per hour. The difference is that the 747 carries 500 passengers where as the 737 only carries 150. The Boeing 747 is three times bigger than the Boeing 737, not faster.
Now, if you wanted to go from New York to London, the Boeing 747 is not going to get you there three times faster. It will take just as long as the 737.
In fact, if you were really in a hurry to get to London quickly, you'd take Concorde, which cruises around 1350 miles per hour. It only seats 100 passengers though, so it's actually the smallest of the three. Size and speed are not the same thing.
On the other hand, If you had to transport 1500 people and you only had one aeroplane to do it, the 747 could do it in three trips where the 737 would take ten, so you might say the Boeing 747 can transport large numbers of people three times faster than a Boeing 737, but you would never say that a Boeing 747 is three times faster than a Boeing 737.
That's the problem with communications devices today. Manufacturers say "speed" when they mean "capacity". The other problem is that as far as the end-user is concerned, the thing they want to do is transfer large files quicker. It may seem to make sense that a high-capacity slow link might be the best thing for the job. What the end-user doesn't see is that in order to manage that file transfer, their computer is sending dozens of little control messages back and forth. The thing that makes computer communication different from television is interactivity, and interactivity depends on all those little back-and-forth messages.
The phrase "high-capacity slow link" that I used above probably looked very odd to you. Even to me it looks odd. We've been used to wrong thinking for so long that correct thinking looks odd now. How can a high-capacity link be a slow link? High-capacity means fast, right? It's odd how that's not true in other areas. If someone talks about a "high-capacity" oil tanker, do you immediately assume it's a very fast ship? I doubt it. If someone talks about a "large-capacity" truck, do you immediately assume it's faster than a small sports car?
We have to start making that distinction again in communications. When someone tells us that a modem has a speed of 28.8 kbit/sec we have to remember that 28.8 kbit/sec is itscapacity, not its speed. Speed is a measure of distance divided by time, and 'bits' is not a measure of distance.
I don't know how communications came to be this way. Everyone knows that when you buy a hard disk you should check what its seek time is. The maximum transfer rate is something you might also be concerned with, but the seek time is definitely more important. Why does no one think to ask what a modem's 'seek time' is? The latency is exactly the same thing. It's the minimum time between asking for a piece of data and getting it, just like the seek time of a disk, and it's just as important.

Lessons to learn

ISDN has a latency of about 10ms. Its throughput may be twice that of a modem, but its latency is ten times better, and that's the key reason why browsing the web over an ISDN link feels so much better than over a modem. If you have the option of ISDN, and a good ISP that supports it, and it is not too expensive in your area, then get it.
One of the reasons that telephone modems have such poor latency is that they don't know what you're doing with your computer. An external modem is usually connected through a serial port. It has no idea what you are doing, or why. All it sees is an unstructured stream of bytes coming down the serial port.
Ironically, the Apple Geoport telecom adapter, which has suffered so much criticism, may offer an answer to this problem. The Apple Geoport telecom adapter connects your computer to a telephone line, but it's not a modem. All of the functions of a modem are performed by software running on the Mac. The main reason for all the criticism is that running this extra software takes up memory slows down the Mac, but it could also offer an advantage that no external modem could ever match. Because when you use the Geoport adapter the modem software is running on the same CPU as your TCP/IP software and your Web browser, it could know exactly what you are doing. When your Web browser sends a TCP packet, there's no need for the Geoport modem software to mimic the behaviour of current modems. It could take that packet, encode it, and start sending it over the telephone line immediately, with almost zero latency.
Sending 36 bytes of data, a typical game-sized packet, over an Apple Geoport telecom adapter running at 28.8kb/s could take as little as 10ms, making it as fast as ISDN, and ten times faster than the current best modem you can buy. For less than the price of a typical modem the Geoport telecom adapter would give you Web browsing performance close to that of ISDN. Even better, all the people who already own Apple Geoport telecom adapters wouldn't need to buy anything at all -- they'd just need a software upgrade. Even better, Microsoft wouldn't be able to just copy it for Windows like they do with everything else they see on the Mac, because Wintel clones don't have anything like a Geoport for Microsoft to use. What a PR triumph for Apple that would be! It really would show that Apple is the company that understands the Internet. I'm know that in practice there would be other factors that prevent us from getting the delay all the way down to 10ms, but I'm confident that we could get a long way towards that goal.
So far Apple has shown no interest in making use of this opportunity.

Bandwidth Still Matters

Having said all this, you should not conclude that I believe that bandwidth is unimportant. It is very important, but in a way that most people do not think of. Bandwidth is important not only for it's own sake, but also for it's effect on overall latency. As I said above, the important issue is the total end-to-end transmission delay for a packet.
Many people believe that having a private 64kb/sec ISDN connection is just as good, or even better than having a 1/150 share of a 10MB/sec Ethernet. Telephone companies argue that ISDN is just as good as new technologies like cable modems, because while cable modems have much higher bandwidth, that bandwidth is shared between lots of users, so the average works out the same. This idea, that you can average packets as if they were a fluid in a pipe, is flawed, as the following example will show:
Say we have a game where the state of the virtual world amounts to 40K of data. We have a game server, and in this simple example, the game server transmits the entire current game state to the player once every 10 seconds. That's 40K every 10 seconds, or an average of 4K/sec, or 32kb/sec. That's only half the capacity of a 64kb/sec ISDN line, and 150 users doing this on an Ethernet is only half the capacity of the Ethernet. So far so good. Both links are running at only 50% capacity, so the performance should be the same, right? Wrong. On the Ethernet, when the server sends the 40K to a player, the player can receive that data as little as 32ms later (320kb / 10Mb/sec). If the server is not the only machine sending packets on the Ethernet, then there could be contention for the shared medium, but even in that case the average delay before the player receives the data is only 64ms. On the ISDN line, when the server sends the 40K to a player, the player receives that data 5 seconds later (320kb / 64kb/sec). In both cases the users have the same average bandwidth, but the actual performance is very different. In the Ethernet case the player receives the data almost instantly, but in the ISDN case, by the time the player gets the game information it is already 5 seconds out of date.
The standard mistake is to assume that a 40K chunk every ten seconds and a uniform rate of 4K/second are the same thing. They're not. If they were then ISDN, ATM, and all the other telephone company schemes would be good ideas. The telephone companies assume that all communications are like the flow of fluid in a pipe. You just tell them the rate of flow you need, and they tell you how big the pipe has to be. Audio streams, like voice, are like the flow of fluid in a pipe, but computer data is not. Computer data comes in lumps. The standard mistake is to say that if I want to send 60K of data once per minute, that's exactly the same as sending 1K per second. It's not. A 1K per second connection may be sufficient *capacity* to carry the amound of data you're sending, but that doesn't mean it will deliver the 60K lump of data in a timely fashion. It won't. By the time the lump finishes arriving, it will be one minute old. Just because you don't send data very often doesn't mean you want it delivered late. You may only write to your aunt once a year, but that doesn't mean that on the occasions when you do write her a letter you'd like it to take a year to be delivered.
The conclusion here is obvious. If you're given the choice between a low bandwidth private connection, or a small share of a larger bandwidth connection, take the small share.
Again, this is painfully obvious outside the computer world. If a politician said they would build either a large shared freeway, or a million tiny separate private footpaths, one reserved for each citizen, which would you vote for?

Survey

A lot of people have sent me e-mail disputing what I've said here. A lot of people have sent me e-mail simply asserting that their modem isn't slow at all, and the slow performance they see is due to the rest of the Internet being slow, not their modem link.
To try to get to the truth of the matter, I'm going to do a small-scale survey. If you think your modem has low latency, please try an experiment for me. Run a "traceroute" to some destination a little way away. On the West coast of the US lcs.mit.edu might be a good host to trace to. From the East coast of the US you can trace to core-gateway.stanford.edu. In other places, pick a host of your choice (or use one of those two if you like).
On Unix, you can run a trace by typing "traceroute " (if you have traceroute installed). On the Mac, get Peter Lewis's Mac TCP Watcher and click the "Trace" button. On Windows '95, you have to open a DOS window and type a command like in Unix, except on Windows '95 the "traceroute" command is called "TRACERT". Jack Richard wrote a good article about traceroute for Boardwatch Magazine.
When you get your trace, send it to me, along with any other relevant information, like what brand of modem you're using, what capacity of modem (14.4/28.8/33k/64k ISDN, etc.), whether it is internal or external, what speed serial port (if applicable), who your Internet Service Provider is, etc.
I'll collect results and see if any interesting patterns emerge. If any particular brands of modems and/or ISPs turn out to have good latency, I'll report that.
To start things off, here's my trace:

Name:  Stuart Cheshire
Modem: No modem (Quadra 700 built-in Ethernet)
ISP:   BBN (Bolt, Beranek and Newman)

Hop      Min    Avg    Max    IP              Name
 1  3/3  0.003  0.003  0.004  36.186.0.1      jenkins-gateway.stanford.edu
    2  3/3  0.003  0.006  0.013  171.64.1.161    core-gateway.stanford.edu
 3  3/3  0.004  0.004  0.004  171.64.1.34     sunet-gateway.stanford.edu
    4  3/3  0.003  0.003  0.004  198.31.10.3     su-pr1.bbnplanet.net
 5  3/3  0.004  0.004  0.005  4.0.1.89        paloalto-br1.bbnplanet.net
    6  2/3  0.006  0.006  0.007  4.0.1.62        oakland-br1.bbnplanet.net
 7  3/3  0.036  0.036  0.037  4.0.1.134       denver-br1.bbnplanet.net
    8  3/3  0.036  0.160  0.406  4.0.1.190       denver-br2.bbnplanet.net
 9  3/3  0.056  0.058  0.059  4.0.1.130       chicago1-br1.bbnplanet.net
   10  3/3  0.056  0.058  0.059  4.0.1.194       chicago1-br2.bbnplanet.net
11  3/3  0.076  0.077  0.078  4.0.1.126       boston1-br1.bbnplanet.net
   12  3/3  0.076  0.076  0.076  4.0.1.182       boston1-br2.bbnplanet.net
13  3/3  0.077  0.077  0.078  4.0.1.158       cambridge1-br2.bbnplanet.net
   14  3/3  0.080  0.081  0.083  199.94.205.1    cambridge1-cr1.bbnplanet.net
15  3/3  0.080  0.145  0.212  192.233.149.202 cambridge2-cr2.bbnplanet.net
   16  3/3  0.079  0.081  0.084  192.233.33.3    ihtfp.mit.edu
17  3/3  0.083  0.096  0.104  18.168.0.6      b24-rtr-fddi.mit.edu
  18  3/3  0.082  0.082  0.084  18.10.0.1       radole.lcs.mit.edu
 19  3/3  0.082  0.085  0.089  18.26.0.36      mintaka.lcs.mit.edu

You can see it took my Mac (Quadra 700 running Open Transport) 3ms to get to jenkins-gateway. This is not particularly fast. With a good Ethernet interface it would be less than 1ms. From there, it took 1ms to get to paloalto-br1 (near to Stanford) and another 2ms to get to oakland-br1 (across the bay from San Francisco).
From oakland-br1 to denver-br1 took 30ms, from denver-br1 to chicago1-br1 took 20ms, and from chicago1-br1 to boston1-br1 took another 20ms.
The last stretch from boston1-br1 to mintaka.lcs.mit.edu took another 6ms.
So to summarise where the time's going, there's 6ms spent at each end, and 70ms spent on the long-haul getting across the country. Remember those are round-trip times -- the one-way times are half as much.
Now, let's find out what the breakdown looks like when we try the same experiment with a modem. Send in your results! Hopefully we'll find at least one brand of modem that has good latency.
Note: October 1997. Now that I've got a decent collection of results, please only send me your results if they're a lot faster (or slower) than what's already on the list. Also, please send me results only for consumer technologies. If you're company has a T-1 Internet connection, or if you are a student in University houseing with a connection even faster than that, then it's not a great suprise to find that your connection has good latency. My goal here is to find what consumer technologies are available that offer good latency.

Are we there yet?

The good news is that since I first wrote this rant I've started to see a shift in awareness in the industry. Here are a couple of examples:
From Red Herring, June 1997, page 83, Luc Hatlestad wrote:

Matthew George is the vice president of techhnology at Engage... To Mr George, latency issues are more about modems than about network bandwidth. "Network latency in and of itself is not material to game playing; at least 70 to 90 percent of latency problems we see are due to the end points: the modems," he says.

From MacWeek, 12th May 1997, page 31, Larry Stevens wrote about the new 56k modems:

Greg Palen, director of multimedia at Galzay Marketing Group, a digital communications, prepress and marketing company in Kansas City, Kan., is one of those taking a wait-and-see attitude. "We can always use more bandwidth, but modem speed is not the primary issue at this point. The main issue is latency.

Some modem makers are finally starting to care about latency. One major modem manufacturer has contacted me, and we've been investigating where the time goes. It seems that there is room for improvement, but unfortunately modems will never be able to match ISDN. The problem is that over a telephone line, electrical signals get "blurred" out. In order to decode just one single bit, a 33.6kb/s modem needs to take not just a single reading of the voltage on the phone line at that instant, but that single reading plus another 79 like it, spaced 1/6000 of a second apart. A mathematical function of those 80 readings gives the actual result. This process is called "line equalization". Better line equalization allows higher data rates, but the more "taps" the equalizer has the more delay it adds. The V.34 standard also specifies particular scrambling and descrambling of the data, which also take time. According to this company, the theoretical best round-trip delay for a 14.4kb/s modem (no compression or error recovery) should be 40ms, and for a 33.6kb/s modem 64ms. The irony here is that as the capacity goes up, the best-case latency gets worse instead of better. For a small packet, it would be faster for your modem to send it at 9.6kb/s than at 33.6kb/s!
I don't know what the theoretical optimum for a 56kb/s modem is. The sample rate with these is 16000 times per second (62.5us between samples) but I don't know how many taps the equalizer has.

How to stop crontab writing to var/spool/mail/root file

My crontab is writing to /var/spool/mail/root file... Is there anyway I can stop this? The crontab is running every minute and it would fill the mail pretty quickly.
Solutions:

1)

Can i see the crontab rule ?? you must of put to sent email to root
If use the crontab rule like in below example, it will not send any email

Example:
*/5 17,18 * * 1-5 /sage.sh >/dev/null 2>&1

2)

At the top of your crontab file just add:
MAILTO=""
This will disable mail sending by crond.

Recommended Solution:

Try to set that in the script itself rather than the whole crontab file

WHY AIX MEMORY IS TYPICALLY AROUND 100%

Memory utilization on AIX systems typically runs around 100%. This is often a source of concern. However, high memory utilization in AIX does not imply the system is out of memory. By design, AIX leaves files it has accessed in memory. This significantly improves performance when AIX reaccesses these files because they can be reread directly from memory, not disk*. When AIX needs memory, it discards files using a "least used" algorithm. This generates no I/O and has almost no performance impact under normal circumstances.

Sustained paging activity is the best indication of low memory. Paging activity can be monitored using the "vmstat" command. If the "page-in" (PI) and "page-out" (PO) columns show non-zero values over "long" periods of time, then the system is short on memory. (All systems will show occasional paging, which is not a concern.)

Memory requirements for applications can be empirically determined using the AIX "rmss"command. The "rmss" command is a test tool that dynamically reduces usable memory. The onset of paging indicates an application's minimum memory requirement.

Finally, the "svmon" command can be used to list how much memory is used each process. The interpretation of the svmon output requires some expertise. See the AIX documentation for details. The vmo parameters also needs to be tuned.

Enable remote desktop remotely

1. Run Regedit

2. Select File | Connect Network Registry

3. Enter the name of the remote computer and select Check Names

4. Go to hklm\system\currentcontrolset\control\terminal server\FdenyTSConnection=1

5. Change the FdenyTSConnection to 0

Now you should be able to connect to the remote computer using Remote Desktop.

6. Reboot the remote machine by issuing the following command in Command Prompt:

shutdown -m \\hostname –r

7. Remote Desktop for the remote computer has been enabled and listening on default Remote Desktop port for any incoming Remote Desktop Connection. For security reason, you may want to consider changing the Remote Desktop listening port. There is also plenty of freeware utility that allows user to remotely enable Remote Desktop without modifying registry.

Using PSEXEC:

psexec \\computername reg add "hklm\system\currentcontrolset\control\terminal server" /f /v fDenyTSConnections /t REG_DWORD /d 0

How to Disable ping to Linux server?

How to Disable ping to Linux server?

To disable ping

echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all
To enable ping
echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all

HMC Commands for Documenting LPAR Configuration

For power systems p servers that are HMC managed, there are some tools you can use to document your LPAR profile configurations that in conjunction with other traditional data such as AIX snap files or prtconf output can be useful should a server need to be manually rebuilt. The tools that can make the tasks of documenting the LPARs' configuration include system plan and some queries of the profile data. Queries can be run from the HMC command line and the text output does provide a compact way of describing profile definitions which can also be used to recreate profiles from scripts if desired. The system plan output file can be used to restore profiles automatically should the need ever arise.
Following are some simple queries you can run to document the LPAR configurations.

If you are running HMC v7r3.4 or higher then the following command should allow you to create a system plan that details the profile definitions.

mksysplan -f -m --novios

The plan can be exported using the HMC GUI and used in case you have to manually rebuild your profiles or the plan might be used to automatically recreate the profiles should restoring profile data from other backup methods fail.

Another command you can use to gather your profile data in case there are issues with system plan follows.

lssyscfg -r prof -m

Some additional queries you can do if you have VIO servers that are active include following.

lshwres -m -r virtualio --rsubtype scsi

To get the name of your server to use in the above commands you can run

lssyscfg -r sys -F name

The data you get back from querying the profile data with the lssyscfg command and querying the virtual scsi data from lshwres can be saved along with any other server configuration data you might and would provide good documentation of your LPAR configurations should you ever need to manually rebuild a server the servers.

For more information regarding system plan or other HMC commands you can used the related topics links posted at the end of this tech-note.

How to change Max length of username in aix

On AIX 5.3 to change default length of username:

# chdev -l sys0 -a max_logname=9
sys0 changed

To check the current length of username:

# lsattr -El sys0 -a max_logname
max_logname 9 Maximum login name length at boot time True

# getconf LOGIN_NAME_MAX 21

Disable the Ctrl-Alt-Delete shutdown keys in Linux

On a production system it is recommended that you disable the [Ctrl]-[Alt]-[Delete] shutdown. It is configured using /etc/inittab (used by sysv-compatible init process) file. The inittab file describes which processes are started at bootup and during normal operation. You need to open this file and remove (or comment it) ctrlaltdel entry. 
Ctrlaltdel specifies the process that will be executed when init receives the SIGINT signal. SIGINT is the symbolic name for the signal thrown by computer programs when a user wishes to interrupt the process, for example reboot/shutdown system using [Ctrl]-[Alt]-[Del].). This means that someone on the system console has pressed the CTRL-ALT-DEL key combination. Typically one wants to execute some sort of shutdown either to get into single-user level or to reboot the machine.

Disable CTRL+ALT+Del keys

Open /etc/inittab file, enter:# vi /etc/inittab
Search for line that read as follows:ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
And remove the line or comment out the above line by putting a hash mark (#) in front of it:# ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
Save the file and exit to shell promot. Reboot system to take effect or type command:# init q

Hhow to disable first time password change in AIX

Clear the ADMCHG flag of the user account with "pwdadm -c ".

Example:

# pwdadm –c username

Difference between Ethernet Interface(en0) and Ethernet adapter (ent0)

AIX differentiates between hardware adapters, their interfaces and protocols standards associated.

To recognize interface card AIX uses three notations:

ent, en and et.

All are different and are described below and the sake of completeness I am using 0 at the end:

ent0:

The notation ent0 is used to specify the hardware adapter. It has nothing to do with the TCP/IP address. The parameters associated with ent0 can be seen as below:

# lsattr –El ent0

It will show parameters related to card.

It shows adapter_names, alt_addr, auto_recovery, backup_adapter, hash_mode, mode, netaddr, noloss_failover, num_retries, retry_time, use_alt_addr, use_jumbo_frame.

en0:

en0 represents the interface associated with hardware adapter ent0. The notation en0 is used for Standard Ethernet(inet). The TCP/IP address is associated with this interface.

The parameters associated with en0 can be seen as below:

#lsattr –El en0

It'll shows all the parameters related with the interface en0 of the adapter ent0.

It shows alias4, alias6, arp, authority, broadcast=1500, mtu, netaddr, netaddr6, netmask, prefixlen, remmtu, rfc1323, security, state, tcp_mssdflt, tcp_nodelay, tcp_recvspace, tcp_sendspace.

Rest everything is same except mtu(Maximum Transfer Unit) value. Which is 1500 as per the standard ethernet protocol.

et0:

et0 represents the interface associated with hardware adapter ent0. The notation et0 is used for IEEE 802.3 Ethernet(inet). If you are using standard ethernet protocol then it will not have TCP/IP address.

The parameters associated with et0 can be seen as below:

#lsattr –El et0

It'll shows all the parameters related with the interface et0 of the adapter ent0.

It shows alias4, alias6, arp, authority, broadcast, mtu=1492, netaddr, netaddr6, netmask, prefixlen, remmtu, rfc1323, security, state, tcp_mssdflt, tcp_nodelay, tcp_recvspace, tcp_sendspace.

Note here as well that the MTU shown will be 1492 as per IEEE 802.3 standard. Rest all parameters will be same. Also, netaddr, netmask fields will be empty fr et0.

Are these terms interchangable or is there a difference between them? I always get confused with these terms. What does en0 and ent0 mean and the difference between these?

AIX differentiates between a network adapter and network interface:

Network adapter Represents the layer-2 device, for example, the Ethernet adapter ent0 has a MAC address, such as 06:56:C0:00:20:03

Network interface Represents the layer-3 device, for example the Ethernet interface en0 has an IP address, such as 9.3.5.195

Typically, a network interface is attached to a network adapter, for example, an Ethernet interface en0 is attached to an Ethernet adapter ent0.

There are also some network interfaces in AIX that are not attached to a network adapter, for example, the loopback interface lo0 or a Virtual IP Address (VIPA) interface, such as vi0, if defined.

How to set a static routes in AIX

You can use the route command to set a static route. But this way you don't get it back after reboot. 
To make a route persistent you need to change inet0. First check which routes are already set: 

 # lsattr -El inet0 -a route
 route net,-hopcount,0,,0,192.168.1.1 Route True
 route net,-hopcount,255.255.255.128,,,,,192.168.3.155,192.168.2.1 Route True

These routes would be set with: 

 # chdev -l inet0 -a route=net,-hopcount,0,,0,192.168.1.1
 # chdev -l inet0 -a route=net,-hopcount,255.255.255.128,,,,,192.168.3.155,192.168.2.1

To remove these specific static routes: 

 # chdev -l inet0 -a delroute=net,-hopcount,0,,0,192.168.1.1
 # chdev -l inet0 -a delroute=net,-hopcount,255.255.255.128,,,,,192.168.3.128,192.168.2.1

In this route string 255.255.255.128 is the netmask, 192.168.3.128 the destination net, and 192.168.2.1 the gateway. 
For hostroutes the keyword net has to be replaced with host.

How can I directly read out the VGDA of a PV (hdisk)?

Information about VGx, LVx, filesystems, etc. are stored in the ODM. But these information are also written to the VGDA of the disks itself. You can read the information directly from the disk's VGDA with a command like this: 

 # lqueryvg -Atp hdisk100

You can use 

 # redefinevg -d hdisk100 myvg

to synchronize the ODM with the information of the VGDA. You can also synchronize the VGDA with the information stored in the ODM: 

 # synclvodm myvg

How can I unlock a SAN disk in AIX?

I got my LUN mapped to my system, but when I try to create my Volume Group with  mkvg -f vpath100   all I get is an I/O error. What can I do? 
Probably there is still a SAN lock on the disk. Try to unlock it with: 

 # lquerypv -ch /dev/vpath100

and retry to create your Volume Group.

print commands in Linux

To see a list of available printers:

# lpstat -p -d

To print the file to printer "MyPrinter"

# lpr -P MyPrinter filename

To view (query) the print queue, use the lpq or lpstat command. When entered without arguments, it displays the contents of the default print queue.

# lpq

# lpstat

To list the default printer

# lpstat -d

To know the status of default printer

# lpstat -p

If the printer status is disabled, you need to enable the printer using "enable" command as follows.

# /usr/bin/enable

To disable the printer

# /usr/bin/disable -c

To reset the print queue, you have to disable and reenable as follows

# /usr/bin/disable -c

# /usr/bin/enable

To remove the print job

# lprm

Why is there no free RAM in Linux? or Why Memory usage is 100% in Linux?

Traditional Unix tools like 'top' often report a surprisingly small amount of free memory after a system has been running for a while. For instance, after about 3 hours of uptime, the machine I'm writing this on reports under 60 MB of free memory, even though I have 512 MB of RAM on the system. Where does it all go? 

The biggest place it's being used is in the disk cache, which is currently over 290 MB. This is reported by top as "cached". Cached memory is essentially free, in that it can be replaced quickly if a running (or newly starting) program needs the memory. 

The reason Linux uses so much memory for disk cache is because the RAM is wasted if it isn't used. Keeping the cache means that if something needs the same data again, there's a good chance it will still be in the cache in memory. Fetching the information from there is around 1,000 times quicker than getting it from the hard disk. If it's not found in the cache, the hard disk needs to be read anyway, but in that case nothing has been lost in time. 

To see a better estimation of how much memory is really free for applications to use, run the command:

Code:

free –m

The -m option stands for megabytes, and the output will look something like this:

Code:

total       used       free     shared    buffers     cached Mem:           503        451         52          0         14        293 -/+ buffers/cache:        143        360 Swap:         1027          0       1027

The -/+ buffers/cache line shows how much memory is used and free from the perspective of the applications. Generally speaking, if little swap is being used, memory usage isn't impacting performance at all. 

Notice that I have 512 MB of memory in my machine, but only 503 is listed as available by free. This is mainly because the kernel can't be swapped out, so the memory it occupies could never be freed. There may also be regions of memory reserved for/by the hardware for other purposes as well, depending on the system architecture. 

The difference among VIRT, RES, and SHR in top output 
VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card's RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment. 

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library. 

SHR indicates how much of the VIRT size is actually sharable (memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES. 

The difference between buffers and cache 
Buffers are associated with a specific block device, and cover caching of filesystem metadata as well as tracking in-flight pages. The cache only contains parked file data. That is, the buffers remember what's in directories, what file permissions are, and keep track of what memory is being written from or read to for a particular block device. The cache only contains the contents of the files themselves. 

Corrections and additions to this section welcome; I've done a bit of guesswork based on tracing how /proc/meminfo is produced to arrive at these conclusions. 

Swappiness (2.6 kernels) 
Since 2.6, there has been a way to tune how much Linux favors swapping out to disk compared to shrinking the caches when memory gets full. 

When an application needs memory and all the RAM is fully occupied, the kernel has two ways to free some memory at its disposal: it can either reduce the disk cache in the RAM by eliminating the oldest data or it may swap some less used portions (pages) of programs out to the swap partition on disk. It is not easy to predict which method would be more efficient. 
The kernel makes a choice by roughly guessing the effectiveness of the two methods at a given instant, based on the recent history of activity. 

Before the 2.6 kernels, the user had no possible means to influence the calculations and there could happen situations where the kernel often made the wrong choice, leading to thrashing and slow performance. The addition of swappiness in 2.6 changes this. 

Swappiness takes a value between 0 and 100 to change the balance between swapping applications and freeing cache. At 100, the kernel will always prefer to find inactive pages and swap them out; in other cases, whether a swapout occurs depends on how much application memory is in use and how poorly the cache is doing at finding and releasing inactive items. 

The default swappiness is 60. A value of 0 gives something close to the old behavior where applications that wanted memory could shrink the cache to a tiny fraction of RAM. For laptops which would prefer to let their disk spin down, a value of 20 or less is recommended. 

As a sysctl, the swappiness can be set at runtime with either of the following commands:

Code:

# sysctl -w vm.swappiness=30 # echo 30 >/proc/sys/vm/swappiness

The default when Gentoo boots can also be set in /etc/sysctl.conf:

Code:

# Control how much the kernel should favor swapping out applications (0-100) vm.swappiness = 30

Some patchsets allow the kernel to auto-tune the swappiness level as it sees fit; they may not keep a user-set value.

Disable ipv6 on RHEL 4 and 5

Edit /etc/sysconfig/network and change
NETWORKING_IPV6=yes to
NETWORKING_IPV6=no
Edit /etc/modprobe.conf and add these lines (if they're not in it):
alias net-pf-10 off
alias ipv6 off
Stop the ipv6tables service by typing:
service ip6tables stop
Disable the ipv6tables service by typing:
chkconfig ip6tables off
After these changes, IPv6 will be disabled after the next reboot of your system.

Linux Kernel panic reboot

By default after a kernel panic Linux just waits there for a sysadmin to hit the restart or powercycle button.  This is because of the value set on "kernel.panic" parameter.

[root@linux23 ~]# cat /proc/sys/kernel/panic
0
[root@linux23 ~]# sysctl -a | grep kernel.panic
kernel.panic = 0
[root@linux23 ~]#

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer N greater than zero to the paramter "kernel.panic", where "N" is the number of seconds to wait before a automatic reboot.  For example , if you set N = 10 , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and set it.

[root@linux23 ~]# echo "10" > /proc/sys/kernel/panic
0
[root@linux23 ~]# grep kernel.panic /etc/sysctl.conf
kernel.panic = 10
[root@linux23 ~]#

A Windows user’s guide to Linux

Don’t know your DEB from your RPM? We offer a short guide to Linux for the Windows user.

If you pay any attention to IT at all you’ll have heard of Linux.

[Linux refers to the family of Unix-like computer operating systems using the Linux kernel. Linux can be installed on a wide variety of computer hardware, ranging from mobile phones, tablet computers, routers and video game consoles, to desktop computers, mainframes and supercomputers. Linux is a leading server operating system, and runs the 10 fastest supercomputers in the world.

The development of Linux is one of the most prominent examples of free and open source software collaboration; typically all the underlying source code can be used, freely modified, and redistributed, both commercially and non-commercially, by anyone under licenses such as the GNU General Public License. Typically Linux is packaged in a format known as a Linux distribution for desktop and server use. Some popular mainstream Linux distributions include Debian (and its derivatives such as Ubuntu), Fedora and openSUSE. Linux distributions include the Linux kernel, supporting utilities and libraries and usually a large amount of application software to fulfill the distribution's intended use. ]

You’ll probably also have heard from the some that Linux is complicated, ugly and incomplete. Which couldn’t be further from the truth. In reality, Linux is now a full-fledged member of the operating system club and can do just about anything that any other OS can do and, in many cases, do a lot more than many other OSes can do.

So, if you’ve ever considered giving Linux a try then now is the time. Here are a few tips to get you smoothly onto the Linux road.

Distributions

This is your first step. Linux is not homogeneous like Windows or OS X. Linux comes in a range of different versions, called “distributions”. The majority of the underlying code in each of these distributions is the same with most of the differences being in the interface and some of the management tools. Choosing the right distribution can be tricky, especially as there are literally hundreds of versions of Linux available. Fortunately most of those you can forget about, for now. What you need is an easy to use version of Linux, which leaves you with a short-list of Ubuntu, Fedora, OpenSuse, Mandriva and Linux Mint. Picking one of these will make you life easier as they are all easy to install and pretty simple to maintain.

Desktop Environment

Again, unlike Windows or Mac OS X,

[ Microsoft Windows is a series of operating systems produced by Microsoft. Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces (GUIs). Microsoft Windows came to dominate the world's personal computer market, overtaking Mac OS, which had been introduced in 1984. As of October 2009, Windows had approximately 90% of the market share of the client operating systems for usage on the Internet. The most recent client version of Windows is Windows 7; the most recent server version is Windows Server 2008 R2; the most recent mobile version is Windows Phone 7.]

[Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, Mac OS X has been included with all new Macintosh computer systems. It is the successor to Mac OS 9, released in 1999, the final release of the "classic" Mac OS, which had been Apple's primary operating system since 1984.

Mac OS X, whose X is the Roman numeral for 10 and is a prominent part of its brand identity, is a Unix-based graphical operating system,[8] built on technologies developed at NeXT between the second half of the 1980s and Apple's purchase of the company in late 1996. From its sixth release, Mac OS X v10.5 "Leopard" and onward, every release of Mac OS X gained UNIX 03 certification while running on Intel processors.]

Linux is not limited to just one desktop interface. There are dozens of different interfaces available, each with their own unique capabilities and drawbacks. Much like choosing a distribution, most of them can be ignored, for now. Whichever distribution you choose to install will include its own default desktop interface. In most cases it will be either KDE or Gnome. In Ubuntu’s case it will be its own Unity. The beauty in this process is that if you don’t like the interface you have then you can easily swap to another one.

LiveCDs

This is worth mentioning because one of the great contributions of Linux to the world is the idea of LiveCDs. Most versions of Linux have a LiveCD available which can be downloaded and used to test the OS before installing. Some even run from a USB drive which makes it even easier. With a LiveCD the entire OS is run from the CD without touching your hard drive and is a great way to test if you like the OS before installing it. In most cases the LiveCD doubles as the installer as well.

RPM, DEB

Linux applications are shipped in various formats. What’s important to remember here is that not all packaged applications will install on every version of Linux. Sounds complicated but it’s not really that difficult, especially as now most Linux versions have their own repositories which contain all the applications in the right format for that distribution. You may occasionally come across applications on the internet that you want to install and then you’ll have to choose the right version to download. Ubuntu, for example, uses .deb files which is among the most popular of the formats. Fedora on the other hand uses RPM. Fortunately all major distributions include an application manager that worries about all of this for you.

Windows applications

Perhaps the biggest challenge when moving to Linux is getting used to living without your Windows applications. Fortunately there are multiple ways around this. The easiest way is to install Linux alongside your Windows installation. This dual-boot approach means you can switch between Windows and Linux when you need to. The downside is that you have to log-off one OS to use the other. The other approach is to install something like VirtualBox which is a fantastic piece of virtualisation software. Using VBox you can install a copy of Windows, or Linux, on top of your main OS. You can then simply open your VBox window when you want to use your second OS without needing to log off the first. The downside is that it’s not entirely painless to get the two working in harmony but it’s also not impossible. The third option is to investigate Wine. Wine is a Windows compatibility layer that makes it possible to run many Windows applications directly on Linux. Not all applications are supported but most major ones are.

Alternatives

Although there are many ways to mimic Windows on Linux it does sort of defeat the point of trying Linux in the first place. Especially as there are many fantastic native Linux applications already available. The Linux Alternative project is just one of many lists of applications for Linux that can be use to replace Windows ones. Applications such as OpenOffice.org, Gimp and Inkscape are not only suitable alternatives to many Windows applications but in some cases are even better that the originals.

With Linux improving at a rapid rate it’s now easier than ever to use, so there is no better time to give Linux a try.

10 Secure Linux Distributions You Need Know About

With security constantly in the news lately, you can't help but feel ill at ease and vulnerable -- vulnerable to teams of hackers whose only motivations are to expose and attack their victims. Perhaps you think you've done due diligence by keeping your patches updated, installing security fixes, and maintaining a corporate firewall. Those methods are effective about 50 percent of the time. For the other 50 percent, you need to do more. You need penetration testing, security audits, intrusion prevention and intrusion detection, and you need to plug security holes that only hackers know about by using the tools they use to compromise your systems.

Security is expensive no matter how you slice it but it doesn't have to be a death knell for your business. This list of 10, in no particular order, security-enhanced Linux distributions can give you peace of mind by beating hackers on their turf.

1. Astaro Security Appliance - Formerly known as Astaro Security Linux, the Astaro Security Appliances come in three flavors: Hardware, software and virtual. In the virtual appliance category, Astaro offers appliances built specifically for network security, mail security, Web security and Web application security. Its virtual appliances hold the VMware Ready certfication.

2. The network security virtual appliance, for example, includes a configurable firewall, intrusion protection, DoS attack protection, NAT tools, VPN, IPSec Remote Access, LDAP authentication integration, and bandwidth control. Sophos recently acquired Astaro to create one of the world's leading security companies. Sophos boasts over 100 million worldwide business users in more than 150 countries.

3. BackTrack Linux - BackTrack Linux is the highest rated and most acclaimed Linux security distribution. BackTrack is not a business desktop or server system but is a security-oriented system built solely for the purpose of network and computer penetration testing. BackTrack can be run from a bootable DVD, a thumbdrive or a hard disk. BackTrack Linux is a specialized distribution created to assist security professionals in performing security audits on target networks. But, with BackTrack Linux, you don't have to be a seasoned security professional to use it -- even security newcomers will find BackTrack easy to setup, use, and update. You can download BackTrack as an ISO image or as a VMware virtual machine.

4. IPFire - IPFire is a firewall distribution that is small, highly secure and easy to use. IPFire developers and maintainers are experienced security professionals. Like BackTrack, IPFire enjoys widespread adoption and an active user community. IPFire has its own special packaging system called Pakfire. The Pakfire system is unique to IPFire and delivers all updates and new packages via encrypted transfer and digital signatures. IPFire also features easy addon installation. Addons include Samba, NFS, mail services, anti-virus, multimedia applications, VoIP applications, intrusion detection, network tools, security tools, backup tools and dozens of other applications.

5. Lightweight Portable Security - The Lightweight Portable Security (LPS) distribution boots a thin Linux system from a CD or USB flash drive. It isn't meant to be run from a local hard disk. The intended use for LPS-Public version is to allow safe, public, general-purpose Web browsing and LPS-Remote Access is only for accessing internal networks. Since the system allows no traces of activity or browsing history, administrators must pay strict attention to limit where LPS users may browse by means of filtering through a proxy server. Users should reboot between sessions to clear any potential malware or browser hijacking that took place during previous sessions. LPS provides secure browsing during banking transactions or other security-sensitive sessions.

6. Live Hacking DVD - This live DVD distribution is exactly what it sounds like: An ethical hacker's playground (workbench). There is also a CD version (Live Hacking CD). The DVD comes with a fully graphical desktop interface (GNOME) and the CD version is command line only. The CD version is as powerful as its graphical counterpart because most of the hacker tools are command line. The Live Hacking system requirements are minimal. You can use an old Pentium III or IV class system and as little as 512 MB RAM, although the developers recommend 1 GB RAM. To download and use the Live Hacking distribution, you must accept the Terms and Conditions which state that the tools are for ethical hacking only.

7. EnGarde Secure Linux - EnGarde Linux is a Linux server distribution that is secure and perfect for use as an Internet server. It features intrusion detection, simple administration, secure network services, built-in alerts, Web services, DNS services, firewall, mail services and access to the Guardian Digital Support Network (GDSN). The GDSN provides free access to all system and security updates. EnGarde Regularly scheduled updates the first Tuesday of every month. Try before you buy with a downloadable live CD version of EnGarde.

8. NetSecL - NetSecL is an OpenSUSE-based distribution that features GrSecurity, chroot hardening, auditing, and includes penetration testing software. It is versatile enough to be used as a desktop, server, or ethical hacking system. It is a live DVD but you can also install it to a hard disk. GrSecurity is an independent suite of security enhancements used by ISPs, hosting companies, and projects like NetSecL. Other tools included with NetSecL are Amap, Ettercap, Hydra, Kismet, Nessus, Nmap, Metasploit, and PADS.

9. SmoothWall Express - The SmoothWall Open Source project began in 2000 and continues to be an excellent business firewall solution. SmoothWall Express (SWX) is a security-hardened GNU/Linux operating system with a simple to use web interface. The primary goals of the SWX project are to create and maintain a simple firewall system, support a variety of hardware, work with multiple connection methods, run on inexpensive and commodity hardware, develop a supportive user community and support the project via the commercial venture SmoothWall Limited. SmoothWall Limited manufactures several different SmoothWall hardware security appliances suitable for networks of all sizes.

10. Openwall GNU/Linux - Openwall GNU/Linux (OWL) is a small, security-enhanced distribution suitable for virtual appliances, hardware appliances, and physical servers. OWL is binary compatible with Red Hat Enterprise Linux. OWL is also a distribution used by many security professionals for security penetration testing and password cracking. Openwall also develops other security products such as the famous John the Ripper password crack utility, phpass, passwdqc, and tcb.

11. Vyatta - Vyatta is a commercial security appliance vendor delivering appliances for every network class including cloud architectures. Included in Vyatta's product line-up is the Vyatta virtual network appliance. Vyatta virtual appliances work in VMware, Xen, XenServer, and KVM environments. The virtual security appliance includes a stateful firewall, IPSec and SSL-based VPN, intrusion detection, filtering, dynamic routing and router-based services such as NAT, DHCP and is IPv6-ready.

Data Center Designing: Network Connectivity Standards

Network Connectivity Standards

IF your organization looking for a solution to host it’s entire application environment in an external vendor’s data center, your organization must be considering lot many factors to define the standards of the data center. Below post describes some of the Network Connectivity standards used for development, implementation and management of a Data Center managed by a vendor.

This post assumes that you should have an understanding of the concepts and terminology associated with data communications networks, protocols, and equipment used in this Schedule.

Network Transport Options – Connection

The Transaction Link between provides the end users access to the DC Environment managed by vendor.

The following Standards are designed to facilitate the security, reliability, and scalability of the Network Connectivity, via the Transaction Link, to the Environment at the Data Center.

��� IP Addressing for the Data Center

To avoid IP address conflicts, Vendor will only route network traffic to globally unique public IP addresses registered through one of the regional Internet registry organizations. Vendor will not route traffic to private IP addresses, including addresses in the following IP address ranges:

• Class A: 10.0.0.0/8
• Class B: 172.16.0.0/12
• Class C: 192.168.0.0/16

Network Address Translation and Port Address Translation

If Organization uses private addresses, Network Address Translation (NAT) and Port Address Translation (PAT) can be used to map Organization’s private addresses to public addresses. Organization is responsible for providing the public IP addresses and for performing the NAT/PAT.

NAT and PAT Policies

One-to-one static NAT must be used for all network devices, including, without limitation, Organization’s printers or print servers to which Vendor will initiate a connection. PAT is allowed for any network device that initiates the connection with Vendor , such as user workstations connecting to the Vendor Programs.

Ports

During network initialization, Vendor will provide a list of ports that must be open on Organization’s network to establish Network Connectivity to the Environment at the Data Center. The list of required ports is specific to the type of Network Connectivity selected by Organization and will include source and destination addresses. Access through these ports is required before Vendor can begin installing or troubleshooting Network Connectivity.

VPN Requirements The following table lists the ports that must be enabled for Vendor to manage the Vendor VPN.

Application Name	Port	Protocol
Ping	ICMP	IP
Traceroute	ICMP	IP
SSH & SFTP (if applicable)	22	TCP
HTTPS	443	TCP
ISAKMP	500	UDP
IPSEC	50	IP
NTP	123	UDP

Additional ports may be required. During network initialization, the Network Connectivity Form will define detailed source and destination port requirements

Network Design Considerations

Organization should consider certain factors described in this Section when selecting a Network Connectivity design for the Transaction Link between Organization and the Data Center.

Bandwidth

Bandwidth refers to the amount of traffic to be carried through a network and must be considered by Organization when designing Network Connectivity. The amount of bandwidth required depends on the particular applications being used, the number of users concurrently accessing the Environment, and the nature of the transactions being processed. Network bandwidth must be monitored and adjusted as application usage grows and changes.

Bandwidth Considerations

Vendor recommends the bandwidth sizes listed in the following table as a starting point for network sizing.

Sample Application	Bandwidth For Each User Concurrently Accessing the Environment
Self-service applications	4-6 kbps
Forms applications	10-12 kbps
Internal Portal	2-6 kbps
Files access	10-12 kbps
Siebel application	10-12 kbps
PeopleSoft application	10-12 kbps

Organization must design its network to reduce overall network latency wherever possible. The network design should remove as much distance as possible between the End User’s locations and the application server. This can be accomplished with a network design that does not add unnecessary distance or hops between endpoints and ensuring that the network provider has the capacity available on the most efficient network cable system between the two end points.

Properly configured and sized network routers usually induce very little delay. However, the delay through routers can become a major source of overall network latency when the network connections are congested or routers are improperly configured. Organization should ensure that all network devices and links are properly sized and configured and are running optimally.

Error Rate

Organization must consider the error rate when designing Network Connectivity. All network links are subject to transmission errors such as dropped packets. A high error rate can lead to increased latency and poor network performance. Each segment in a network can experience independent errors that add to the total error rate for the entire link. To ensure that a network operates at peak performance, Organization should ensure that the end-to-end error rate does not exceed 0.01%.

Connection Types

Organization must use a Vendor -standard VPN for Network Connectivity. Vendor supplies, configures, and manages an IPSec compliant VPN device that is installed on Organization’s network. This VPN is used to establish secure Network Connectivity to Vendor over the Transaction Link.

There are various standard configurations available for the Vendor provided VPN device that may accommodate Organization’s network and security policies.

ISP Circuit Types

The Vendor standard VPN connection uses Organization’s ISP circuit. Organization should choose the type of circuit that best meets its use of the Computer and Administration Services.

Dedicated Internet Circuit

A dedicated circuit is Organization’s Internet circuit that is only used for application connection activities. A dedicated circuit can help prevent problems related to over use by isolating traffic to Vendor from Organization’s other Internet traffic. A dedicated circuit can help Organization achieve stable and predictable Network Connectivity. This option is the recommended minimum for a Transaction Link.

Shared Circuit

An Internet circuit that is used both for Network Connectivity to Vendor and Organization’s other Internet activities is called a shared circuit. If Organization selects this circuit type, Organization must ensure that the existing circuit has enough unused capacity to support Vendor traffic.

A shared circuit should only be considered for a Transaction Link if the circuit performance is highly stable and actively monitored for performance and capacity.

VPN Configuration

Vendor provides, configures, and manages a VPN device for an Vendor standard VPN connection over the Transaction Link. Vendor configures the VPN device based on Organization’s network topology and Vendor policies. This section describes the various VPN configurations and Vendor requirements.

The Vendor -provided VPN device has two interfaces (one external and one internal). Vendor generally uses both interfaces (dual-arm mode), but can use a configuration with only one interface (single-arm mode) as described below.

External Interface

The following guidelines apply to external interfaces:

• The external interface must be connected to a switch between Organization’s border router and firewall.
• The external interface may be connected to a firewall DMZ interface.
• The external interface must not be directly connected to the Internet. The external untrusted interface should be connected to the Internet behind Organization’s border router to enable Organization to apply Access Control Lists (ACLs) to secure Organization’s Environment from unsolicited traffic.
• The external interface must have a globally unique public IP address. Private addressing is not permitted on this interface.

Internal Interface

The following guidelines apply to internal interfaces:

• The internal interface must be connected to a firewall DMZ interface.
• The internal interface must not be connected to the same subnet as the external interface.
• The internal interface must not be connected to Organization’s Internet.
• The internal interface must have a public IP address.
• The internal interface must not have a private IP address, unless Organization configures its firewall to use the internal interface as a transit link.

Dual-Arm Configuration

Vendor recommends a dual-arm configuration if Organization has or requires a DMZ port on its firewall. A standard dual-arm configuration is shown in the following diagram.

Description

A dual arm configuration uses both of the VPN device interfaces. The external (or untrusted) interface handles the encrypted traffic between Vendor and Organization over the Transaction Link. The internal (or trusted) interface is connected to a secure portion of Organization’s network and receives and transmits the unencrypted traffic.

The Vendor VPN is logically located between Organization’s firewall and Internet border router. A second connection is placed within a DMZ on Organization’s firewall.

Routing Path

When using a routing path for a dual arm configuration, layer 3 connectivity to Organization’s Environment is established by directing routes towards Organization’s firewall. This can be accomplished by either using default routing or by using routing protocols for packet redirection to the firewall. A static route that redirects traffic to the Vendor VPN is placed on Organization’s firewall.

Advantages

The advantages of using a routing path for a dual-arm configuration are:

• Minimal routing changes are required.
• All traffic is auditable by Organization using the firewall.
• The firewall can be used for access control.

Disadvantages

The disadvantages of using a routing path for a dual-arm configuration are:

A DMZ on Organization’s firewall is required.

Single-Arm Configuration

A single-arm configuration may be used if Organization does not have a DMZ interface on its firewall or an available globally unique public IP address on a separate subnet. A standard single-arm configuration is shown in the following diagram.

��� Description

A single-arm configuration uses only the external interface for the VPN tunnel. The Vendor VPN is logically located between Organization’s firewall and Internet border router and both the encrypted and unencrypted traffic over the Transaction Link are handled by the single interface.

���Routing Path

When using a routing path for a single-arm configuration, layer 3 connectivity to Organization’s Environment is established by directing routes towards Organization’s firewall. This can be accomplished by either utilizing default routing or by using routing protocols for packet redirection to the firewall. A static route that will redirect traffic to the Vendor VPN is placed on Organization’s firewall.

���Advantages

The advantages of using a routing path for a single-arm configuration are:

• Minimal routing changes are required.
• One IP address is required for the VPN. (Note: Additional IP addresses are required to set up printing since printers require a one-to-one static NAT.)
• One switch port is required.
• All traffic is auditable by Organization using the firewall.
• The firewall can be utilized for access control.

Disadvantages

The disadvantages of using a routing path for a single-arm configuration are:

• Data flow between the firewall and the VPN is not encrypted. This weakness can be mitigated by confirming that no hosts are capable of reading traffic, the VPN equipment is in a secured location, and proper access control lists (ACLs) are created on Organization’s border router.
• The full throughput of VPN cannot be used.

Vendor High Availability VPN

If required, Organization may request a high availability (HA) VPN configuration involving two VPNs. Additional fees applies. Vendor ’recommended high availability configuration provides Organization with two Vendor -configured VPN devices running in dual arm mode. A standard high availability installation is shown in the following diagram.

Description

A high availability dual arm VPN configuration uses both of the VPN device interfaces. The external (or untrusted) interface handles the encrypted traffic between Vendor and Organization over the Transaction Link. The internal (or trusted) interface is connected to a secure portion of Organization’s network and receives and transmits the unencrypted traffic

The Vendor VPN is logically located between Organization’s firewall and Internet border router. A second connection is placed within a DMZ on Organization’s firewall.

Routing Path

When using a routing path for the high availability VPN configuration, layer 3 connectivity to Organization’s Environment is established by directing Vendor routes towards Organization’s firewall. This can be accomplished by either using default routing or by using routing protocols for packet redirection to the firewall. A static route that redirects traffic to the Vendor VPN is placed on Organization’s firewall.

Advantages

The advantages of using a routing path for a high availability dual arm VPN configuration are:

• • Minimal routing changes are required.
  • All traffic is auditable by Organization using the firewall.
  • The firewall can be utilized for access control.

Disadvantage

The disadvantages of using a routing path for a high availability dual arm VPN configuration are:

A DMZ on the Organization’s firewall is required.

Required Ports and Protocols

Vendor requires access to the VPN device to establish and maintain Network Connectivity through the VPN tunnel. The following table lists the ports that are required to be open between Vendor and the VPN device. During VPN installation, Vendor will provide a more detailed list that contains source and destination addresses.

The following IP ports and protocols are required for establishing an IPSec tunnel, managing the Vendor -provided network equipment, and monitoring the network link to the Organization-side VPN device.Application Name	Port #	Protocol	Comments
SSH	22	TCP	Network Management for Netscreen VPN
HTTPS	443	TCP	Network Management for Netscreen VPN
ICMP	Ping	IP	Monitoring and Diagnostics
ICMP	Traceroute	IP	Monitoring and Diagnostics
ISAKMP	500	UDP	VPN Tunnel
IPSEC	50	IP	VPN Tunnel
NTP	123	UDP	Network Time Protocol for VPN device