Thursday, April 24, 2014

Puppet vs. Chef vs. Ansible vs. Salt,3

The leading configuration management and orchestration tools take different paths to server automation

Puppet or Chef? Ansible or Salt?
Whereas Puppet and Chef will appeal to developers and development-oriented shops, Salt and Ansible are much more attuned to the needs of system administrators. Ansible's simple interface and usability fit right into the sys admin mindset, and in a shop with lots of Linux and Unix systems, Ansible is quick and easy to run right out of the gate.
Salt is the sleekest and most robust of the four, and like Ansible it will resonate with sys admins. Highly scalable and quite capable, Salt is hamstrung only by the Web UI.
Puppet is the most mature and probably the most approachable of the four from a usability standpoint, though a solid knowledge of Ruby is highly recommended. Puppet is not as streamlined as Ansible or Salt, and its configuration can get Byzantine at times. Puppet is the safest bet for heterogeneous environments, but you may find Ansible or Salt to be a better fit in a larger or more homogenous infrastructure.
Chef has a stable and well-designed layout, and while it's not quite up to the level of Puppet in terms of raw features, it's a very capable solution. Chef may pose the most difficult learning curve to administrators who lack significant programming experience, but it could be the most natural fit for development-minded admins and development shops.
Orchestrators at a glance
 Puppet 3.0Chef 11.4Ansible 1.3Salt 0.17
  • Modules can be written in Ruby or a simpler, Puppet-specific subset of Ruby
  • Push commands let you trigger modifications immediately
  • Web UI handles reporting, inventorying, and real-time node management
  • Detailed and in-depth reporting on agent runs and node configurations
  • Cookbooks and recipes can leverage the full power of Ruby
  • Centralized JSON-based "data bags" allow scripts to populate variables during runtime
  • Web UI lets you search and inventory nodes, view node activity, and assign Cookbooks, roles, and nodes
  • Modules can be written in nearly any language
  • No agent required on managed clients
  • Web UI lets you configure users, teams, and inventories, and apply Playbooks to inventories
  • Extremely simple to set up and get running
  • State files can be simple YAML configuration templates or complex Python or PyDSL scripts
  • Can communicate with clients through SSH or through a locally installed agent
  • Web UI offers views of running jobs, minion status, and event logs, and lets you execute commands on clients
  • Extremely scalable
  • Requires learning Puppet DSL or Ruby
  • Installation process lacking in error checking and error reporting
  • Requires knowledge of Ruby programming
  • Currently lacks functional push commands
  • Documentation is sometimes vague
  • Lacks support for Windows clients
  • Web UI doesn't tie into an existing Ansible deployment automatically; inventories must be imported
  • Web UI is not as mature or complete as competitors
  • Lacks deep reporting capabilities
PricingFree open source version; Puppet Enterprise costs $100 per machine per yearFree open source version; Enterprise Chef free for 5 machines, $120 per month for 20 machines, $300 per month for 50 machines, $600 per month for 100 machines, and so onFree open source version; AWX free for 10 machines, then $100 or $250 per machine per year depending on supportFree open source version; SaltStack Enterprise costs $150 per node per year, with volume discounts and site licenses available

12 clever cloud tools for devs and ops

Look to these automation, orchestration, and configuration management tools to keep your server farms running smoothly..

Clever cloud tools
Was it only a few years ago when deploying a single server was a big occasion? First we needed to unpack it, mount it, test it, and install the software. Then we could put it to work.
Today, we're asked to spin up 200 servers before lunch, configure them, crunch some numbers, and return them before the 1 p.m. staff meeting. Time is money, and servers are bought by the hour now.
That explains the explosion of interest in tools that let us manage hundreds of machines with a few quick clicks. Here's a list of some of the most intriguing new tools for making our server farms jump to attention in less time than it took to slice open the tape on a server box. 

More than a dozen years ago, developers at U.C. Berkeley looked at the clusters of computers throughout the campus and realized they needed a way to keep track of them -- so they built and rebuilt Ganglia. Today, Ganglia is one of the most comprehensive sets of tools for monitoring a diverse cluster of machines. It collects statistics with a lightweight protocol and graphs them so that you can watch performance over time.
Distributed with the BSD license.

There are hundreds of graphing libraries and visualization tools, but they're usually aimed at a fairly static collection of data. Graphite is designed to support real-time data from dozens of streams. The data goes in one side of the application, where it's stored in a real-time, scalable database optimized to handle the numerical streams of information. The system is designed to keep up with fast streams of data from multiple sources. There's even an elaborate caching system to make sure the graphs are up to date.
The back end is written in pure Python, and the Web app on top of Django. The graphing is done with the Cairo libraries. Orbitz built the software to handle internal monitoring, then released it with the Apache...

Tools like Graphite and Ganglia may make it easier to monitor the state of the system, but they require someone to sit and watch the screen as the lines dance up and down. Tattle is designed to watch the graphs for you, then send off alerts if a line crosses over into some place it's not supposed to be. These alerts usually go out as text messages and emails, but you can configure them with a plug-in if you want special behavior like a warning siren and flashing red light connected to an Arduino controller.
Written in PHP for both Graphite and Ganglia.

There are only three things that are certain for sys admins: death, taxes, and logs full of events. The earth goes 'round the sun, and the log files fill up with entries about visits to the websites and all of the random housekeeping around the stack.
Logstash is an open source package designed to put a pretty face on the endless streams of data. It parses the entries, segments them, and builds graphs so that you can drill down looking for items that need investigating, all through the Web interface. If you need to customize the system, you can write new filter plug-ins that fit into the flow of data.
Distributed under the Apache 2.0 license.

Logstash may build elegant, rolling log files, but these can only be searched with basic queries. Kibana is glue between Logstash and Elasticsearch. It dumps Logstash log information into Elasticsearch so that you can write Lucene queries to find what you want. Not only can you use Boolean searches and wild cards to pick through the log files, but Kibana goes one step further by running these queries periodically and packaging the results into a dashboard that's also published with RSS and some other standards.
Available under the MIT license.

Everyone loves the idea of the cloud, but no one wants to let go of the security of owning the servers. CloudVelocitywants to ease companies into the cloud (specifically Amazon Web Services for now) by offering a hybrid that mixes the cloud machines with the private boxes. Moving data and operations between the two is meant to be as easy as possible, perhaps as easy as dragging some icons across a Web page. The company promises to let you test your operations with separate test rigs built by cloning existing machines. Now if only it would let us clone ourselves so we can get everything done that the management wants.

Ravello Systems
Another tool for easy cloud migration, Ravello is a layer that sits on top of the major clouds (Amazon, HP, Rackspace) and helps development teams deploy multitier apps into these clouds for testing. You upload the VMs into Ravello and use the drag-and-drop GUI to map out their relationships; Ravello spins them up in the cloud while preserving their network and storage configurations. It's one key feature lets you get several VMs running on just one cloud machine, a nice enhancement if you happen to have a bunch of lightweight VMs that don't need a full machine. 

A long time ago, developing software was more straightforward. You had your favorite editor and the compiler. Your file either compiled or it didn't. Today, life is much more complex. There are libraries galore and extra tools for pre-processing or post-processing or housekeeping. All of these are updated periodically, ensuring that everyone on the development team ends up with diverging environments. Eventually, code will compile on some machines and not others, causing the developers to start pointing fingers.
Vagrant juggles all of the packages for all of the tools so that everyone is working with the same environment. It smooths out the differences and ensures that the libraries and dependencies are in sync, saving everyone the hair-pulling and blame-casting.
Available under the MIT license.

Keeping a cloud of machines working together is not so simple, especially if you want new machines to come and go on demand. Salt is an open source cloud management tool that offers a Python-based communications backbone that links together the "master" and "minion" nodes. The master issues orders, and the minions carry them out. SaltStack, a "branch" of Salt backed by professional support, boasts deployments of several hundred thousand managed machines.
Salt is developed under the Apache 2.0 license.

Ubuntu Juju
Ubuntu's Juju is a project that starts to hide the machines behind a wall of services. In Juju the façade is called a "charm," which is the packaging wrapped around the service. Behind the façade is a load balancer that can ask any number of machines to answer the request for data. If you need more power, you can add more machines to each charm without reconfiguring anything else. The load balancer inside the charm knows what to do with the extra help. Ubuntu already offers a number of standard charmsthat are ready to deploy, or you can customize them for your own needs.
Ubuntu Juju is released under the GNU Affero Public License.

Chef is one of the two big tools used for configuration management. If you like writing straightforward code in Ruby for installing the right packages for your new machines, Chef is the tool for you. You write the instructions for which packages should be installed in which order, then Chef does the work for you. There is a wide variety of plug-ins ready to install many common packages.

The other major configuration management tool is Puppet. While many features are the same, the main language for specifying which packages are necessary is a bit different. Puppet asks you for a list of dependencies, then figures out how to install the right packages so that everything is ready to run. Puppet Labs maintains a large collection of plug-ins that simplify many chores.

National Train Enquiry System App (NTES)

National Train Enquiry System App
Official Indian Railways app for travelers using railways for taking holiday vacation trips, official trips, tours, and daily commute. Site provides train-running related and real-time status queries for all trains of India.