Monthly Archives: October 2010

Motorola Backflip charging

Nightmare.

Chargers:

AC1) Motorola DC4050US0301 5.1V DC 850MA
AC2) AT&T 03577 5.0V 1000ma
DC1) AT&T USB VPC03578
DC2) AT&T USB + MiniUSB MV302927

Cables:

M1) Motorola SKN6378A
M2) “Motorola” SKN6238A
M3) Monoprice generic microusb

Dead Phone, AC1, M1 OR M2 OR M3
Green light on Phone, OS starts, displays charging battery

Dead Phone, AC2, M1 OR M2 OR M3
Blue light on AC2, Green light on Phone, OS starts
Green light / OS cycle every 15 seconds

Dead Phone, DC1, M1 OR M2 OR M3
White light on DC1, Green light on Phone, OS starts
Green light / OS cycle every 15 seconds

Dead Phone, DC2, M1 OR M2 OR M3
White light on DC2, Green light alternates on/off on Phone

Phone on, AC1, M1 OR M2 OR M3
Green light on, charge symbol in battery on display

Phone on, AC2, M1
Blue light on AC2 for five seconds

Phone on, AC2, M2 OR M3
Blue light on AC2
Green light on, no charge symbol in battery on display

I have an AT&T AC charger at work that I believe works as well as the stock Motorola. The AT&T AC charger here at home, listed above, is a “five star” model that consumes 0W when not charging, I assume that is what the blue light turning off indicates. Hopefully the combinations that keep the green light on the phone on are charging, just very slowly, and are still somewhat useful. More to come.

Munin Aggregation with Multigraph

Six months ago I made note to the pattern for referring to stacked graph data sources in munin:

load.double.stack one=localhost.localdomain:load.load two=localhost.localdomain:load.load

This syntax evaluates to:
graph.value.stack line=host.domain:plugin.value

I’ve been using multigraph more since then, which is a boon to performance, but it complicates stacked graphs a little. This hurts because it remains very difficult to tell why your graphs are not drawing when you incorrectly reference a data source. To debug, as the munin user (use ‘su -l munin’, ‘sudo -s -u munin’ or ‘chpst -u munin’) run:
/usr/share/munin/munin-graph --service 'load.double.stack' --debug
Be sure to replace “load.double.stack” with the name of the graph you’re trying to draw.

The munin wiki example for stacked graphs explains data source names as:

snmp_ups_current.inputtotal.sum \
---------------- ---------- ---
        |             |      |
        |             |      `-- The sum mechanism
        |             `--------- One of this virtual plugin's values
        `----------------------- The name of the virtual plugin

ups-5a:snmp_ups_ups-5a_current.inputcurrent \
ups-5b:snmp_ups_ups-5b_current.inputcurrent
------ ----------------------- ------------
   |               |                 |
   |               |                 `------ The "inputcurrent" value from the real plugin
   |               `------------------------ The real plugin's name (symlink)
   `---------------------------------------- The host name from which to seek information

However, with multigraph the name of the plugins symlink isn’t necessarily the name of the graph. The trick I found was to connect the the munin node and call the multigraph plugin, looking for the ‘multigraph’ line.

$ nc localhost 4949
# munin node at server.example.org
cap multigraph # tell munin-node that you are multigraph aware
cap multigraph
fetch diskstats # fetch the diskstats multigraph plugin
multigraph diskstats_latency
sdb_avgwait.value 0
multigraph diskstats_latency.sdb
avgwait.value 0
.

I’ve removed a significant portion of the returned data here. Pay attention to the fact that this plugin returned a “diskstats_latency” graph that contains data for all of the disks, as well as individual graphs for each disk, here “diskstats_latency.sdb” In this example your stacked graph configuration would be:

disk.double.stack \
  one=localhost.localdomain:diskstats_latency.sdb.avgwait \
  two=localhost.localdomain:diskstats_latency.sdb.avgwait
  -1- ----------2---------- -----------3--------- ---4---

(1) The alias and label for this host or data point
(2) The configured node name of the host
(3) The original graphs name, either the plugin or multigraph name
(4) The value from the plugin/graph

Notice that while the period is used to separate the value from the rest of the field, there may be periods in the rest of the field. Also keep in mind that in the past I have seen dashes in configured graph names become underscores at the end of the day.

Community Cooking

It’s been a year since the Opscode Cookbook site was launched and a recent project got me thinking about what parts of my hopes that I wrote about then have taken effect so far. I recently heard that a major Chef user has switched to Ubuntu from another Linux distribution because that is what most of the cookbooks that Opscode maintains are written for and tested on. Choice of distribution is typically something that is very dear to administrators and somewhere in the world there is a flame war on this topic every second. Consequently, this is huge and I’ve been thinking about it for a while.

It is one thing for a company to choose a distribution based on a software package that is significant to them; in the past I have had to battle against stakeholders that wanted to choose a particular distribution solely on the availability of support. Chef runs on a lot of platforms, but of course some get much more attention than others because that is where the community is. But here we’re seeing a company choose their distribution not because of Chef’s support for it, but for the community support for Cookbooks that run on it. This is clear evidence that what I wrote about is starting to happen.

I’ve been working on a Chef + Cobbler writeup in my spare time. I went out the other day and bought a consumer desktop to use as a libvirt/kvm host for this project. It often tends to be least painful to use cloud resources, but sometimes that which is taken care of for you is too unknown, too much of a “black box,” and you need the deep visibility into your infrastructure that building it yourself provides. There is indication that some have gotten Cobbler running on and deploying Ubuntu, but it doesn’t appear to have taken hold. There’s a launchpad spec claiming there is a package, but I couldn’t find it. Another spec makes it clear that files for debian packaging from upstream are not finished. It is here that I first ran into problems. I couldn’t get the init.d scripts provided in the debian/ directory of the upstream repository to work. They clearly needed some help, and after spending some time on them it became clear that they’re untested templates created by debhelper.

My goal wasn’t to fix these init scripts, I just wanted to get the cobbler server running. Then I remembered that we had a great existing runit cookbook that I was familiar with. The API for the cookbook site has progressed since release. Unfortunately the documentation for the cookbook API is a little behind, but the new functionality has been built into knife, the command line tool that interacts with the Chef server or Opscode platform, as well as multiple cloud providers. From within my personal chef-repo, I ran:

knife cookbook site vendor runit

This downloaded the runit cookbook from the Cookbooks site into a branch in my chef-repo git repository, then merged it into my current branch, allowing me to track changes between my local copy and the upstream copy using git. Then I added a couple templates and a call to the runit_service definition to my cookbook:

templates/default/sv-cobbler-run.erb:

#!/bin/sh
PATH=/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
exec 2>&1
exec /usr/bin/env cobblerd -F

templates/default/sv-cobbler-log-run.erb:

#!/bin/sh
exec svlogd -tt ./main

recipes/default.rb:

# SNIP
runit_service "cobbler"

And then cobblerd was running under runit. There’s beauty in being able to take something somewhat complex like runit, and make it easy. So easy, that I used it rather than fixing up an init script.

Then I found that cobbler wanted to called through apache as proxy. No problem though, I vendored the apache2 cookbook as well. I spent a few minutes determining that I needed a couple of Apache modules, as the Cobbler instructions are pretty centric to Redhat and I got the impression that they make assumptions about what that gives you. Then I used the apache2 cookbook to proxy cobbler by adding this to the top of my recipe:

recipes/default.rb:

include_recipe "apache2"
include_recipe "apache2::mod_proxy_http"
include_recipe "apache2::mod_python"

I had some permission problems with mod_proxy, again likely a difference between Redhat and Ubuntu, but it wasn’t out of my way to ship the apache config provided by upstream using Chef with a small modification:

<Proxy http://localhost:25151>
    AddDefaultCharset off
    Order deny,allow
    Allow from all
</Proxy>

I’ll write about the Cobbler cookbook more later. You can, of course, follow it on the Cookbook site in the interim. I want to emphasize how I used a single command to grab existing code and leverage it to make it easier for me to do what I was trying to do: get Cobbler running. The Cookbook site combined with the awesome Chef community made this possible. If you haven’t used the “cookbook site” subcommands in knife yet, take a moment to try them out.

$ knife cookbook site --help
Available cookbook site subcommands: (for details, knife SUB-COMMAND --help)

** COOKBOOK SITE COMMANDS **
knife cookbook site vendor COOKBOOK [VERSION] (options)
knife cookbook site show COOKBOOK [VERSION] (options)
knife cookbook site share COOKBOOK CATEGORY (options)
knife cookbook site search QUERY (options)
knife cookbook site list (options)
knife cookbook site download COOKBOOK [VERSION] (options)
knife cookbook site unshare COOKBOOK

Northeast travels

I’ll be traveling in the northeastern US from 10/28 through 11/13. Current plans put me in DC from 10/28 – 11/1. Then I’ll be in Boston on Tuesday 11/2 for the Boston Devops Meetup. On Wednesday 11/3, I will be in NYC for the NYC Devops Meetup at drop.io. Following that, I will be stationed back in my hometown in Maine until returning to Seattle on 11/13. If you’d like to catch up with me about operations with Chef while I am in the region, or have me stop by somewhere and give a presentation, send me an email and we can make arrangements.