Start a Docker host in AWS with docker-machine

Download the appropriate binary for your workstation:

https://docs.docker.com/machine/

sudo curl -L https://github.com/docker/machine/releases/download/v0.2.0/docker-machine_linux-amd64 > /usr/local/bin/docker-machine
sudo chmod +x /usr/local/bin/docker-machine

Create the host

docker-machine  create docker -d amazonec2 \
  --amazonec2-access-key ABCDEFGHIJKLMNOP \
  --amazonec2-secret-key '1234567890abcdefghijklm' \
  --amazonec2-subnet-id subnet-12345678 \
  --amazonec2-vpc-id vpc-1234abcd \
  --amazonec2-zone c

What you get:

INFO[0001] Launching instance...
INFO[0026] Waiting for SSH on 54.165.56.23:22
sudo: unable to resolve host ip-10-2-0-254
sudo: unable to resolve host docker
sudo: unable to resolve host docker
INFO[0242] "docker" has been created and is now the active machine.
INFO[0242] To point your Docker client at it, run this in your shell: eval "$(docker-machine env docker)"

Los números de 2014

Los duendes de las estadísticas de WordPress.com prepararon un informe sobre el año 2014 de este blog.

Aquí hay un extracto:

La sala de conciertos de la Ópera de Sydney contiene 2.700 personas. Este blog ha sido visto cerca de 31.000 veces en 2014. Si fuera un concierto en el Sydney Opera House, se se necesitarían alrededor de 11 presentaciones con entradas agotadas para que todos lo vean.

Haz click para ver el reporte completo.

Getting the list of IP address ranges for a specific AWS Region

Amazon just published a JSON file with the details of the IP Address ranges used on each region and service.

jq can be used to get the list of ranges for a specific AWS Region and service.

curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.region | contains("us-east-1")) | select(.service | contains("EC2"))  | .ip_prefix'
50.19.0.0/16
75.101.128.0/17
54.208.0.0/15
54.172.0.0/15
50.16.0.0/15
54.156.0.0/14
174.129.0.0/16
54.92.128.0/17
54.224.0.0/15
72.44.32.0/19
184.73.0.0/16
54.204.0.0/15
54.160.0.0/13
54.196.0.0/15
54.210.0.0/15
184.72.128.0/17
216.182.224.0/20
54.144.0.0/14
54.198.0.0/16
54.88.0.0/14
54.226.0.0/15
54.234.0.0/15
54.242.0.0/15
204.236.192.0/18
67.202.0.0/18
54.174.0.0/15
107.20.0.0/14
54.236.0.0/15
54.221.0.0/16
184.72.64.0/18
54.80.0.0/13
23.20.0.0/14

Cloud-init simplest example (with RHEL7 and AWS)

Cloud-init is the defacto multi-distribution package that handles early initialization of a cloud instance

A simple usage example is to install and start an Apache web service in a AWS instance based on Red Hat Enterprise Linux 7.
To get that, we are going to run a script that adds the package and starts the daemon. To make cloud-init run the script, the first line must specify the script interpreter, in this case “#!/bin/bash”

#!/bin/bash
yum install -y httpd
service httpd start

If you are launching the instance from the AWS Console, it’s just to use the Advanced options on the step 3, and put our script as “UserData”.

In case of using command line, write the script in a file and use option -f


ec2-run-instances ami-785bae10 --instance-type t2.micro -k rhel7 --subnet subnet-518ff579 -f userdata.txt

To debug eventual problems you can check the cloud-init.log

[ec2-user@ip-10-1-2-13 ~]$ sudo cat /var/log/cloud-init.log
Jul  5 10:25:36 localhost cloud-init: Cloud-init v. 0.7.4 running 'init-local' at Sat, 05 Jul 2014 14:25:35 +0000. Up 26.53 seconds.
Jul  5 10:25:37 localhost cloud-init: Cloud-init v. 0.7.4 running 'init' at Sat, 05 Jul 2014 14:25:37 +0000. Up 27.58 seconds.
Jul  5 10:25:37 localhost cloud-init: ci-info: ++++++++++++++++Net device info+++++++++++++++++
Jul  5 10:25:37 localhost cloud-init: ci-info: +--------+-------+---------+------+------------+
Jul  5 10:25:37 localhost cloud-init: ci-info: | Device |   Up  | Address | Mask | Hw-Address |
Jul  5 10:25:37 localhost cloud-init: ci-info: +--------+-------+---------+------+------------+
Jul  5 10:25:37 localhost cloud-init: ci-info: |  lo:   | False |    .    |  .   |     .      |
Jul  5 10:25:37 localhost cloud-init: ci-info: | eth0:  | False |    .    |  .   |     .      |
Jul  5 10:25:37 localhost cloud-init: ci-info: +--------+-------+---------+------+------------+
Jul  5 10:25:37 localhost cloud-init: ci-info: ++++++++++++++++++++++++++++++Route info++++++++++++++++++++++++++++++
Jul  5 10:25:37 localhost cloud-init: ci-info: +-------+-------------+----------+---------------+-----------+-------+
Jul  5 10:25:37 localhost cloud-init: ci-info: | Route | Destination | Gateway  |    Genmask    | Interface | Flags |
Jul  5 10:25:37 localhost cloud-init: ci-info: +-------+-------------+----------+---------------+-----------+-------+
Jul  5 10:25:37 localhost cloud-init: ci-info: |   0   |   0.0.0.0   | 10.1.2.1 |    0.0.0.0    |    eth0   |   UG  |
Jul  5 10:25:37 localhost cloud-init: ci-info: |   1   |   10.1.2.0  | 0.0.0.0  | 255.255.255.0 |    eth0   |   U   |
Jul  5 10:25:37 localhost cloud-init: ci-info: +-------+-------------+----------+---------------+-----------+-------+
Jul  5 10:25:38 localhost cloud-init: Cloud-init v. 0.7.4 running 'modules:config' at Sat, 05 Jul 2014 14:25:38 +0000. Up 28.98 seconds.
Jul  5 10:25:38 localhost cloud-init: Cloud-init v. 0.7.4 running 'modules:final' at Sat, 05 Jul 2014 14:25:38 +0000. Up 29.32 seconds.
Jul  5 10:25:39 localhost cloud-init: Loaded plugins: amazon-id, rhui-lb
Jul  5 10:25:41 localhost cloud-init: Resolving Dependencies
Jul  5 10:25:41 localhost cloud-init: -->; Running transaction check
Jul  5 10:25:41 localhost cloud-init: --->; Package httpd.x86_64 0:2.4.6-17.el7 will be installed
Jul  5 10:25:42 localhost cloud-init: -->; Processing Dependency: httpd-tools = 2.4.6-17.el7 for package: httpd-2.4.6-17.el7.x86_64
Jul  5 10:25:43 localhost cloud-init: -->; Processing Dependency: /etc/mime.types for package: httpd-2.4.6-17.el7.x86_64
Jul  5 10:25:43 localhost cloud-init: -->; Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-17.el7.x86_64
Jul  5 10:25:43 localhost cloud-init: -->; Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-17.el7.x86_64
Jul  5 10:25:43 localhost cloud-init: -->; Running transaction check
Jul  5 10:25:43 localhost cloud-init: --->; Package apr.x86_64 0:1.4.8-3.el7 will be installed
Jul  5 10:25:43 localhost cloud-init: --->; Package apr-util.x86_64 0:1.5.2-6.el7 will be installed
Jul  5 10:25:43 localhost cloud-init: --->; Package httpd-tools.x86_64 0:2.4.6-17.el7 will be installed
Jul  5 10:25:43 localhost cloud-init: --->; Package mailcap.noarch 0:2.1.41-2.el7 will be installed
Jul  5 10:25:43 localhost cloud-init: -->; Finished Dependency Resolution
Jul  5 10:25:43 localhost cloud-init: Dependencies Resolved
Jul  5 10:25:43 localhost cloud-init: ================================================================================
Jul  5 10:25:43 localhost cloud-init: Package       Arch     Version        Repository                          Size
Jul  5 10:25:43 localhost cloud-init: ================================================================================
Jul  5 10:25:43 localhost cloud-init: Installing:
Jul  5 10:25:43 localhost cloud-init: httpd         x86_64   2.4.6-17.el7  rhui-REGION-rhel-server-releases   1.2 M
Jul  5 10:25:43 localhost cloud-init: Installing for dependencies:
Jul  5 10:25:43 localhost cloud-init: apr           x86_64   1.4.8-3.el7   rhui-REGION-rhel-server-releases   103 k
Jul  5 10:25:43 localhost cloud-init: apr-util      x86_64   1.5.2-6.el7   rhui-REGION-rhel-server-releases    92 k
Jul  5 10:25:43 localhost cloud-init: httpd-tools   x86_64   2.4.6-17.el7  rhui-REGION-rhel-server-releases    77 k
Jul  5 10:25:43 localhost cloud-init: mailcap       noarch   2.1.41-2.el7  rhui-REGION-rhel-server-releases    31 k
Jul  5 10:25:43 localhost cloud-init: Transaction Summary
Jul  5 10:25:43 localhost cloud-init: ================================================================================
Jul  5 10:25:43 localhost cloud-init: Install  1 Package (+4 Dependent packages)
Jul  5 10:25:43 localhost cloud-init: Total download size: 1.5 M
Jul  5 10:25:43 localhost cloud-init: Installed size: 4.3 M
Jul  5 10:25:43 localhost cloud-init: Downloading packages:
Jul  5 10:25:44 localhost cloud-init: --------------------------------------------------------------------------------
Jul  5 10:25:44 localhost cloud-init: Total                                              2.1 MB/s | 1.5 MB  00:00
Jul  5 10:25:44 localhost cloud-init: Running transaction check
Jul  5 10:25:44 localhost cloud-init: Running transaction test
Jul  5 10:25:44 localhost cloud-init: Transaction test succeeded
Jul  5 10:25:44 localhost cloud-init: Running transaction
Jul  5 10:25:44 localhost cloud-init: Installing : apr-1.4.8-3.el7.x86_64                                       1/5
Jul  5 10:25:44 localhost cloud-init: Installing : apr-util-1.5.2-6.el7.x86_64                                  2/5
Jul  5 10:25:44 localhost cloud-init: Installing : httpd-tools-2.4.6-17.el7.x86_64                              3/5
Jul  5 10:25:44 localhost cloud-init: Installing : mailcap-2.1.41-2.el7.noarch                                  4/5
Jul  5 10:25:45 localhost cloud-init: Installing : httpd-2.4.6-17.el7.x86_64                                    5/5
Jul  5 10:25:46 localhost cloud-init: Verifying  : mailcap-2.1.41-2.el7.noarch                                  1/5
Jul  5 10:25:46 localhost cloud-init: Verifying  : httpd-tools-2.4.6-17.el7.x86_64                              2/5
Jul  5 10:25:46 localhost cloud-init: Verifying  : apr-util-1.5.2-6.el7.x86_64                                  3/5
Jul  5 10:25:46 localhost cloud-init: Verifying  : apr-1.4.8-3.el7.x86_64                                       4/5
Jul  5 10:25:46 localhost cloud-init: Verifying  : httpd-2.4.6-17.el7.x86_64                                    5/5
Jul  5 10:25:46 localhost cloud-init: Installed:
Jul  5 10:25:46 localhost cloud-init: httpd.x86_64 0:2.4.6-17.el7
Jul  5 10:25:46 localhost cloud-init: Dependency Installed:
Jul  5 10:25:46 localhost cloud-init: apr.x86_64 0:1.4.8-3.el7                 apr-util.x86_64 0:1.5.2-6.el7
Jul  5 10:25:46 localhost cloud-init: httpd-tools.x86_64 0:2.4.6-17.el7        mailcap.noarch 0:2.1.41-2.el7
Jul  5 10:25:46 localhost cloud-init: Complete!
Jul  5 10:25:46 localhost cloud-init: iptables: Index of insertion too big.
Jul  5 10:25:46 localhost cloud-init: The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.
Jul  5 10:25:46 localhost cloud-init: Redirecting to /bin/systemctl restart  iptables.service
Jul  5 10:25:46 localhost cloud-init: Failed to issue method call: Unit iptables.service failed to load: No such file or directory.
Jul  5 10:25:46 localhost cloud-init: Redirecting to /bin/systemctl start  httpd.service
Jul  5 10:25:46 localhost cloud-init: Cloud-init v. 0.7.4 finished at Sat, 05 Jul 2014 14:25:46 +0000. Datasource DataSourceEc2.  Up 37.45 seconds

Full documentation: http://cloudinit.readthedocs.org/en/latest/index.html

7 Cloud-based tools for the modern sysadmin

The old days spending long hours building on premise infrastructure is over. There are so many cloud-based tools, with such a good quality and easiness for deployment, that the valuable sysadmin time cannot be spent implementing complex tools. Prices are accessible, and generally, there are different levels of licensing where companies can find their right fit.

Most of these tools can be cataloged as Software as a service (SaaS)

New relic

Focus of New Relic is real-time performance monitoring for applications. It covers the most used platforms as Java, Python, PHP, .NET, etc. The insights the tool can provide are excellent, even in some cases it seems to be magical, for example automatically linking different services of your platform.

Not only application stats can be collected, New relic has a series of agents or plugins to monitor from the OS level to the database level. Although it is not an inexpensive tool.

In addition to metrics, the tool can capture app errors. And using thresholds for the performance metrics and error rates is possible to send alarms, what implies a much more intelligent control of the infrastructure.

PagerDuty

Now a days receiving alerts by e-mail is not enough. A tool like PagerDuty can manage alarms from different sources, and send them via e-mail, SMS or phone calls. It is a great tool to centralize events, and smartly distribute them, for example based on an on-call schedule.

The most basic method to receive alters is by email, but more sophisticated sources can be used, the APIs provided by Pager Duty enables ntegration with tools like New Relic and services like AWS, among others.

Loggly

Log analysis is a central task during the troubleshooting of issues, and even to proactively detect problems.

The Unix power tools, like grep, are ever present, but sometimes even more powerful tools are required. Graphical representation of events could be key to identify root-causes and easy and fast searches save a lot of time.

Loggly can aid the sysadmin in these matters. Although there are excellent open-source tools, like logstash, to deal with logs, it has a quite difficult learning curve, and sometimes it could have too much complexity. Loggly is an SaaS tool, where you can have log analysis up and running as fast as in few minutes.

Confluence Atlassian OnDemand

Documenting, at least for me, is an enjoyable task. Even more with neat tools.

The wiki concept is here since several years ago. There are plenty of free wiki systems, and maybe they can be enough for most of the environments. But if you want to have an enterprise-class tool, I think Confluence is the right choice. Even at an affordable price.

Atlassian also is leader with tools like Jira, which have a very good integration with Confluence

Draw.io

Network diagramming, was always a choice between two extremes: on one side a feature-rich and expensive tool (Visio), on the other, rustic tools with a bad usability.

Draw.io is a quite good tool, with the basic features required to make a good looking diagrams. Although It doesn’t have a wide number of stencils as in Visio, nor is extensible with new ones, if you don’t need such a fancy graphics, this tool is more than enough.

Regarding the cloud features, it allows saving your files in Google Drive, Dropbox and other cloud-storages.

In addition… it’s free

Toggl

Time tracking and logging is always unattractive for the technical expert, it can be an awful task if the tools available are old, hard to use, or too rigid. Not to mention the multitasking nature of any sysadmin role, what makes even harder to associate each task to a given issue, project, or client.

A tool like Toggl makes the task easier and a bit more attractive. In the web version, with a single click you can start tracking time for your daily tasks. The mobile app does not let you make excuses of not having the computer close.

Checkvist

Finaly a simple tool to keep to do lists.