[Alexandre Martins] Continuous Delivery: Deployment Smoke Tests

June 5th, 2014

I’ve always been a huge fan and advocate of using tests for developing applications. For me, working on a software without a decent suite of test is like walking on eggshells, each modification brings out the risk of breaking something on the system. To mitigate this risk I always make sure I have a minimum set of unit, integration and acceptance tests covering my application.

But does all that gives us the confidence that the system will work perfectly when it’s deployed to any of the environments, on its way through the release pipeline? I thought so until work with this guy and read this book. Tom Czarniecki firstly introduced me the concept os smoke tests, then reading Jez Humble and David Farley’s Continuous Delivery I could grok the real values of using it in conjunction with a build pipeline.

What are smoke tests?

As aforementioned, deployment smoke tests are quite handy because they give you the confidence that your application is actually running after being deployed. It uses automated scripts to launch the application and check that the main pages are coming up with the expected contents, and also check that any services your application depends on— like database, message bus, third-party systems, etc —are up and running. Alternatively you can reuse some acceptance or integration tests as smoke ones, given that they are testing critical parts of the system. The name smoke test is because it checks each of the components in isolation, and see if it emits smoke, as did with electronic circuits.

Provide clear failure diagnostics

If something goes wrong, then your smoke tests should give you some basic diagnostics explaining the reasons why your application is not working properly. In our current project at Globo.com, we are progressing towards start using Cucumber to write our smoke tests, thus having a set of meaningful and executable scripts, like this one below.

Feature: database configure
  System should connect to the database
  Scenario: should connect to the database
    When I connect to "my_app" database as root
    Then it should contain tables "users, products"

For those who like using Nagios for monitoring infrastructure, Lindsay Holmwood wrote a program called cucumber-nagios which allows you to write Cucumber tests that output the format expected of Nagios plugins, so that you can write BDD-style tests in cucumber and monitor the results in Nagios.

Knowing quickly whether you are ready or not!

Clearly rapid feedback and safety are the two major benefits of introducing smoke tests as part of a release process.

Rapid feedback

In our project, we implemented a deployment pipeline, so each new commit into the source repository is a potential deployable version to any environment, even to production. So we have the commit-stage where we run all the quick tests, and as soon as all of them passes, the acceptance-test-stage is automatically triggered, and the longer tests— integration and acceptance —are run, and once they’ve also passed, the application is automatically deployed into the dev environment. Getting a green at this stage means that it’s successfully deployed and smoke tested. But there still some exploratory testing to be performed before releasing this version into the staging environment. And in our team, this is done by the product owner, together with a developer. So as soon as they are ready to sign the story off, all they have to do is click the manual button which in turn deploy the application into the qa1 (UAT) environment, and if it’s green they can proceed, otherwise they pull the cord because something is malfunctioning, as you can see on the picture.

Don’t let the application deceive you

It’s quite frustrating, when all you need is the system to work as expected, because you are about to showcase it to your customers, and the first thing you click, all you see is a big and ugly error screen, instead of the page they were expecting. And later on you find out that it was due to database breakdown. What an embarrassing situation that could have been avoided by simply checking the smoke test diagnostics before showcasing.

[Renan Oliveira] Ontologias e sua utilização em aplicações semânticas – UFF – CASI – 2014

May 20th, 2014

Ontologias e sua utilização em aplicações semânticas – UFF – CASI – 2014 from Renan Oliveira

[Francisco Souza] Speaking at OSCON 2014

April 12th, 2014

Wow, one year without any posts! But I’m trying to get back…

This is a very short post, just to tell everybody that this year, I will have the opportunity to speak at OSCON 2014. I’m speaking about tsuru, and check more details of the talk in the tsuru blog.

[Andrews Medina] Dia 18/03 tem Meetup sobre Docker

March 16th, 2014

Na próxima semana (terça-feira, 18/03) teremos um meetup de Docker na
semana que vem, na Globo.com. É uma oportunidade bem legal para quem quiser conhecer mais sobre o projeto.

Quem tiver algum projeto relacionado com docker legal pra apresentar,
teremos lightning talks também!

Mais informação na página do meetup: http://www.meetup.com/Docker-Rio-de-Janeiro/events/165622462/

[Tiago Peczenyj] MooseX - a new Ruby DSL for Object Oriented Programming (and much more)

February 7th, 2014

MooseX is a postmodern object DSL for Ruby Build Status Gem Version

This is a DSL for object creation, aspects, method delegation and much more. It is based on Perl Moose and Moo, two important modules who add a better way of Object Orientation development (and I enjoy A LOT). Using a declarative stype, using Moose/Moo you can create attributes, methods, the entire constructor and much more. But I can’t find something similar in Ruby world, so I decide port a small subset of Moose to create a powerfull DSL for object construction.

Of course, there is few similar projects in ruby like

But the objetive of MooseX is different: this is a toolbox to create Classes based on DSL, with unique features like

  • method delegation ( see ‘handles’)
  • lazy attributes
  • roles
  • parameterized roles
  • composable type check
  • events

and much more.

This rubygem is based on this modules:

See also:

  • Joose, a javascript port of Moose.
  • Perl 6 Perl 6 OO programming style.

Why MooseX? Because the namespace MooseX/MooX is open to third-party projects/plugins/extensions. You can upgrade your Moo(se) class using other components if you want. And there is one gem called ‘moose’ :/

THIS MODULE IS EXPERIMENTAL YET! BE CAREFUL!

Talk is cheap. Show me the code!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
require 'moosex'

class Point
  include MooseX

  has x: {
    is: :rw,      # read-write (mandatory)
    isa: Integer, # should be Integer
    default: 0,   # default value is 0 (constant)
  }

  has y: {
    is: :rw,
    isa: Integer,
    default: lambda { 0 }, # you should specify a lambda
  }

  def clear!
    self.x= 0     # to run with type-check you must
    self.y= 0     # use the setter instad @x=
  end
end

# now you have a generic constructor
p1  = Point.new                       # x and y will be 0
p2  = Point.new( x:  5 )              # y will be 0
p3  = Point.new( x:  5, y: 4)

Installation

Add this line to your application’s Gemfile:

gem 'moosex'

And then execute:

$ bundle

Or install it yourself as:

$ gem install moosex

You need ruby 2.0.x or superior.

Description

MooseX is an extension of Ruby object system. The main goal of MooseX is to make Ruby Object Oriented programming easier, more consistent, and less tedious. With MooseX you can think more about what you want to do and less about the mechanics of OOP. It is a port of Moose/Moo from Perl to Ruby world.

Read more about Moose on http://moose.iinteractive.com/en/

Motivation

It is fun

Usage

You just need include the MooseX module in your class and start to describe the attributes with our DSL. This module will inject one smart constructor, acessor and other necessary methods.

Instead the normal way of add accessors, constructor, validation, etc

1
2
3
4
5
6
7
8
9
10
11
12
class Foo
  attr_accessor :bar, :baz, :bam

  def initialize(bar=0, baz=0, bam=0)
    unless [bar, baz, bam].all? {|x| x.is_a? Integer }
      raise "you should use only Integers to build Foo"
    end
    @bar = bar
    @baz = baz
    @bam = bam
  end
end

you can do this:

1
2
3
4
5
6
7
8
9
class Foo
  include MooseX

  has [:bar, :baz, :bam], {
    is: :rw,
    isa: Integer,
    default: 0
  }
end

Contributing

  1. Fork it ( http://github.com/peczenyj/MooseX/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request Push to the branch (git push origin my-new-feature)

[Andrews Medina] venha fazer parte da globo.com

January 8th, 2014

A globo.com é líder absoluta no mercado de portais de notícias, esportes e entretenimento em toda a América Latina. São milhões de acessos por dia aos portais (g1.globo.com, globoesporte.globo.com, gshow.globo.com) com a home da globo.com sendo a principal fonte de acessos com cerca de 45 milhões de visualizacoes por dia.

Trabalhar na globo.com significa que você dividirá seu dia-a-dia com algums dos melhores profissionais do país, usará as melhores ferramentas (open source) para completar seus desafios e fará parte de um ótimo ambiente de trabalho. Terá a oportunidade de participar de eventos assistindo ou palestrando, e terá um macbook a sua disposição.

Se você ficou interessado, basta mandar um email para andrews@corp.globo.com com informações que você julgue relevante (currículo, LinkedIn, Blog, GitHub, etc.) e uma breve explicação do porquê você quer fazer parte da nossa equipe.

ps: as vagas são para trabalhar no nosso escritório no rio de janeiro

[Flávio Ribeiro] Naming Things

December 14th, 2013

There are only two hard things in Computer Science: cache invalidation and naming things.

Phil Karlton

I was thinking about how important naming things is to me. Since the beginning of my undergraduate studies, I paid close attention to the names of every project I’ve done and every company or lab that I’ve worked at.

I’m also catching myself arguing about the name of things on my daily work and I was able to notice how bad I am at it. This post aims on forcing me to improve my naming things skill.

Regarding Project names

The last project we did at Globo.com had a good name. It’s called Sauron, and it’s main purpose is to collect and show server’s health, QoS/QoE analysis, handle video signals, network throughput, etc. It fits well as Sauron represents himself as an all seeing eye. It’s short, compact, nerdy, and I like it.

But we have a lot of bad names there: Streams-analyzer, EvoHandler and VMine are just some of then that are too vague or don’t really explain what they do (sorry coworkers, don’t take it personally).

My undergraduate project was called Marmota, an acronym for making arduino and mobile devices talk that forms the name of a funny animal in Brazilian Portuguese. I don’t know if anyone has looked at this fact, but I have no doubt that the name helped on its 15-minutes of fame [1] [2] [3] [4] [5].

Now I’m trying to engage myself on bem.tv, an attempt to scale live video streaming using P2P. Actually, bem.tv is another joke with a bird name and TV suffix, which refers to video broadcast. It’s small, simple and easy to say.

Just to name other cool project names that aren’t associated to animals but are good ones: HTTPretty, Flask, PyPy, Spdy, Bundle, YouTube.

Who would guess — even amid its barrage of Super Bowl ads — that Go Daddy is the name of an Internet domain registration site?

Paul Farhi

Yes, you must be thinking about the projects whose names make no sense but they’re much bigger and successfull than the ones that I listed. I’m not here judging the formula of success. I’m just concluding that short, solid names that make associations with cool stuff and the project purpose itself is a good formula. For me, a good logo as bower, gruntjs or yeoman also adds value to the project.

Regarding code

Naming classes, functions and modules are even more complicated. At our office we work with code that interacts with multimedia, and it’s common to deal with names that have more than one meaning. Stream, Signal, Media and Bitrate are constantly used in our codes, and those are a constant source of confusion.

For now we don’t have a solution for this, but docstrings on the code or a small meeting followed by a document describing the decisions and conventions should fix it. Since our projects are growing exponentially, perhaps it’s time to schedule this reunion.

Uncle Bob’s Clean Code book have an entire chapter explaining names and how to use and improve it in your code. He defends that names must be pronounceable and searchable, and you as developer should avoid reader’s to mentally translate names (avoid mental mapping).

Length is not a virtue in a name; clarity of expression is.

Rob Pike

I agree that length isn’t directly related to self-explanation, and a good trick is look at your language syntactic sugars to improve the readability on the context at where your variable name or function is being used. I’m sure you’ll find something to enforce the name you chose.

Once in a while, ask people outside your team to review your code before pushing it to master. They will ask questions that were obvious to you and your teammates, but will alert where you are using bad names.

Outsiders have other context in mind, and their questions will probably improve your code design and names. It’s like visual design, information architecture. 4, 6 or 8 eyes is always better than 2.

More References:

[Flávio Ribeiro] Starting a blog again..

December 7th, 2013

Hey! I’m just testing my blog and adjusting layout and stuff..

See you.

[Tiago Motta] Meu primeiro aplicativo Android

October 14th, 2013

Publiquei essa semana no Google Play meu primeiro aplicativo Android. O 9Gag Offline trata-se de um aplicativo que permite que você baixe os gags do site 9Gag para momentos em que não haja conexão com internet. Por exemplo se você for viajar, no avião você ficará isolado do mundo mas poderá rir bastante com os gags que estarão guardados no seu celular.

Já fazia mais ou menos uns seis meses que estava estudando esporadicamente o sistema operacional da Google e fazendo diversos testes. A um mês atrás resolvi fechar de vez o aplicativo e colocá-lo ao público. Deu pra aprender bastante sobre o framework e reviver os problemas de sincronia e concorrência que haviam na época em que programava para desktop com Delphi, Kylix e Java Swing.

Espero que gostem e se acharem qualquer problema ou tiverem alguma sugestão é só avisar. Para baixar o 9Gag Offline basta clicar aqui e ir ao Google Play.

[Flavia Missi] Building and installing lxml with PyPy

September 4th, 2013

Introduction

The major issue my colleagues and I found when we started running some projects with PyPy was the lxml library. It uses Cython, which can run with PyPy if you write your code portably enough. So an effort began to port lxml to use CFFI. This effort can be found on this fork and this is the code we’re going to install from.

Resolving dependencies

We are going to install lxml on a ubuntu 13.04, be warned that installation in OSX might give you serious headaches (<10.8). Start by running the following apt-get:

$ sudo apt-get install libxml2 libxslt1-dev zlib1g-dev

These packages are needed to build lxml (with Python or PyPy).

Bootstraping your environment

You’ll need to have PyPy’s binary to build lxml with, you can folow Andrews Medina’s guide to install it (but it’s in portuguese…)

Assuming you have it installed let’s create a virtual environment (with virtualenv and virtualenvwrapper) to install lxml in:

$ mkvirtualenv lxml-pypy -p /path/to/pypy/bin/pypy

Now clone the lxml fork and checkout to the CFFI branch:

$ git clone https://github.com/amauryfa/lxml.git
$ cd lxml
$ git checkout origin/cffi

Now build and install (double check if you’re in the right virtual environment):

$ python setup.py build
$ python setup.py install

Done!