[Bernardo Heynemann] thumbor is a freaking awesome project or why open source will win every time

People who know me are well aware that I love Open Source with all my heart. I have more than 50 open-source repositories in my github account. Some are maintained, some are not.

This post is not to talk about me, though. It is to talk about how freaking incredibly awesome Open Source is and how people will surprise you every time.

tl;dr

Thumbor is a much better project because of the MANY MANY contributions we have received from the comunity.

I can’t stress enough how incredibly fortunate we are that there are so many VERY SMART people out there willing to contribute back to our project.

That’s why Open-Source will win every time against proprietary software. Because of the people. Keep reading if you want to know more about our story.

The Project

Thumbor is an open-source image operations server. Don’t let this simple description fool you. IT IS powerful. It does INCREDIBLE stuff that saves our company a boatload of money.

When we started the project, the development team decided on open-sourcing the project.

It is general purpose enough as not to require any of our internal information and/or business details to leak.

This decision comes with some trade-offs, that seem very negative at first. A couple questions that came up?

  • How are we going to change the way Thumbor stores images? Do we need to fork the project to have our “company version” of it?
  • How do we load images from our domains only? (Repeat first question proposed solution and rinse)
  • How do we stop attackers from overloading our servers with requests for images of different sizes?
  • How do we stop competitors from using our software to their advantage?
  • How is this any valuable compared to using a proprietary solution (given we have the money to buy it)?

As you can see, there are many questions people came up with NOT to open-source the project.

We decided we would tackle each of those problems when their time came.

The Team

I want to give a brief description of the team behind Thumbor just to clarify why we decided to open-source it even in the face of so many questions.

First, there’s Fábio Costa. He’s a kick-ass developer, committer of the MooTools project and a great colleague. He’s also a BIG supporter of the Open-Source philosophy.

Rafael Carício is also a big-time supporter of Open-Source projects, being committer of Pyvows and many other open-source projects. Recently he spent two days just fixing issues with the default Python interpreter. Pretty awesome if you ask me.

Then, there’s Cezar Sá. Again, an avid Open-Source supporter. He’s committer of Rubinius, an alternative implementation of the Ruby language. He’s the guy behind Thumbor filters architecture.

The Decision

If we were going to open-source Thumbor, we needed to make sure it was as extensible as possible.

Every single part of Thumbor needed to be easily switchable for a different part with the same contract.

This kind of architecture is not simple to build, so we came up with parts that would be general enough so you can start using Thumbor right away.

We also needed to come up with a system to stop people from exploiting Thumbor to generate an infinite number of images and thus overload the server. We came up with encrypted URLs. We don’t believe in security by obscurity either, meaning that even if the software was closed source, people would exploit it.

The company we work for, globo.com, has many, many images (millions) and many users (nearly 5B page views/mo). So we had to make sure Thumbor was up to the task. So we fine tuned it.

The Premises

Ok, so what were our premises for thumbor?

  1. We need everything to be extensible, so we also need to come up with reasonable implementations of the extensible parts;
  2. We need Thumbor to be safe, so we must stick to secure by convention, meaning that if you don’t change a thing, Thumbor is secure;
  3. We need Thumbor to be fast so it can handle many operations per second without requiring expensive clusters

I’m skipping intentionally the main premise which is we want Thumbor to be the best software at cropping images which is what drove us to build it in the first place.

Skip a couple months to the future…

Ok, we have the first version done! Let’s go live with it.

So we fire up our servers and Thumbor is a go. We notice it’s a little slow, but hey, it’s doing its job and we started with a small team of users.

Then the unexpected happens!

Community created issues start popping up! And then they start coming up with CODE ATTACHED.

Now let’s stop for a moment and analyse this. There are MANY companies out there that charge A LOT of money for testing services.

We have FREE skilled testers in our project now. People who are proactively testing it for us and reporting back their findings.

Not only that, they are fixing our software for us and giving us back the code with NO STRINGS ATTACHED.

Let me say this again, these people, highly skilled individuals, all of them WITH JOBS, are working for free in a project they did not start.

This is humans at their best if you ask me!

Extreme Makeover

Remember I said that we’d implemented all the extensible parts and security?

That’s another incredible aspect of Open-Source Software: people READ your code.

People read ours. They found MANY, MANY things to improve/change/add/remove. We are grateful for every single one of them.

The project would not be as good as it is for our users if it wasn’t for the people that are contributing.

Why do I say that Thumbor underwent Extreme Makeover? Because if you look at the first version that we released and how Thumbor is right now there’s no way you would say its the same software.

Through contributions we improved storage, loading, graphics engines, security (A LOT), performance (A LOT) and our software practices.

That’s actually one thing of the process of developing open source software that is very humbling. People pay more attention to software practices like testing and continuous integration when they are trying to get their patches accepted.

And they call on you when you are slipping on your side of the fence. And we got called! And we listened. All of us came out of the process better at our craft.

The Conclusion

Thumbor has already payed for itself many times. It is so useful to us that we don’t care if our competitors use it, as long as the community keeps improving it.

As for buying a proprietary software, I haven’t found a single one that does the same as Thumbor and even if we do, we’ll never get this level of creativity, support and diversity from any given company.

This means if we have to choose again between open or closed source, I think we’ll stay with open source every single time.

HUGE MEGA THANKS WITH RAINBOWS AND UNICORNS

I think I did stress in this post how much I appreciate all the contributions, but I still feel obligated to thank you guys. Your contributions have been incredible and are all INVALUABLE.

So sincere thanks to (in no particular order):