Project-FiFo Blog

Articles and Blog Posts related to Project FiFo

Building a Cloud with Erlang and SmartOS

By

Heinz Gies: Erlang User Conf 2014

Building a Cloud with Erlang and SmartOS

How Hard Could it Possibly Be?

Project FiFo has been around for two years now and grown from a for fun project to run my hobby server to powering private systems installation in multiple universities and last but not least multiple datacenters for the Lucera public cloud.

Between the start and what we have today, FiFo has become a rather complex distributed system and it was not always a smooth ride to get here. This talk will go over the biggest issues and failures and what I have learned from them.

Talk objectives:

Sharing some of the issues throughout FiFo’s history, giving an idea how they were solved and what I learned from it. Also if possible giving everyone a good laugh and my stupidity.

Target audience:

People interested in distributed systems, failure stories, how ‘clouds’ are built and/or SmartOS.

DalmatinerDB metrics get tags

By

We build DalmatinerDB for one purpose, be able to ingest and query more metrics then any other metric store that exists. I am rather confident that we succeeded with that goal. Part of why this has been possible is that it is build for simplicity, we use the same tree structure for metrics that Graphite uses, we use flat files instead of a elaborate database to store metrics, we leverage existing technology like ZFS and Riak Core instead of trying to roll our own clustering, compression, file integrity etc tools, all that removes overhead.

Last week the team of dalaloop.io came over and we set together to discuss and work on tags, or labels, or dimensions, however you want to call them. They are very nice and helpful, however they are not necessarily simple. They conflict with the file layout and data-structures in the DalmatinerDB Backend wich already is not fast for looking up metrics on wildcards. The good news is the modular design of DalmatinerDB allows to not bother the backend with this kind of problems. [Read more…]

The stack we choose: Erlang, SmartOS, Clojure

By

Often we hear that Project FiFo uses an unusual technology stack. Looking at the past few years of development experience, the stack has proven to be a positive one that continues to prove itself to be the right choice. The following should serve as a rationale for the choices behind each component, and some of the experiences we have had along the way.

The OS

We choose to run all FiFo and related services on SmartOS, the reason for this is simple. Since FiFo manages SmartOS clouds – it would be silly not to “eat our own dog food” and to run FiFo on something else.

We use vanilla SmartOS, so that there is no dependency on FiFo for your running VM’s. You could just switch FiFo off and all your VM’s would continue to just work. It also [Read more…]