Skip to content

Using Titanoboa as an alternative to Ansible

Jump to bottom
Miro Kubicek edited this page Mar 8, 2019 · 14 revisions

I built titanoboa to be a flexible workflow tool, that can be as scalable and performant as possible while also being reliable and fault tolerant.

My initial aim was to use it as a lightweight and flexible Enterprise Service Bus (ESB) that would be cloud ready, platform independent and easily scalable. I know that in the world of micro-services and serverless hype, the ESB has become a non-cool, even dirty, word and whoever uses it automatically gives away his year of birth, which I am sure can be no later than early Pleistocene.

But even in companies with heavy focus on services, there is often a need for a simple lightweight-yet-robust integration layer that can be used as a wrapper for those (legacy) systems that cannot cater for particular API requirements. Also often you need to orchestrate various API calls and you just cannot (or do not want to) offload the orchestration onto the client.

But somehow, almost by accident, I just found out I can also use titanoboa for radically different purpose: IT automation. This means using it to automate builds and deployments. Something you normally use Cheff/Puppet for or - if you are concerned about using agents (as I am) - Ansible.

Since I often need to provision new titanoboa instances in cloud (mainly AWS) I had a bunch of scripts to do that and was going to use Ansible to put more structure to it. But before I did that I just tried an experiment: how would titanoboa handle such workflow? And I went on and built it:

And it actually worked! I did not give titanoboa any breaks and I leveraged fully its parallel processing capabilities (even found a bug in the process), so whatever tasks can be run in parallel, they are.

The workflow basically creates a clean AWS setup with dedicated RDS, EFS and EC2 instances with fresh titanoboa setup running. Step-by-step it does following:

  • using provided AWS credentials (or credentials set in env properties) and EC2 key pair it:
  • obtains information about user's default VPC and its subnets
  • creates a new security groups dedicated for RDS, EFS and EC2
  • creates a new EFS instance (in the corresponding security group) and mounts it to specified subnet
  • creates a new RDS instance in the corresponding security group and assigns it specified user credentials
  • creates a new EC2 instance using specified AMI
  • specifies EC2 user data to
    • mount EFS
    • install java
    • download titanoboa zip from specified URL and install it
    • set up necessary directory structure and file permissions
  • automatically generates all necessary titanoboa configuration (using provided template)
  • uploads to the EC2 server via sftp:
    • titanoboa configuration files
    • all other necessary resources (e.g. SSL keystore as well as JWT key pair for authentication)
  • connects to the EC2 server via SSH to
    • run titanoboa's scripts to set up its database (for user authentication and job archival);
    • add new specified titanoboa users and their credentials
    • start titanoboa on port 443

Among other details the workflow returns also the DNS of the newly set up server so you can just paste it into your browser and straight away connect to your newly set up titanoboa server!

Clone this wiki locally