Tuesday, May 5, 2015

Kronometrix Analytics Appliance: Xen vs ESXi

Latest Update: Tue May  5 15:54:10 EEST 2015

We heavily use Redis as part of our Kronometrix analytics software. For those which do not know what Redis is, see here.

Recently we have upgraded our kernel from Redis 2.8.x branch to 3.x, latest release. And we were curious to see what sort of improvements and how latest Redis release works for our application in regard with ESXi and Xen. As already said, Kronometrix Appliance can works as a:
  • physical appliance, operating system + analytics software on bare metal
  • virtual appliance, running as one or many virtual machines within Xen or ESXi
We are planning to check the redis-server proces utilization, extract a stack trace for the redis-server on both hypervisors.


Kronometrix 1.0.0 on Xen

Generic X86_64 Server
Ubuntu Server 12.04.5 LTS, Xen 4.2.2 amd64
1 x Intel(R) Core(TM) i7 CPU 930 @ 2.80GHz
Hyperthreading: Available, ON
1 TB internal storage for dom0, 2TB NAS for domU

We have configured dom0 to boot and have 2 assigned vCPUs and 6 GB RAM:  GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=6500M dom0_max_vcpus=2 dom0_vcpus_pin"

Kronometrix Kernel Build 97, Redis 3.0.1, 450 DS

We are testing Kronometrix Kernel against 450 data sources, a data source here means a computer system like Linux, or FreeBSD running as virtual machine, sending 5 data messages per host, every minute, running on Xen.

CPU Utilization:
Xen redis-server process, 18% CPU usage


Kronometrix 1.0.0 on ESXi

Dell PowerEdge 2950
ESXi 5.5.0 Update 2
1 x Intel(R) Xeon Core(TM) E5430 @ 2.66GHz
Hyperthreading: Not Available, OFF
20 GB RAM, 2 x 1 TB internal storage RAID 1 via Dell PERC 6/i (LSI)

We have configured the 2 x internal disks as RAID 1 mirror where we have installed the ESXi hypervisor and have the datastorage for VMs.

Kronometrix Kernel Build 97, Redis 3.0.1, 450 DS

We are testing Kronometrix Kernel against 450 data sources, a data source here means a computer system like Linux, or FreeBSD running as virtual machine, sending 5 data messages per host, every minute, running on ESXi.

CPU Utilization:
ESXi, redis-server process 70% CPU usage


1. DTrace 60seconds, stack trace dump:

dtrace -x ustackframes=100 -n 'profile-997 /pid == 3263/ { @[ustack()] = count(); } tick-60s { exit(0); }' -o stacks


Monday, May 4, 2015

The Appliance ...

Our analytic software, Kronometrix, runs on top of an operating system, tuned and configured for data analysis, bundled as a ready image, which can be downloaded and installed. The idea is simple, we wanted to offer something which is already tested, tuned and configured for hundreds, thousands of data sources. We call this, the Kronometrix Appliance and we want it to be ready to:
  • work with or without WAN connection
  • work on batteries and solar panels
  • handle large or small volume of data in real-time
  • analyse data from different data sources, different industries
  • have zero-administration and no maintenance
  • be affordable


Memory it's the new hard disk 

Another PHP or Java application on top of MySQL ? Or maybe PostgreSQL ? Right ? Boringggg.

When we started Kronometrix, we basically cleaned the table and we wanted to see things aligned for future not 1978. The first thing, was where to store all the time series data and how to access that. And we went full speed into the future: a NoSQL database all mapped to RAM, fast and efficient to store any sort of data structure we might need. Then it came the persistence to disk, since any system might go South, so persistence was an important factor. So we went for Redis. A solid NoSQL database with persistence on disk, fast, efficient and lots of data structures to chose from running across a number of Linux and UNIX operating systems.

Note: we evaluated at that dedicated time series databases, like InfluxDB but we found it complex, and buggy, hard to maintain and operate unless you want to deploy 20 computing nodes, poor time-series analysis capabilities within.

It's Samba time

Then it came the language and the web framework around it. Java, PHP, Python, Ruby all with their own pro and cons. But this time we looked further and stopped at Pontifical Catholic University of Rio de Janeiro in Brazil. Welcome Lua, a very powerful, fast, lightweight, embeddable scripting language running on OpenResty, one of the fastest web frameworks on planet Earth.

Kronometrix has been designed to work out of the box, no complicated installation, no extra tuning, no hours spent in adding more data base cluster nodes, etc etc. It just works as: physical or virtual appliance.

1. Physical appliance

Suppose you want to receive data from 1500 hosts in a data center, every minute and process up to 100 performance metrics, each metric computing the MIN, MAX, COUNT, SUM, AVG, LAST across 10 predefined time range intervals. That would take a bit some effort, and if you look further into even more dense data centers with hundreds of racks the physical appliance is there to help. For those ready to digest data in real-time from thousands of data sources, with minimal storage latency and full speed, we have a ready appliance which be easily racked and powered on.

Or think about a remote damaged facility, a warehouse or factory which needs to be continuous monitored using a private LAN, without a permanent WAN connection. Worse the power electricity supply on that location is not stable. For that a light and mobile appliance, can be installed to receive data from dozens of environmental sensors placed on the factory.

Another example, a small airfield, 1 run-way small airport which would like to share meteorological data to the instructor pilots and local operators, all using the private LAN. More, they would like to keep the meteorological raw data and analyse it every 6 months in order to see how many cancellations were made due bad weather and poor visibility. For such setup, Kronometrix Mobile Appliance helps in gathering data from all weather stations and nicely displaying it to pilots, operators and personnel.

2. Virtual appliance

Kronometrix can easily be installed as a virtual image under any modern hypervisor: Xen, KVM, ESX, or on any public cloud providers like Amazon EC2, Rackspace, Joyent or any other public cloud operator. Same top features are available as a virtual appliance, focusing on zero-administration, no maintenance, easy deployment. 

Using a cloud or virtualization infrastructure software will allow Kronometrix to quickly expand and use more virtual machines to handle larger amounts of data with minimal effort. 

The software is available as a ready to be installed ISO image.