Posts

Web Performance, tell me the story

Image
Suppose you would like to know how your web site, behaves to clients from two different geo locations: for example, Finland and Ireland regarding response time. How quickly can you see and understand visually what's going on and how things are.

Do you need to improve something ? Change your network provider ? Tune your application(s) ? Spend more money. Start from a top level view, understanding the big picture and having the possibility to dive and analyze at the detail level. Finally, can you visually tell the story of your web application as in a short chart ?
Enter Kronometrix Web Performance ...


Finland Something happened between 1:00 - 1:15 AMWe can see more than one request affected by this eventOverall we see all requests are executing fast, below 0.5 secondsAnd we have some exceptions, some are taking more than 0.5 seconds
Ireland Same things from IrelandIt is different, requests usually take longer to executeSome requests are much slower than in Finland

Then we want to know…

Web Performance subscription

Image
We been busy to add support for Web Performance to our appliance. That means anyone running any HTTP applications can send data to Kronometrix for analysis. Our idea of monitoring Web applications is a simple one: it starts from the operating system, HTTP server and the application itself. We report the most important metrics including the response times for the application. To make things even easier we wanted to add support for complete solution stacks, like LAMP. (We still have lots of work to fully support them).

And to have a complete picture of the Web service, we have introduced the workload management concept inside Kronometrix to gather and report data from one or many data sources and bound those to a SLA for upper management reporting. Nice and easy.

Some first development snapshots from our implementation. Let's first switch to night mode, it is 23:10 here in Finland. So, here you go:

All requests dashboard
This shows a number of HTTP requests gathered as a stream grap…

60 messages per second

Image
Kx 1.3.2 is our next release of Kronometrix for X64 platform.  Currently we are busy testing some new features in our QA environment. We are sending data from 750 data sources, data delivered by our DataCenter Simulator. Each system is delivering 5 messages. 
Since yesterday we been processing more than 5 million messages at 60% usage.
Appliance Usage These are the main screens within the appliance administration, which shows the utilization and throughput of our system in real-time. As well we include some other information about how many users, subscriptions and data messages etc we are processing. 


and here the system utilization:



This runs our latext Kronometrix on a *very* old hardware appliance, on purpose to test and stress our software on a bit older spec. Red and Blue pipelines working just fine.
Image
Kronometrix can consume events from different sources: IT computer systems, weather stations, loggers, applications in real-time. This means we can process and display data as soon as it arrives to our machinery.  And we can prepare the information for consumption to different dashboards, continuously. But not all data can be dispatched and displayed as soon as it arrives. Here is why and how we do it.


The red (hot) pipeline Inside our kernel we process all incoming data, the raw data on a main line, the red pipeline. This is the most important pipeline within our kernel analytics. Here we transform raw data in statistics, like MAX, MIN, in AVGes for charting across different time intervals.

All sorts of UI dashboards are attached to the red pipeline, to display data as immediate as it has been processed. On this line a high level of redundancy and efficiency is enforced. Here the kernel will spend as much as possible to filter and calculate things at a very fast rate so we spend lots…

The STALL data filter

Image
Currently, Kronometrix, is supporting two types of raw data filters: the STALL and RANGE filters.

A raw data filter is a mechanism to ensure that incoming raw data follows certain rules or conditions before the numerical processing and data visualization, within Kronometrix. For example the STALL filter will ensure raw data is properly arriving to our appliance and there are no delays. The RANGE filter, will ensure incoming raw data is sane and valid, and stays within certain range of values. For example the air temperature is valid as long as it is between -50C to 50C or the CPU Utilization of a Windows server is between 0 and 100%.


The STALL Filter Defined under messages.json and part of the Library of Monitoring Objects, the STALL filter is part of the data message definition. For example this is the way to define the STALL for a data message, called sysrec, which belongs to a data source type: Linux (a computer system running Linux CentOS operating system). 
stall: seconds
The STAL…

Windows, Linux, FreeBSD all welcome

Image
By default for IT business, we support in Kronometrix, monitoring objects from Linux and FreeBSD data sources. Recently we been porting our data recorders to Windows operating system and start to offer ready made objects for this.

Example, latest Kronometrix 1.2 we plan to support Linux, FreeBSD and Windows 2008, 2012 Server editions 64 bit. Below several data sources within Kronometrix:





and then we can drill down per OS, example clicking centos:



Thats all. This is part of executive dashboard view.

Programming Republic of Perl, the Windows story

Image
Task: port Kronometrix from Linux, FreeBSD to Windows platform, including all data recorders and the transporter. Preferable use a programming language, like Perl to easy the porting process and re-utilize whatever we have already in Kronometrix.

Timeline: ASAP

Open Source: Yes


Goals Some top rules for developing the new recorders:
all recorders, must be coded in a scripting languagepreferable, all recorders must work as CLI and Windows servicesall raw data, should be easy to access, via C:\Kronometrix\log\ , no more mysteries about AppData directorytransporter should be done similar way, coded using a scripting languagememory footprint 64MB RAM Perl5 We been experimenting previously with C/C++ for Kronometrix on Windows. Nothing wrong with C/C++ except, that for every small change we had to do a lot of work & testing. We looked to PowerShell and other langauges but nothing came closer and felt like home, than Perl.

All our data recorders are simple Perl5 citizens already, so why not…