Comments from a newbie

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Comments from a newbie

dpal
Hello,

I am newbie to this forum and unfortunately do not have a lot of time to research the archives on whether some of the concerns that I have with the general CEE direction have been previously discussed.
But first let me introduce myself. I am a dev manager at Red Hat responsible of IPA (Identity, Policy, Audit)(www.freeipa.org). I joined the project a little bit less than 3 years ago. At that time we were seriously thinking about doing something in the audit space. Now with IPA we mostly focusing on I & P with the plans to eventually come back to the Audit part some day. But while we were looking at the audit we have done some evaluation of the IDMEF, XDAS and other standards and came to pretty much same conclusions as the CEE team came - i.e. there is no good "usable" standard that people across the board will adopt.
As a result we started looking at different alternatives and innovating as a result we started an ELAPI (https://fedorahosted.org/ELAPI/) project that I dedicated quite some time. About a year ago however we realized that we do not have time and resources to continue this project so the code and ideas are dusting in vain. And this is what I wanted to share with.

First of all there have been quite a lot of effort, thoughts and energy has been put into this code. It is definitely not in any way complete but may be usable for the effort like CEE.

Here are several considerations that came to mind when I read the CEE Architectural overview document:
1) The idea of the dictionary has a big flaw in the use case of the log archival. Think about the server that have been collecting logs for years and then needs to spit back and interpret some data that has been collected several years ago. If interpretation of every event is relied on a dictionary and fixed specifications then such server needs to be aware of what version of the dictionary was used by the application at the moment the event was logged so either each event needs to carry a tag that identifies a dictionary version and server has to have all the versions of the dictionary ever used handy or the events should be more self interpreting and dictionary can just be used as a hint. We decided that it is better to not rely on the dictionary but rather to go with the following model:
Application would use an interface that would treat any even as a collection of the key-value pairs. The application would enter data in the format it has the data in (string, binary, boolean etc) but the interface will translate it to the internal representation which is a string using a provided format specifier (see examples https://fedorahosted.org/ELAPI/browser/elapi_ut.c). The interpretation on the server side is then always based on the names of the keys and if the server for whatever reason can't interpret the value of the event field (though it can use some heuristics based on a dictionary) it can at least handle it as a sting and try do searches and pattern matching against it.
From the document it was not clear how enforcing is the dictionary.
2) The idea of the CELR is great! But it should be guiding but not enforcing. The event profile definitions should be treated as a very valuable and quite reliable hint but a server should not require these definitions to process data.
3) The logging interface should support both asynchronous or synchronous operations.  In some cases the logging should be a best effort operation and having a fail over capability that ELAPI provides is sufficient so asynchronous call is best, but in some cases the event should be not only synchronous but with guarantied end to end delivery. The problem with delivery is that there are all sorts of errors that can happen and it is hard to report error back if the message was not logged on the server because, for example, it ran out of disk space. How you carry this back to the application and how you allow the application to decide what to do (retry, fail over, stop) ? The idea was that the event can be amended on the fly by any party that is responsible for delivery of the message to carry additional data about the delivery status. This way the error handling and communicating errors back and forth can be simplified and streamlined.
4) Another idea was to be able to have a set of the standardized fields that are known and required and a way for the interface to automatically inject another field that would carry a signature of the values in the identified subset of fields creating a way to confirm that the critical data in the event has not been altered in transition.    
5) I see a lot of synergy between the CEE and ELAPI. I would to help is someone takes a deeper look at ELAPI and decides to reuse some of the ideas, approaches or technologies implemented there. Some implementation ideas are still in my head and being realistic I acknowledge that the probability that I would every return to this project is very close to 0. But I would love to share and may be they will benefit CEE in any way. I am in greater Boston area so the team that works on this project is Bedford based I can come and give a quick overview of ELAPI.  
 
Sorry for a long post.

Thank you,
Dmitri Pal
Reply | Threaded
Open this post in threaded view
|

Re: Comments from a newbie

dpal
It looks like I created a topic on the wrong level. Sorry for that. Please move it under the discussion umbrella.

Thanks
Dmitri