Taxonomy talk

classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Raffael Marty-3
I like it. I think the taxonomy should encode this notion in the status.
--
   Raffael Marty
   Chief Security Strategist                           @ Splunk>
   Security Visualization: http://secviz.org       raffy.ch/blog



On Dec 8, 2008, at 7:09 AM, David Corlette wrote:

> OK, I think we understand each other then.  I guess I would argue  
> that even though these two classes of events might represent  
> different semantic levels, I don't see any reason why they couldn't  
> be contained within a single data model.  That data model might  
> include optional fields like "accuracy" which are only relevant to  
> certain types of message.  I guess we'll see if this works.
>
>
>>>> On 06.12.2008 at 19:31, in message
> <[hidden email]
> v.microsoft.com>, Eric Fitzgerald <[hidden email]>  
> wrote:
>> No, I've made no such assumption.  Let me restate my argument.
> --SNIP--
>> I would love to have this discussion in person as it really is an
>> information theory discussion.  My argument is that events from use  
>> case #2
>> occupy a different semantic level than events from use case #1. ...
>>
>> -----Original Message-----
>> From: David Corlette [mailto:[hidden email]]
>> Sent: Saturday, December 06, 2008 12:12 PM
>> To: [hidden email]
>> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>>
>> Eric, I think you've made an assumption about where the event is  
>> generated,
>> which is why the initiator/target/observer separation is important  
>> in XDAS.
>> I agree that IF the observer (that generates the event) is ALSO the  
>> target
>> (affected by the event), then we can be sure that it's 100%  
>> accurate in
>> reporting the effects of the activity.  But there are plenty of  
>> cases where
>> the observer is NOT the target, like as you point out with IDSs and  
>> other
>> types of systems.  In those cases there's always a risk that the  
>> observed
>> behavior is not what actually happened.  But I don't see those as a  
>> separate
>> class of events.
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Eric Fitzgerald
Status and probability/accuracy seem like different concepts to me...

-----Original Message-----
From: Raffael Marty [mailto:[hidden email]]
Sent: Wednesday, December 17, 2008 5:27 PM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

I like it. I think the taxonomy should encode this notion in the status.
--
   Raffael Marty
   Chief Security Strategist                           @ Splunk>
   Security Visualization: http://secviz.org       raffy.ch/blog



On Dec 8, 2008, at 7:09 AM, David Corlette wrote:

> OK, I think we understand each other then.  I guess I would argue
> that even though these two classes of events might represent
> different semantic levels, I don't see any reason why they couldn't
> be contained within a single data model.  That data model might
> include optional fields like "accuracy" which are only relevant to
> certain types of message.  I guess we'll see if this works.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Raffael Marty-3
Yeah...

However, if you have a success ... you could encode the accuracy in  
there.

How I did that in the past is that I said it was an attempt. I haven't  
completely thought this through. I think you have a point. Maybe it's  
an additional taxonomy field. Not quite sure. Maybe it's part of the  
device type taxonomy field (OASD). (See some other emails where I  
argue that we should probably have a device type). Based on the type  
of device you can make a determination of the accuracy.

   -raffy

--
   Raffael Marty
   Chief Security Strategist                           @ Splunk>
   Security Visualization: http://secviz.org       raffy.ch/blog



On Dec 17, 2008, at 5:31 PM, Eric Fitzgerald wrote:

> Status and probability/accuracy seem like different concepts to me...
>
> -----Original Message-----
> From: Raffael Marty [mailto:[hidden email]]
> Sent: Wednesday, December 17, 2008 5:27 PM
> To: [hidden email]
> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>
> I like it. I think the taxonomy should encode this notion in the  
> status.
> --
>   Raffael Marty
>   Chief Security Strategist                           @ Splunk>
>   Security Visualization: http://secviz.org       raffy.ch/blog
>
>
>
> On Dec 8, 2008, at 7:09 AM, David Corlette wrote:
>
>> OK, I think we understand each other then.  I guess I would argue
>> that even though these two classes of events might represent
>> different semantic levels, I don't see any reason why they couldn't
>> be contained within a single data model.  That data model might
>> include optional fields like "accuracy" which are only relevant to
>> certain types of message.  I guess we'll see if this works.
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Sanford Whitehouse-2
In reply to this post by Eric Fitzgerald
To me as well.

Sanford

Eric Fitzgerald wrote:

> Status and probability/accuracy seem like different concepts to me...
>
> -----Original Message-----
> From: Raffael Marty [mailto:[hidden email]]
> Sent: Wednesday, December 17, 2008 5:27 PM
> To: [hidden email]
> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>
> I like it. I think the taxonomy should encode this notion in the status.
> --
>    Raffael Marty
>    Chief Security Strategist                           @ Splunk>
>    Security Visualization: http://secviz.org       raffy.ch/blog
>
>
>
> On Dec 8, 2008, at 7:09 AM, David Corlette wrote:
>
>> OK, I think we understand each other then.  I guess I would argue
>> that even though these two classes of events might represent
>> different semantic levels, I don't see any reason why they couldn't
>> be contained within a single data model.  That data model might
>> include optional fields like "accuracy" which are only relevant to
>> certain types of message.  I guess we'll see if this works.
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Wolfkiel, Joseph
Just as an FYI, the device assessment model we're developing in the DoD
contains an optional attribute called "confidence", which is a decimal value
between 0 and 1 to represent an assigned probability that the sensor output
is correct (on a per-element basis).  This seems roughly analogous to the
probability/accuracy value you're discussing.

If you add a similar construct to CEE, it would be useful if it is
consistent.

Our full set of status attributes include a timestamp for when an assessment
was taken, status (true, false, error, unknown, not applicable, not
evaluated), confidence, source (the sensor that did the assessment), and
checkRef (the criteria/signature whose evaluation resulted in the finding).

These attributes are bundled into an XML attribute group and associated with
findings from assessment tools.  So far, this attribute group seems to be
sufficient for most of our implementations.

We'll be working through several operational implementations that use this
status group.  Issues we haven't fully nailed down include assignment of
defaults, inheritance by lower-level elements, and aggregation of multiple
sensor inputs with differing confidence values for the same finding.

Lt Col Joseph L. Wolfkiel
Director, Computer Network Defense Research & Technology (CND R&T) Program
Management Office
9800 Savage Rd Ste 6767
Ft Meade, MD 20755-6767
Commercial 410-854-5401 DSN 244-5401
Fax 410-854-6700

-----Original Message-----
From: Sanford Whitehouse [mailto:[hidden email]]
Sent: Saturday, December 20, 2008 10:18 PM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

To me as well.

Sanford

Eric Fitzgerald wrote:

> Status and probability/accuracy seem like different concepts to me...
>
> -----Original Message-----
> From: Raffael Marty [mailto:[hidden email]]
> Sent: Wednesday, December 17, 2008 5:27 PM
> To: [hidden email]
> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>
> I like it. I think the taxonomy should encode this notion in the status.
> --
>    Raffael Marty
>    Chief Security Strategist                           @ Splunk>
>    Security Visualization: http://secviz.org       raffy.ch/blog
>
>
>
> On Dec 8, 2008, at 7:09 AM, David Corlette wrote:
>
>> OK, I think we understand each other then.  I guess I would argue
>> that even though these two classes of events might represent
>> different semantic levels, I don't see any reason why they couldn't
>> be contained within a single data model.  That data model might
>> include optional fields like "accuracy" which are only relevant to
>> certain types of message.  I guess we'll see if this works.
>>
>>
>

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Eric Fitzgerald
Actually I like the term "confidence" very much for the property we are discussing, and I also like expressing it as a numeric probability between 0 and 1.
 
You define confidence as the probability tha the sensor output is correct (I agree with this btw).  It is my belief that this only applies to certain classes of sensors, however, namely sensors that process some kind of "raw" data and issue output that is really opinions or conclusions drawn based on analysis of the input data.  For example, an IDS takes various kinds of data as input, performs some kind of analysis on the data, and then issues outputs of some form (which may be similar in form [e.g. event-like] to the input data) but which are conclusions of the analysis performed.  Is this correct?
 
The reason I ask is because I am trying to determine whether confidence broadly applies to all events (should debug/trace events have a confidence field) or whether we have a separate class of events to which this property uniquely applies.  Understanding the answer to this allows us to associate this property with the right set of events in the taxonomy.
 
Confidence aggregation is a hard problem- I worked on this problem recently.  I'd love to talk with you about it sometime.
 
Best regards,
Eric
 
 

From: Wolfkiel, Joseph [[hidden email]]
Sent: Monday, December 22, 2008 5:59 AM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

Just as an FYI, the device assessment model we're developing in the DoD
contains an optional attribute called "confidence", which is a decimal value
between 0 and 1 to represent an assigned probability that the sensor output
is correct (on a per-element basis).  This seems roughly analogous to the
probability/accuracy value you're discussing.

If you add a similar construct to CEE, it would be useful if it is
consistent. 

Our full set of status attributes include a timestamp for when an assessment
was taken, status (true, false, error, unknown, not applicable, not
evaluated), confidence, source (the sensor that did the assessment), and
checkRef (the criteria/signature whose evaluation resulted in the finding).

These attributes are bundled into an XML attribute group and associated with
findings from assessment tools.  So far, this attribute group seems to be
sufficient for most of our implementations.

We'll be working through several operational implementations that use this
status group.  Issues we haven't fully nailed down include assignment of
defaults, inheritance by lower-level elements, and aggregation of multiple
sensor inputs with differing confidence values for the same finding.

Director, Computer Network Defense Research & Technology (CND R&T) Program
Management Office 
9800 Savage Rd Ste 6767
Ft Meade, MD 20755-6767
Commercial 410-854-5401 DSN 244-5401
Fax 410-854-6700 

-----Original Message-----
From: Sanford Whitehouse [mailto:[hidden email]] 
Sent: Saturday, December 20, 2008 10:18 PM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

To me as well.

Sanford

Eric Fitzgerald wrote:
> Status and probability/accuracy seem like different concepts to me...
> 
> -----Original Message-----
> From: Raffael Marty [mailto:[hidden email]]
> Sent: Wednesday, December 17, 2008 5:27 PM
> To: [hidden email]
> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
> 
> I like it. I think the taxonomy should encode this notion in the status.
> --
>    Raffael Marty
>    Chief Security Strategist                           @ Splunk>
>    Security Visualization: http://secviz.org       raffy.ch/blog
> 
> 
> 
> On Dec 8, 2008, at 7:09 AM, David Corlette wrote:
> 
>> OK, I think we understand each other then.  I guess I would argue 
>> that even though these two classes of events might represent 
>> different semantic levels, I don't see any reason why they couldn't 
>> be contained within a single data model.  That data model might 
>> include optional fields like "accuracy" which are only relevant to 
>> certain types of message.  I guess we'll see if this works.
>>
>>
> 
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Wolfkiel, Joseph
In general, I expect to apply a confidence value to all events.  I've been thinking through some of the basic assumptions we make about how confidence is assigned.  Originally, I just set the confidence value default to 1.0 on the assumption that anything that didn't provide a confidence value was most likely a "direct observer" of an event or condition of some kind, which should just report "full" confidence in that sensor's results. 
 
However, on further thought, the sensor we are "most" confident of would generally be an agent-based sensor watching activities or assessing conditions on a device, assuming it was, in-turn, monitored by a TPM.  A sensor that isn't monitored by a TPM would merit lower confidence.  An external sensor would merit lower confidence still, with the exception of assessed activities or conditions that are best measured externally (e.g. is the device REALLY exposing a service on a particular port & protocol -- a process on the box might report it is, but it may actually be blocked by a host-based firewall).  Additionally, different signatures/checkRefs will have different confidences associated with them based on the signature/checkRef provider's assessment of accuracy.  Finally, values provided by humans probably merit different values than those assessed by automated sensors.
 
In implementation, I'm thinking that, on a per-sensor basis, we need to provide a max confidence value for each sensor, a default confidence value per sensor, and the ability to carry individual confidence values for each element reported.
 
Based on all the different places and rationales where confidence values may be different, we provided confidence values for pretty much all reported data elements.  We're still working through what implementation will look like.  Navy Research Lab is on the hook to figure out how to aggregate and correlate collections of multiple sensor outputs with varying confidence levels into a comprehensive network inventory.  This is a different use case than what I expect for CEE, but the parallels for confidence value usage are fairly strong (IMO).
 

Lt Col Joseph L. Wolfkiel
Director, Computer Network Defense Research & Technology (CND R&T) Program Management Office
9800 Savage Rd Ste 6767
Ft Meade, MD 20755-6767
Commercial 410-854-5401 DSN 244-5401
Fax 410-854-6700

 


From: Eric Fitzgerald [mailto:[hidden email]]
Sent: Tuesday, December 23, 2008 2:38 AM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

Actually I like the term "confidence" very much for the property we are discussing, and I also like expressing it as a numeric probability between 0 and 1.
 
You define confidence as the probability tha the sensor output is correct (I agree with this btw).  It is my belief that this only applies to certain classes of sensors, however, namely sensors that process some kind of "raw" data and issue output that is really opinions or conclusions drawn based on analysis of the input data.  For example, an IDS takes various kinds of data as input, performs some kind of analysis on the data, and then issues outputs of some form (which may be similar in form [e.g. event-like] to the input data) but which are conclusions of the analysis performed.  Is this correct?
 
The reason I ask is because I am trying to determine whether confidence broadly applies to all events (should debug/trace events have a confidence field) or whether we have a separate class of events to which this property uniquely applies.  Understanding the answer to this allows us to associate this property with the right set of events in the taxonomy.
 
Confidence aggregation is a hard problem- I worked on this problem recently.  I'd love to talk with you about it sometime.
 
Best regards,
Eric
 
 

From: Wolfkiel, Joseph [[hidden email]]
Sent: Monday, December 22, 2008 5:59 AM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

Just as an FYI, the device assessment model we're developing in the DoD
contains an optional attribute called "confidence", which is a decimal value
between 0 and 1 to represent an assigned probability that the sensor output
is correct (on a per-element basis).  This seems roughly analogous to the
probability/accuracy value you're discussing.

If you add a similar construct to CEE, it would be useful if it is
consistent. 

Our full set of status attributes include a timestamp for when an assessment
was taken, status (true, false, error, unknown, not applicable, not
evaluated), confidence, source (the sensor that did the assessment), and
checkRef (the criteria/signature whose evaluation resulted in the finding).

These attributes are bundled into an XML attribute group and associated with
findings from assessment tools.  So far, this attribute group seems to be
sufficient for most of our implementations.

We'll be working through several operational implementations that use this
status group.  Issues we haven't fully nailed down include assignment of
defaults, inheritance by lower-level elements, and aggregation of multiple
sensor inputs with differing confidence values for the same finding.

Director, Computer Network Defense Research & Technology (CND R&T) Program
Management Office 
9800 Savage Rd Ste 6767
Ft Meade, MD 20755-6767
Commercial 410-854-5401 DSN 244-5401
Fax 410-854-6700 

-----Original Message-----
From: Sanford Whitehouse [mailto:[hidden email]] 
Sent: Saturday, December 20, 2008 10:18 PM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

To me as well.

Sanford

Eric Fitzgerald wrote:
> Status and probability/accuracy seem like different concepts to me...
> 
> -----Original Message-----
> From: Raffael Marty [mailto:[hidden email]]
> Sent: Wednesday, December 17, 2008 5:27 PM
> To: [hidden email]
> Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
> 
> I like it. I think the taxonomy should encode this notion in the status.
> --
>    Raffael Marty
>    Chief Security Strategist                           @ Splunk>
>    Security Visualization: http://secviz.org       raffy.ch/blog
> 
> 
> 
> On Dec 8, 2008, at 7:09 AM, David Corlette wrote:
> 
>> OK, I think we understand each other then.  I guess I would argue 
>> that even though these two classes of events might represent 
>> different semantic levels, I don't see any reason why they couldn't 
>> be contained within a single data model.  That data model might 
>> include optional fields like "accuracy" which are only relevant to 
>> certain types of message.  I guess we'll see if this works.
>>
>>
> 

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

David Corlette
Based on all this discussion, I'd like to suggest the thought that evaluation of confidence lies outside the scope of the raw events themselves, but instead in the domain of event analysis.  Maintaining a database of which sensors are "reliable" is not something you would do on each sensor, but something you would add to the event or use to evaluate the event later.  

The actual use case (1) where you would apply a confidence at the source seems to me to only be relevant for a specific type of system, namely ones where you are applying some sort of analysis of network traffic or data, based on fuzzy rules, and producing an evaluation of results.  This is a quite limited use case.

All the other use cases (2) for confidence that have been suggested fall more into the category of "asset classification" meaning something you would keep in a separate database and use as input for event evaluation.  Namely, something a SIEM tool would do.

So - does splitting the world into these two types of confidence help anyone?  I would think that only use case (1) is relevant for CEE, as (2) is on the consumer side.


>>> On 23.12.2008 at 07:24, in message
<[hidden email]>,
"Wolfkiel, Joseph" <[hidden email]> wrote:

> In implementation, I'm thinking that, on a per-sensor basis, we need to
> provide a max confidence value for each sensor, a default confidence value
> per sensor, and the ability to carry individual confidence values for each
> element reported.

> From: Eric Fitzgerald [mailto:[hidden email]]
> The reason I ask is because I am trying to determine whether confidence
> broadly applies to all events (should debug/trace events have a confidence
> field) or whether we have a separate class of events to which this property
> uniquely applies.  Understanding the answer to this allows us to associate
> this property with the right set of events in the taxonomy.
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

Wolfkiel, Joseph
I agree.  Right now discussions on confidence are probably an unnecessary
distraction for CEE.

For our configuration assessment models, our first step was to define the
data elements we want to capture.  The confidence values were added later.
Since the assessment models are all in XML, adding confidence was just a
matter of adding the replication attribute groups to classes and elements as
appropriate.

It makes sense that CEE should focus first on what elements it wants to
collect, what they should be called, how they should be packaged, and (where
possible) what enumerated lists should be used.


Lt Col Joseph L. Wolfkiel
Director, Computer Network Defense Research & Technology (CND R&T) Program
Management Office
9800 Savage Rd Ste 6767
Ft Meade, MD 20755-6767
Commercial 410-854-5401 DSN 244-5401
Fax 410-854-6700

-----Original Message-----
From: David Corlette [mailto:[hidden email]]
Sent: Tuesday, December 23, 2008 7:36 AM
To: [hidden email]
Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk

Based on all this discussion, I'd like to suggest the thought that
evaluation of confidence lies outside the scope of the raw events
themselves, but instead in the domain of event analysis.  Maintaining a
database of which sensors are "reliable" is not something you would do on
each sensor, but something you would add to the event or use to evaluate the
event later.  

The actual use case (1) where you would apply a confidence at the source
seems to me to only be relevant for a specific type of system, namely ones
where you are applying some sort of analysis of network traffic or data,
based on fuzzy rules, and producing an evaluation of results.  This is a
quite limited use case.

All the other use cases (2) for confidence that have been suggested fall
more into the category of "asset classification" meaning something you would
keep in a separate database and use as input for event evaluation.  Namely,
something a SIEM tool would do.

So - does splitting the world into these two types of confidence help
anyone?  I would think that only use case (1) is relevant for CEE, as (2) is
on the consumer side.


>>> On 23.12.2008 at 07:24, in message
<[hidden email]>,
"Wolfkiel, Joseph" <[hidden email]> wrote:

> In implementation, I'm thinking that, on a per-sensor basis, we need
> to provide a max confidence value for each sensor, a default
> confidence value per sensor, and the ability to carry individual
> confidence values for each element reported.

> From: Eric Fitzgerald [mailto:[hidden email]]
> The reason I ask is because I am trying to determine whether
> confidence broadly applies to all events (should debug/trace events
> have a confidence
> field) or whether we have a separate class of events to which this
> property uniquely applies.  Understanding the answer to this allows us
> to associate this property with the right set of events in the taxonomy.

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Taxonomy talk

heinbockel
I see "confidence" as more as an event data dictionary attribute
over a taxon.

However, I do agree that this issue does not need to be addressed
immediately.
Let us return to the previous discussion of the taxonomy...

I've seen several different proposals, along with pros and cons.
Generally, the discussions have been surrounding a more
"limited language" (aka, object-action-status-devicetype) approach
versus a more enumerated taxonomy of event types
(or, as Eric suggested, a combined taxonomy allowing for
both approaches)


Following a similar line of logic as Sanford, and thinking as
if I were a developer wanting to specify an event in terms of
a taxonomy, I'd start with a higher order classification --
say looking for a "firewall" taxon if the event was related
to typical firewall functionality, or looking for a "network"
event category. I'd expect there to be at least some notional
hierarchies to navigate down.

With an enumerate taxonomy, this hierarchy seems to be more
straightforward. With the OASD approach, I guess I'd expect
to be able to start at the "object" or "devicetype" and work
my way into related taxons.


That said, I think the only approach to that will yield the best
results is to consider both approaches and work through some
sample devices and events...
Though the more I think about it, I believe that the two approaches
will end up merging -- some sort of hierarchical OASD?


William Heinbockel
The MITRE Corporation


>-----Original Message-----
>From: Wolfkiel, Joseph [mailto:[hidden email]]
>Sent: Tuesday, 23 December 2008 09:02
>To: cee-discussion-list CEE-Related Discussion
>Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>
>I agree.  Right now discussions on confidence are probably an
>unnecessary
>distraction for CEE.
>
>For our configuration assessment models, our first step was to
>define the
>data elements we want to capture.  The confidence values were
>added later.
>Since the assessment models are all in XML, adding confidence was
>just a
>matter of adding the replication attribute groups to classes and
>elements as
>appropriate.
>
>It makes sense that CEE should focus first on what elements it
>wants to
>collect, what they should be called, how they should be packaged,
>and (where
>possible) what enumerated lists should be used.
>
>
>Lt Col Joseph L. Wolfkiel
>Director, Computer Network Defense Research & Technology (CND R&T)
>Program
>Management Office
>9800 Savage Rd Ste 6767
>Ft Meade, MD 20755-6767
>Commercial 410-854-5401 DSN 244-5401
>Fax 410-854-6700
>
>-----Original Message-----
>From: David Corlette [mailto:[hidden email]]
>Sent: Tuesday, December 23, 2008 7:36 AM
>To: [hidden email]
>Subject: Re: [CEE-DISCUSSION-LIST] Taxonomy talk
>
>Based on all this discussion, I'd like to suggest the thought that
>evaluation of confidence lies outside the scope of the raw events
>themselves, but instead in the domain of event analysis.
>Maintaining a
>database of which sensors are "reliable" is not something you
>would do on
>each sensor, but something you would add to the event or use to
>evaluate the
>event later.
>
>The actual use case (1) where you would apply a confidence at the
>source
>seems to me to only be relevant for a specific type of system,
>namely ones
>where you are applying some sort of analysis of network traffic or
>data,
>based on fuzzy rules, and producing an evaluation of results.
>This is a
>quite limited use case.
>
>All the other use cases (2) for confidence that have been
>suggested fall
>more into the category of "asset classification" meaning something
>you would
>keep in a separate database and use as input for event evaluation.
>Namely,
>something a SIEM tool would do.
>
>So - does splitting the world into these two types of confidence
>help
>anyone?  I would think that only use case (1) is relevant for CEE,
>as (2) is
>on the consumer side.
>
>
>>>> On 23.12.2008 at 07:24, in message
><E065F6D82DAA2346B865ACFE1400156212F833@MSIS-FNX-
>UEA01.corp.nsa.gov>,
>"Wolfkiel, Joseph" <[hidden email]> wrote:
>
>> In implementation, I'm thinking that, on a per-sensor basis, we
>need
>> to provide a max confidence value for each sensor, a default
>> confidence value per sensor, and the ability to carry individual
>> confidence values for each element reported.
>
>> From: Eric Fitzgerald [mailto:[hidden email]]
>> The reason I ask is because I am trying to determine whether
>> confidence broadly applies to all events (should debug/trace
>events
>> have a confidence
>> field) or whether we have a separate class of events to which
>this
>> property uniquely applies.  Understanding the answer to this
>allows us
>> to associate this property with the right set of events in the
>taxonomy.

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

How to sell common event standards

Dan Blum
In reply to this post by Raffael Marty-3
I wrote a blog entry on this last week called: "How to plug common event and log standards."

Here's a taste of it that comes out of an email I actually sent to someone.

"As you point out, standards for events and logs are not urgent. We could go on generating a tower of logging Babel in IT forever. We could continue to have expensive SIEM that doesn't get much done. That doesn't interoperate across organizations. That limits the control over cloud computing and other outsourced environments. We could continue having lousy metrics and being mostly unable to automate security responses because we can't understand the full context of events. We could continue having mediocre security."

Then it goes on to talk about Steven Covey's time management framework, the difference between what's "urgent" and what's "important." For the full meal, go to: http://security-architect.blogspot.com/2009/01/how-to-plug-common-event-and-log.html

Also want to thank many of the folks on this list who provided feedback and review to my document "Leveraging Event and Log Information: A Strong Case for Standards."

Best regards,
Dan Blum
Burton Group
12