Quantcast

Suggestion for an abstract profile type

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Suggestion for an abstract profile type

Keith Robertson
List,

I've been reviewing the CEE schemas and I would like to suggest the
inclusion of an abstract type for creating vendor specific extensions
(see [1] for and example).  This type would make it extremely easy for
binders to handle events with extensions that they are not familiar
with.  Furthermore, it makes it easy to compile new extensions into a
corpus of known ones.

How would you use it?
- Notice that the <event/> type in the CEE XSD has a <native/> element.  
This <native/> element contains a sequence of any types [2] which
basically means that you can insert whatever you want into a <native/>
element.  I recommend that elements placed into this sequence should be
of some abstract base type (e.g. <BaseProfileType/>).

Why is this important?
- If all profiles extend an abstract base type you will be able to use
run-time typing and inheritance to determine what is being supplied by
the provider.  Further, this ensures that event parsers need not be
aware of *every* profile extension, and that they can provide minimal
support for profiles that they have never seen before.

Cheers,
Keith

[1 ]
     <xs:complexType name="BaseProfileType">
         <xs:sequence>
         <!-- The URL that points to the schema which defines the
profile. -->
             <xs:element name="schema" minOccurs="0" maxOccurs="1"
type="xs:anyURI" >
                 <xs:annotation>
                     <xs:documentation>
                     As a convenience to event consumers, it would be
considered a best practice to
                     provide a URL to the the schema that describes the
structured profile.
                     </xs:documentation>
                 </xs:annotation></xs:element>
             <xs:element name="ver"    minOccurs="0" maxOccurs="1"
type="xs:string" />
         </xs:sequence>
     </xs:complexType>
     <xs:element abstract="true" name="BaseProfile"
type="BaseProfileType" />

[2]
<xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Suggestion for an abstract profile type

Wunder, John A.
Hi Keith,

There's actually a very related topic I wanted to bring up. What you suggest below is one option for vendor or community-created profiles: all fields that are NOT in the CEE field dictionary and taxonomy go inside the <native> element. As you say this makes it easy for consumers to understand vendor profiles based on the schema namespace and strong typing.

The other option, which is what CEE 1.0-beta1 currently implements, is that as long as a vendor or community provides a profile their fields go directly inside the CEE <event> element. The only fields that go in <native> are fields that are not defined in ANY profile. The main advantage to this approach is that I don't think we'll ever achieve significant coverage of event fields in CEE Core Profile; but the CEE Profile Repository WILL have significant coverage.

On the consumer side, they would need to retrieve the schema and validate the <event> element against whichever schema every time though (or, rather than validating against the schema they could just pull the list of element/values and discard the ones they don't understand. So it's definitely a less XML-y validation approach.

Personally I prefer that approach because I see most of the value of CEE in coming from communities of interest publishing their own profiles so I don't see why we would want to give Core Profile special standing. But I can see for ease of implementation why you would want ONLY the fields from Core Profile directly in the <event> element. It just feels wrong philosophically to me.

Thoughts?

(Sorry to hijack your thread, but I think your proposal depends on switching the way CEE 1.0-beta1 does that so wanted to clear this up first).

John

>-----Original Message-----
>From: Keith Robertson [mailto:[hidden email]]
>Sent: Wednesday, September 26, 2012 2:22 PM
>To: cee-discussion-list CEE-Related Discussion
>Subject: [CEE-DISCUSSION-LIST] Suggestion for an abstract profile type
>
>List,
>
>I've been reviewing the CEE schemas and I would like to suggest the
>inclusion of an abstract type for creating vendor specific extensions
>(see [1] for and example).  This type would make it extremely easy for
>binders to handle events with extensions that they are not familiar
>with.  Furthermore, it makes it easy to compile new extensions into a
>corpus of known ones.
>
>How would you use it?
>- Notice that the <event/> type in the CEE XSD has a <native/> element.
>This <native/> element contains a sequence of any types [2] which
>basically means that you can insert whatever you want into a <native/>
>element.  I recommend that elements placed into this sequence should be
>of some abstract base type (e.g. <BaseProfileType/>).
>
>Why is this important?
>- If all profiles extend an abstract base type you will be able to use
>run-time typing and inheritance to determine what is being supplied by
>the provider.  Further, this ensures that event parsers need not be
>aware of *every* profile extension, and that they can provide minimal
>support for profiles that they have never seen before.
>
>Cheers,
>Keith
>
>[1 ]
>     <xs:complexType name="BaseProfileType">
>         <xs:sequence>
>         <!-- The URL that points to the schema which defines the
>profile. -->
>             <xs:element name="schema" minOccurs="0" maxOccurs="1"
>type="xs:anyURI" >
>                 <xs:annotation>
>                     <xs:documentation>
>                     As a convenience to event consumers, it would be
>considered a best practice to
>                     provide a URL to the the schema that describes the
>structured profile.
>                     </xs:documentation>
>                 </xs:annotation></xs:element>
>             <xs:element name="ver"    minOccurs="0" maxOccurs="1"
>type="xs:string" />
>         </xs:sequence>
>     </xs:complexType>
>     <xs:element abstract="true" name="BaseProfile"
>type="BaseProfileType" />
>
>[2]
><xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Suggestion for an abstract profile type

Keith Robertson
On 10/03/2012 03:10 PM, Wunder, John A. wrote:
> Hi Keith,
>
> There's actually a very related topic I wanted to bring up. What you suggest below is one option for vendor or community-created profiles: all fields that are NOT in the CEE field dictionary and taxonomy go inside the <native> element. As you say this makes it easy for consumers to understand vendor profiles based on the schema namespace and strong typing.
>
> The other option, which is what CEE 1.0-beta1 currently implements, is that as long as a vendor or community provides a profile their fields go directly inside the CEE <event> element.
So are you suggesting that vendors/app developers should provide
profiles which would then get added to the
'http://cee.mitre.org/1.0/profile' namespace? That whole process might
be a bit cumbersome for some application developers particularly those
in the open source community.  This could be a barrier to adoption IMHO.

Another problem that I see with this approach is that it means that log
parsers will constantly have to phone home to the 'repository' to look
for descriptions to event objects that they have never seen before.  
Consider the case where log analyzer company X ships their log analyzer
with built in support for the core profile and the N profiles in the CEE
repository on ship date.  When the log analyzer application encounters
an element not in the core profile or in the N compiled profiles it is
basically stuck because the event object doesn't provide any additional
meta-data about it's source.

>   The only fields that go in <native> are fields that are not defined in ANY profile. The main advantage to this approach is that I don't think we'll ever achieve significant coverage of event fields in CEE Core Profile; but the CEE Profile Repository WILL have significant coverage.
Not sure I totally agree with this.  I think I'd like to see a situation
where those wishing to provide custom extensions can submit their
profile to a repository but are not required to. However, all extensions
should derive from base type X so that some minimum set of meta-data is
conveyed.

>
> On the consumer side, they would need to retrieve the schema and validate the <event> element against whichever schema every time though (or, rather than validating against the schema they could just pull the list of element/values and discard the ones they don't understand. So it's definitely a less XML-y validation approach.
>
> Personally I prefer that approach because I see most of the value of CEE in coming from communities of interest publishing their own profiles so I don't see why we would want to give Core Profile special standing. But I can see for ease of implementation why you would want ONLY the fields from Core Profile directly in the <event> element. It just feels wrong philosophically to me.
I am not trying to enforce a policy where only 'Core' elements may be
placed into the <event/> element, rather to ensure that elements placed
into the <event/> element convey enough meta-data so that a parser can
easily determine what type of object they are receiving.

>
> Thoughts?
>
> (Sorry to hijack your thread, but I think your proposal depends on switching the way CEE 1.0-beta1 does that so wanted to clear this up first).
>
> John
>
>> -----Original Message-----
>> From: Keith Robertson [mailto:[hidden email]]
>> Sent: Wednesday, September 26, 2012 2:22 PM
>> To: cee-discussion-list CEE-Related Discussion
>> Subject: [CEE-DISCUSSION-LIST] Suggestion for an abstract profile type
>>
>> List,
>>
>> I've been reviewing the CEE schemas and I would like to suggest the
>> inclusion of an abstract type for creating vendor specific extensions
>> (see [1] for and example).  This type would make it extremely easy for
>> binders to handle events with extensions that they are not familiar
>> with.  Furthermore, it makes it easy to compile new extensions into a
>> corpus of known ones.
>>
>> How would you use it?
>> - Notice that the <event/> type in the CEE XSD has a <native/> element.
>> This <native/> element contains a sequence of any types [2] which
>> basically means that you can insert whatever you want into a <native/>
>> element.  I recommend that elements placed into this sequence should be
>> of some abstract base type (e.g. <BaseProfileType/>).
>>
>> Why is this important?
>> - If all profiles extend an abstract base type you will be able to use
>> run-time typing and inheritance to determine what is being supplied by
>> the provider.  Further, this ensures that event parsers need not be
>> aware of *every* profile extension, and that they can provide minimal
>> support for profiles that they have never seen before.
>>
>> Cheers,
>> Keith
>>
>> [1 ]
>>      <xs:complexType name="BaseProfileType">
>>          <xs:sequence>
>>          <!-- The URL that points to the schema which defines the
>> profile. -->
>>              <xs:element name="schema" minOccurs="0" maxOccurs="1"
>> type="xs:anyURI" >
>>                  <xs:annotation>
>>                      <xs:documentation>
>>                      As a convenience to event consumers, it would be
>> considered a best practice to
>>                      provide a URL to the the schema that describes the
>> structured profile.
>>                      </xs:documentation>
>>                  </xs:annotation></xs:element>
>>              <xs:element name="ver"    minOccurs="0" maxOccurs="1"
>> type="xs:string" />
>>          </xs:sequence>
>>      </xs:complexType>
>>      <xs:element abstract="true" name="BaseProfile"
>> type="BaseProfileType" />
>>
>> [2]
>> <xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Suggestion for an abstract profile type

dpal
On 10/04/2012 01:04 AM, Keith Robertson wrote:

> On 10/03/2012 03:10 PM, Wunder, John A. wrote:
>> Hi Keith,
>>
>> There's actually a very related topic I wanted to bring up. What you
>> suggest below is one option for vendor or community-created profiles:
>> all fields that are NOT in the CEE field dictionary and taxonomy go
>> inside the <native> element. As you say this makes it easy for
>> consumers to understand vendor profiles based on the schema namespace
>> and strong typing.
>>
>> The other option, which is what CEE 1.0-beta1 currently implements,
>> is that as long as a vendor or community provides a profile their
>> fields go directly inside the CEE <event> element.
> So are you suggesting that vendors/app developers should provide
> profiles which would then get added to the
> 'http://cee.mitre.org/1.0/profile' namespace? That whole process might
> be a bit cumbersome for some application developers particularly those
> in the open source community.  This could be a barrier to adoption IMHO.

+1. It should be possible to have project or vendor repositories rather
than one single central one. The central one should act as a catalog of
the others though. LDAP referrals come to mind as a technology analogy.

>
> Another problem that I see with this approach is that it means that
> log parsers will constantly have to phone home to the 'repository' to
> look for descriptions to event objects that they have never seen
> before.  Consider the case where log analyzer company X ships their
> log analyzer with built in support for the core profile and the N
> profiles in the CEE repository on ship date.  When the log analyzer
> application encounters an element not in the core profile or in the N
> compiled profiles it is basically stuck because the event object
> doesn't provide any additional meta-data about it's source.

The definitions can be cached. This reminds me of the anti-virus
software: use definitions you know and check for new ones periodically
based on the configuration. Same logic can apply here.


>
>>   The only fields that go in <native> are fields that are not defined
>> in ANY profile. The main advantage to this approach is that I don't
>> think we'll ever achieve significant coverage of event fields in CEE
>> Core Profile; but the CEE Profile Repository WILL have significant
>> coverage.
> Not sure I totally agree with this.  I think I'd like to see a
> situation where those wishing to provide custom extensions can submit
> their profile to a repository but are not required to. However, all
> extensions should derive from base type X so that some minimum set of
> meta-data is conveyed.
>
>>
>> On the consumer side, they would need to retrieve the schema and
>> validate the <event> element against whichever schema every time
>> though (or, rather than validating against the schema they could just
>> pull the list of element/values and discard the ones they don't
>> understand. So it's definitely a less XML-y validation approach.
>>
>> Personally I prefer that approach because I see most of the value of
>> CEE in coming from communities of interest publishing their own
>> profiles so I don't see why we would want to give Core Profile
>> special standing. But I can see for ease of implementation why you
>> would want ONLY the fields from Core Profile directly in the <event>
>> element. It just feels wrong philosophically to me.
> I am not trying to enforce a policy where only 'Core' elements may be
> placed into the <event/> element, rather to ensure that elements
> placed into the <event/> element convey enough meta-data so that a
> parser can easily determine what type of object they are receiving.
>>
>> Thoughts?
>>
>> (Sorry to hijack your thread, but I think your proposal depends on
>> switching the way CEE 1.0-beta1 does that so wanted to clear this up
>> first).
>>
>> John
>>
>>> -----Original Message-----
>>> From: Keith Robertson [mailto:[hidden email]]
>>> Sent: Wednesday, September 26, 2012 2:22 PM
>>> To: cee-discussion-list CEE-Related Discussion
>>> Subject: [CEE-DISCUSSION-LIST] Suggestion for an abstract profile type
>>>
>>> List,
>>>
>>> I've been reviewing the CEE schemas and I would like to suggest the
>>> inclusion of an abstract type for creating vendor specific extensions
>>> (see [1] for and example).  This type would make it extremely easy for
>>> binders to handle events with extensions that they are not familiar
>>> with.  Furthermore, it makes it easy to compile new extensions into a
>>> corpus of known ones.
>>>
>>> How would you use it?
>>> - Notice that the <event/> type in the CEE XSD has a <native/> element.
>>> This <native/> element contains a sequence of any types [2] which
>>> basically means that you can insert whatever you want into a <native/>
>>> element.  I recommend that elements placed into this sequence should be
>>> of some abstract base type (e.g. <BaseProfileType/>).
>>>
>>> Why is this important?
>>> - If all profiles extend an abstract base type you will be able to use
>>> run-time typing and inheritance to determine what is being supplied by
>>> the provider.  Further, this ensures that event parsers need not be
>>> aware of *every* profile extension, and that they can provide minimal
>>> support for profiles that they have never seen before.
>>>
>>> Cheers,
>>> Keith
>>>
>>> [1 ]
>>>      <xs:complexType name="BaseProfileType">
>>>          <xs:sequence>
>>>          <!-- The URL that points to the schema which defines the
>>> profile. -->
>>>              <xs:element name="schema" minOccurs="0" maxOccurs="1"
>>> type="xs:anyURI" >
>>>                  <xs:annotation>
>>>                      <xs:documentation>
>>>                      As a convenience to event consumers, it would be
>>> considered a best practice to
>>>                      provide a URL to the the schema that describes the
>>> structured profile.
>>>                      </xs:documentation>
>>>                  </xs:annotation></xs:element>
>>>              <xs:element name="ver"    minOccurs="0" maxOccurs="1"
>>> type="xs:string" />
>>>          </xs:sequence>
>>>      </xs:complexType>
>>>      <xs:element abstract="true" name="BaseProfile"
>>> type="BaseProfileType" />
>>>
>>> [2]
>>> <xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />


--
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Suggestion for an abstract profile type

Fletcher, Boyd C IV Mr CIV OSD
In reply to this post by Keith Robertson
Wouldn't that require the SIEM vendors to have to have all the schemas
loaded and potentially write new code for each new device that decides to
declare its own types?







On 10/4/12 1:04 AM, "Keith Robertson" <[hidden email]> wrote:

>On 10/03/2012 03:10 PM, Wunder, John A. wrote:
>> Hi Keith,
>>
>> There's actually a very related topic I wanted to bring up. What you
>>suggest below is one option for vendor or community-created profiles:
>>all fields that are NOT in the CEE field dictionary and taxonomy go
>>inside the <native> element. As you say this makes it easy for consumers
>>to understand vendor profiles based on the schema namespace and strong
>>typing.
>>
>> The other option, which is what CEE 1.0-beta1 currently implements, is
>>that as long as a vendor or community provides a profile their fields go
>>directly inside the CEE <event> element.
>So are you suggesting that vendors/app developers should provide
>profiles which would then get added to the
>'http://cee.mitre.org/1.0/profile' namespace? That whole process might
>be a bit cumbersome for some application developers particularly those
>in the open source community.  This could be a barrier to adoption IMHO.
>
>Another problem that I see with this approach is that it means that log
>parsers will constantly have to phone home to the 'repository' to look
>for descriptions to event objects that they have never seen before.
>Consider the case where log analyzer company X ships their log analyzer
>with built in support for the core profile and the N profiles in the CEE
>repository on ship date.  When the log analyzer application encounters
>an element not in the core profile or in the N compiled profiles it is
>basically stuck because the event object doesn't provide any additional
>meta-data about it's source.
>
>>   The only fields that go in <native> are fields that are not defined
>>in ANY profile. The main advantage to this approach is that I don't
>>think we'll ever achieve significant coverage of event fields in CEE
>>Core Profile; but the CEE Profile Repository WILL have significant
>>coverage.
>Not sure I totally agree with this.  I think I'd like to see a situation
>where those wishing to provide custom extensions can submit their
>profile to a repository but are not required to. However, all extensions
>should derive from base type X so that some minimum set of meta-data is
>conveyed.
>
>>
>> On the consumer side, they would need to retrieve the schema and
>>validate the <event> element against whichever schema every time though
>>(or, rather than validating against the schema they could just pull the
>>list of element/values and discard the ones they don't understand. So
>>it's definitely a less XML-y validation approach.
>>
>> Personally I prefer that approach because I see most of the value of
>>CEE in coming from communities of interest publishing their own profiles
>>so I don't see why we would want to give Core Profile special standing.
>>But I can see for ease of implementation why you would want ONLY the
>>fields from Core Profile directly in the <event> element. It just feels
>>wrong philosophically to me.
>I am not trying to enforce a policy where only 'Core' elements may be
>placed into the <event/> element, rather to ensure that elements placed
>into the <event/> element convey enough meta-data so that a parser can
>easily determine what type of object they are receiving.
>>
>> Thoughts?
>>
>> (Sorry to hijack your thread, but I think your proposal depends on
>>switching the way CEE 1.0-beta1 does that so wanted to clear this up
>>first).
>>
>> John
>>
>>> -----Original Message-----
>>> From: Keith Robertson [mailto:[hidden email]]
>>> Sent: Wednesday, September 26, 2012 2:22 PM
>>> To: cee-discussion-list CEE-Related Discussion
>>> Subject: [CEE-DISCUSSION-LIST] Suggestion for an abstract profile type
>>>
>>> List,
>>>
>>> I've been reviewing the CEE schemas and I would like to suggest the
>>> inclusion of an abstract type for creating vendor specific extensions
>>> (see [1] for and example).  This type would make it extremely easy for
>>> binders to handle events with extensions that they are not familiar
>>> with.  Furthermore, it makes it easy to compile new extensions into a
>>> corpus of known ones.
>>>
>>> How would you use it?
>>> - Notice that the <event/> type in the CEE XSD has a <native/> element.
>>> This <native/> element contains a sequence of any types [2] which
>>> basically means that you can insert whatever you want into a <native/>
>>> element.  I recommend that elements placed into this sequence should be
>>> of some abstract base type (e.g. <BaseProfileType/>).
>>>
>>> Why is this important?
>>> - If all profiles extend an abstract base type you will be able to use
>>> run-time typing and inheritance to determine what is being supplied by
>>> the provider.  Further, this ensures that event parsers need not be
>>> aware of *every* profile extension, and that they can provide minimal
>>> support for profiles that they have never seen before.
>>>
>>> Cheers,
>>> Keith
>>>
>>> [1 ]
>>>      <xs:complexType name="BaseProfileType">
>>>          <xs:sequence>
>>>          <!-- The URL that points to the schema which defines the
>>> profile. -->
>>>              <xs:element name="schema" minOccurs="0" maxOccurs="1"
>>> type="xs:anyURI" >
>>>                  <xs:annotation>
>>>                      <xs:documentation>
>>>                      As a convenience to event consumers, it would be
>>> considered a best practice to
>>>                      provide a URL to the the schema that describes the
>>> structured profile.
>>>                      </xs:documentation>
>>>                  </xs:annotation></xs:element>
>>>              <xs:element name="ver"    minOccurs="0" maxOccurs="1"
>>> type="xs:string" />
>>>          </xs:sequence>
>>>      </xs:complexType>
>>>      <xs:element abstract="true" name="BaseProfile"
>>> type="BaseProfileType" />
>>>
>>> [2]
>>> <xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
Loading...