Guidance on using STIX for event data.

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Guidance on using STIX for event data.

Johnny Vestergaard
Hi,

I am currently investigating using STIX to facilitate sharing data
from a wide range of honeypots.
One fact i find confusing is how one is expected to nest the different
types of observables, indicators, incidents, etc.

The use case i am currently investigating:
We have a attack on a Glastopf honeypot and want to share attack data
using STIX. We have the following information available: raw http
request, attackers ip, attackers source port and attack type (SQLI,
LFI, RFI, etc).

How would one describe this using the eight core STIX constructs? My
initial take would be something like: create 2 observables (http
session and network activity) put these inside a Indicator(but which
type?), the indicator goes inside a Incident and finally the Incident
get's wrapped in a STIX package. But somehow it does not feel quite
right.

Any hints or guidance would be highly appreciated.

regards,
Johnny Vestergaard
The Honeynet Project
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

pmaroney
Johnny,

Sean Barnum typically provides the best substantive and informed recommendations/responses to these types of questions.  I would defer to him for "authoritative" response.

However, given no one has yet responded, I think there are a number of ways you could represent the data you outlined below.  There is a great deal of flexibility.

Along with the methods you describe (passing an Incident report within the package, passing the indicators within this report, or passing them as separate elements, you could describe the context of the package within a STIX header, etc. and pass the data in a table using a number of XML forms.  You could also pass the data as json, in a std CSV file, etc. though these would seem inelgant when more descriptive forms are available in the lexicon of the STIX/Cybox schemas.  I think a lot depends on the easiest way for you to package it up while retaining/conveying as much context as possible.  You can pass IOCs individually, in a list (e.g. a list of 'attacker ip"), etc.

So in this case, it appears to be a "flat table" with rows of IOCs and categorization values describing "Attack Type".   If you were trying to convey additional context for how the items in the table relate to other "things", "entities", "actions", etc. then the choice of form becomes much  more important.  For example bad guy "Bubba Burgers" sent this stream of targeted emails to these employee roles and that package contained an exploit targeting CVE nnnn , 'attackers ip" is registered to "Bubba" and was used in these 'campaigns'...

I still struggle with a lot of the concepts when it gets down to actually tackling a specific use case.  The wide range of flexibility in expressiveness and representation is a double edged sword.

At the end of the day it's most important that we have a common understanding within a given "Community of Trust" of the data representation.

Discussions like this help us all.

Patrick Maroney
Executive Director
Defense Security Information Exchange (DSIE)
Office: (856)983-0001
Cell: (609)841-5104
[hidden email]
________________________________________
From: [hidden email] <[hidden email]> on behalf of Johnny Vestergaard <[hidden email]>
Sent: Saturday, November 23, 2013 12:21:31 PM
To: [hidden email]
Subject: Guidance on using STIX for event data.

Hi,

I am currently investigating using STIX to facilitate sharing data
from a wide range of honeypots.
One fact i find confusing is how one is expected to nest the different
types of observables, indicators, incidents, etc.

The use case i am currently investigating:
We have a attack on a Glastopf honeypot and want to share attack data
using STIX. We have the following information available: raw http
request, attackers ip, attackers source port and attack type (SQLI,
LFI, RFI, etc).

How would one describe this using the eight core STIX constructs? My
initial take would be something like: create 2 observables (http
session and network activity) put these inside a Indicator(but which
type?), the indicator goes inside a Incident and finally the Incident
get's wrapped in a STIX package. But somehow it does not feel quite
right.

Any hints or guidance would be highly appreciated.

regards,
Johnny Vestergaard
The Honeynet Project
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
In reply to this post by Johnny Vestergaard
Hi,

> The use case i am currently investigating:
> We have a attack on a Glastopf honeypot and want to share attack data
> using STIX. We have the following information available: raw http
> request, attackers ip, attackers source port and attack type (SQLI,
> LFI, RFI, etc).
>
> How would one describe this using the eight core STIX constructs? My

We are currently contemplating a similar use case. At the moment, we
tend towards leaving out the 'Indicator' construct here, since in our
mind the Indicator is something that (usually) takes some human
work of assessment, analysis, etc. Whereas here, you want to
communicate stuff observed in an automated way by a honeypot.
So the top-level element would be an 'Incident' with 'Related Observables'
that describe what has been observed and possibly a link to 'Leveraged TTP'
for communicating the findings about attack type. That all is
then wrapped into a STIX Package -- you then have to decide
whether you want to put just a single incident into the package,
or wrap several incidents -- for your use-case, probably one
incident per STIX package is more natural?

> initial take would be something like: create 2 observables (http
> session and network activity) put these inside a Indicator(but which
                                          ^^^^^^
Note that whenever you can put something inside something else in
STIX, you can also make a reference to it... which may make things
easier for generation and processing. So, you could create a STIX package,
that lists one incident, a number of observables and TTPs, and have
the incident reference the observables and TTPs via the idref-attribute.

Regards,

Bernd, Siemens CERT
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Barnum, Sean D.
I love it when the community beats me to answering questions. This is the
sort of community collaboration we are aiming for and I am sure all of you
appreciate.

My short answer is that Bernd is correct, it depends on what you are
wanting to convey.
If you want to simply convey reports of activity that the honeypots are
seeing then this would like best be captured inside of Incidents. Each
activity event would have its own Incident. Each Incident would capture
what was seen inside the Incident/Related_Observables structure. Raw HTTP
request, attackers ip and attackers source port can all be captured within
a single instance of Network_Connection Object
(Source_Socket_Address/IP_Address, Source_Socket_Address/Port, and
Layer7_Connections/HTTP_Session). Each Incident could also convey the sort
of attack activity observed using the Leveraged_TTP structure with simple
Behavior/Attack_Patterns/Attack_Pattern/Description content naming the
attack type SQLI, LFI, RFI, etc. (you can also use the @capec_id attribute
to reference standardized descriptions of these sorts of attacks.

As Bernd describes, if you wanted to do more than simply report what was
seen but actually convey that this sort of activity is something you
should be looking out for then the activity could also be conveyed within
Indicators where each Indicator specifies the pattern of what to look for
(could use the same observable structure defined in the Incident instance
just with adding @condition attributes to each specified field) and then
specifying the attack type using the Indicated_TTP structure.

Bernd is also correct that you could do these as atomic units (an Incident
or an Indicator containing everything about it) or you could define each
Observable (either or potentially both the instances and patterns) under
the STIX_Package/Observables section and the TTP under the TTPS section
and then specify the Incidents and/or Indicators and simply use references
to the Observables and TTP defined at the global level. If you were
including both Incidents and Indicators this would, at the very least,
avoid you having to repeat the specification of TTP entries for each and
every Incident and Indicator. You would specify them once and then
reference them.

Does this make sense? And is it helpful?

sean

On 11/25/13 2:32 AM, "Grobauer, Bernd" <[hidden email]> wrote:

>Hi,
>
>> The use case i am currently investigating:
>> We have a attack on a Glastopf honeypot and want to share attack data
>> using STIX. We have the following information available: raw http
>> request, attackers ip, attackers source port and attack type (SQLI,
>> LFI, RFI, etc).
>>
>> How would one describe this using the eight core STIX constructs? My
>
>We are currently contemplating a similar use case. At the moment, we
>tend towards leaving out the 'Indicator' construct here, since in our
>mind the Indicator is something that (usually) takes some human
>work of assessment, analysis, etc. Whereas here, you want to
>communicate stuff observed in an automated way by a honeypot.
>So the top-level element would be an 'Incident' with 'Related Observables'
>that describe what has been observed and possibly a link to 'Leveraged
>TTP'
>for communicating the findings about attack type. That all is
>then wrapped into a STIX Package -- you then have to decide
>whether you want to put just a single incident into the package,
>or wrap several incidents -- for your use-case, probably one
>incident per STIX package is more natural?
>
>> initial take would be something like: create 2 observables (http
>> session and network activity) put these inside a Indicator(but which
>                                          ^^^^^^
>Note that whenever you can put something inside something else in
>STIX, you can also make a reference to it... which may make things
>easier for generation and processing. So, you could create a STIX package,
>that lists one incident, a number of observables and TTPs, and have
>the incident reference the observables and TTPs via the idref-attribute.
>
>Regards,
>
>Bernd, Siemens CERT
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
Hi,

> I love it when the community beats me to answering questions. This is
> the
> sort of community collaboration we are aiming for and I am sure all of
> you
> appreciate.
>
> My short answer is that Bernd is correct, it depends on what you are
> wanting to convey.

While we are talking about the usage of STIX to convey certain use cases,
let me take the opportunity to restate a question from beginning of October,
(see below) which unfortunately elicited no response on the list. Any kind of
response (if only that the use case is not well thought out and why) would
really be appreciated!

Kind regards,

Bernd

---------------


Hi,

when looking at the STIX "class" diagram or into the XML schema descriptions,
one sees that Campaigns and Incidents can reference related indicators,
but indicators cannot reference campaigns or incidents to which they 'belong'.
Below I talk about campaigns and indicators, but the same holds for incidents.

Since STIX is a transport format, shouldn't it be possible, to do something
to the effect of "Here I send you another indicator, which is associated with
campaign XYZ (provided by reference only) that I informed you about some time
ago. (And implicitly: if you don't have information about XYZ, use the TAXII query mechanism to
request it from me)."

Currently, the only way I see to inform about new indicators for an
existing campaign is to always send updates of an existing campaign
object/structure in which the "RelatedIndicators" element is updated: thus,
I keep resending the same information again and again just to communicate
a small increment regarding the indicators. Or, I 'hack' the format
by associating each campaign with an empty 'standard indicator' which I
then reference from new indicators via the "RelatedIndicators" element.

Apart from data efficiency, there is another aspect to this. Imagine,
you have a non-STIX-enabled contributor, whom you want to enable to
use STIX for some communicating with you in some well-defined
standard use-cases: in effect, you have to look at the data
this contributor produces and then provide
some kind of tooling into which he throws his data in some agreed format,
and the tooling takes care of producing the corresponding STIX data.
In this case, it would be a lot easier to agree on mechanisms to
communicate the base data about newly identified campaigns once
and then refer to that campaign in subsequent updates about incidents,
rather than making the tool always produce a campaign object with all
information in it: the latter would require much tighter integration of
the STIX-tooling into the contributor's tool/data landscape (which
is where we may want to end up anyways, but may not be able to
get things started with).  

Any thoughts?

Kind regards,

Bernd
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Barnum, Sean D.
Hey Bernd,

I apologize. I thought that I had replied to this but apparently it got
lost in the shuffle. Mea culpa.

The quick answer is that this issue is already on our radar and we have a
proposed change that moves the Campaign-Indicator relationship from the
Campaign structure to the Indicator structure and also enables tagging
Indicators with campaign names even before a formal Campaign structure
instance exists.
This proposal should be going out to the list within the next week or two.
I will see what I can do about maybe accelerating it a bit.

We are definitely interested in your thoughts on the proposed fix once you
have a chance to review the proposal.

sean

On 11/25/13 11:24 AM, "Grobauer, Bernd" <[hidden email]> wrote:

>Hi,
>
>> I love it when the community beats me to answering questions. This is
>> the
>> sort of community collaboration we are aiming for and I am sure all of
>> you
>> appreciate.
>>
>> My short answer is that Bernd is correct, it depends on what you are
>> wanting to convey.
>
>While we are talking about the usage of STIX to convey certain use cases,
>let me take the opportunity to restate a question from beginning of
>October,
>(see below) which unfortunately elicited no response on the list. Any
>kind of
>response (if only that the use case is not well thought out and why)
>would
>really be appreciated!
>
>Kind regards,
>
>Bernd
>
>---------------
>
>
>Hi,
>
>when looking at the STIX "class" diagram or into the XML schema
>descriptions,
>one sees that Campaigns and Incidents can reference related indicators,
>but indicators cannot reference campaigns or incidents to which they
>'belong'.
>Below I talk about campaigns and indicators, but the same holds for
>incidents.
>
>Since STIX is a transport format, shouldn't it be possible, to do
>something
>to the effect of "Here I send you another indicator, which is associated
>with
>campaign XYZ (provided by reference only) that I informed you about some
>time
>ago. (And implicitly: if you don't have information about XYZ, use the
>TAXII query mechanism to
>request it from me)."
>
>Currently, the only way I see to inform about new indicators for an
>existing campaign is to always send updates of an existing campaign
>object/structure in which the "RelatedIndicators" element is updated:
>thus,
>I keep resending the same information again and again just to communicate
>a small increment regarding the indicators. Or, I 'hack' the format
>by associating each campaign with an empty 'standard indicator' which I
>then reference from new indicators via the "RelatedIndicators" element.
>
>Apart from data efficiency, there is another aspect to this. Imagine,
>you have a non-STIX-enabled contributor, whom you want to enable to
>use STIX for some communicating with you in some well-defined
>standard use-cases: in effect, you have to look at the data
>this contributor produces and then provide
>some kind of tooling into which he throws his data in some agreed format,
>and the tooling takes care of producing the corresponding STIX data.
>In this case, it would be a lot easier to agree on mechanisms to
>communicate the base data about newly identified campaigns once
>and then refer to that campaign in subsequent updates about incidents,
>rather than making the tool always produce a campaign object with all
>information in it: the latter would require much tighter integration of
>the STIX-tooling into the contributor's tool/data landscape (which
>is where we may want to end up anyways, but may not be able to
>get things started with).
>
>Any thoughts?
>
>Kind regards,
>
>Bernd
>
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
Hi Sean,

thanks for your reply!

> I apologize. I thought that I had replied to this but apparently it got
> lost in the shuffle. Mea culpa.

Come to think of it: I think my question fell right into the US Government
shutdown, which must have also impacted you guys. Bad timing, I guess...
thanks for the prompt answer now!

>
> The quick answer is that this issue is already on our radar and we have
> a
> proposed change that moves the Campaign-Indicator relationship from the
> Campaign structure to the Indicator structure and also enables tagging

That sounds great! Will it be the same for the Incident-structure?

> Indicators with campaign names even before a formal Campaign structure
> instance exists.

Oh, tagging, well ... as I wrote to this list a while ago, I have issues
with tagging, because its implementation with unrestricted XPATH expressions
makes it rather hard for any implementation that does not use XML for
data storage to deal with tagging. But let's see.

> This proposal should be going out to the list within the next week or
> two.
> I will see what I can do about maybe accelerating it a bit.

So this change is scheduled for Stix 1.1?

Another point, since Sean in his reply to Johnny's also talked about
the possibility to define an object/element once and then reference it:
what *really* would make my day in a future iteration of Stix would
be to have the ability to add a timestamp an every place that can
have a 'id' attribute.  We have talked about this on the
list before and the answer from MITRE was that whenever a STIX/CybOX-element
changes, the identifier should also change -- old and new revision of
an object would then be linked with relationships.
Having progressed quite a bit in our internal STIX-implementation since then, I
am even more convinced: The relationship-based approach may be valuable
for special use cases in which elements are fused, derived from each other or
copied over in 'ownership' (as expressed via the identifier namespace)
from one organization to the other, etc.. BUT: the relationship-based approach
is absolutely impractical for revision management on elements/objects 'owned'
by a given organization (as expressed via the identifier namespace) and
managed within that organization. Such elements/objects need to be able
to retain their identifier. And I don't think I need a structure within the
object to track change information, I just need a single attribute that
allows me to record a time stamp for each object.
I think, we never reached common ground on this revision management issue ...
what do you think about the possibility for a timestamp attribute wherever
there is an id-attribute?

Kind regards,

Bernd
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Aharon
In reply to this post by Barnum, Sean D.
Sean, we are currently implementing the Campaign to Indicator structure. This structure more closely follows what usually happens manually today:

1) SOC identifies a pattern in indicators/observables
2) A new campaign is created
3) Indicators are referenced to the Campaign

While I can make this work the other way, where an indicator references a campaign, I had just convinced myself that the campaign to indicato references was the right way to go.

I'd be interested in hearing how this indicator campaign tagging is supposed to work. Why not reference a stub campaign that just has a title or a single name included in the names list?  Seems like we may be moving away from having Campaign as its own construct.

Aharon

DTCC Non-Confidential (White)
---------------------------------------------------
Michael "Aharon" Chernin
Security Automation
DTCC Tampa
[hidden email]
Office: 813-470-2173



> -----Original Message-----
> From: [hidden email] [mailto:owner-stix-
> [hidden email]] On Behalf Of Barnum, Sean D.
> Sent: Monday, November 25, 2013 12:07 PM
> To: Grobauer, Bernd; Johnny Vestergaard; stix-discussion-list Structured
> Threat Information Expression/ST
> Subject: Re: Guidance on using STIX for event data.
>
> Hey Bernd,
>
> I apologize. I thought that I had replied to this but apparently it got lost in the
> shuffle. Mea culpa.
>
> The quick answer is that this issue is already on our radar and we have a
> proposed change that moves the Campaign-Indicator relationship from the
> Campaign structure to the Indicator structure and also enables tagging
> Indicators with campaign names even before a formal Campaign structure
> instance exists.
> This proposal should be going out to the list within the next week or two.
> I will see what I can do about maybe accelerating it a bit.
>
> We are definitely interested in your thoughts on the proposed fix once you
> have a chance to review the proposal.
>
> sean
>
> On 11/25/13 11:24 AM, "Grobauer, Bernd" <[hidden email]>
> wrote:
>
> >Hi,
> >
> >> I love it when the community beats me to answering questions. This is
> >> the sort of community collaboration we are aiming for and I am sure
> >> all of you appreciate.
> >>
> >> My short answer is that Bernd is correct, it depends on what you are
> >> wanting to convey.
> >
> >While we are talking about the usage of STIX to convey certain use
> >cases, let me take the opportunity to restate a question from beginning
> >of October, (see below) which unfortunately elicited no response on the
> >list. Any kind of response (if only that the use case is not well
> >thought out and why) would really be appreciated!
> >
> >Kind regards,
> >
> >Bernd
> >
> >---------------
> >
> >
> >Hi,
> >
> >when looking at the STIX "class" diagram or into the XML schema
> >descriptions, one sees that Campaigns and Incidents can reference
> >related indicators, but indicators cannot reference campaigns or
> >incidents to which they 'belong'.
> >Below I talk about campaigns and indicators, but the same holds for
> >incidents.
> >
> >Since STIX is a transport format, shouldn't it be possible, to do
> >something to the effect of "Here I send you another indicator, which is
> >associated with campaign XYZ (provided by reference only) that I
> >informed you about some time ago. (And implicitly: if you don't have
> >information about XYZ, use the TAXII query mechanism to request it from
> >me)."
> >
> >Currently, the only way I see to inform about new indicators for an
> >existing campaign is to always send updates of an existing campaign
> >object/structure in which the "RelatedIndicators" element is updated:
> >thus,
> >I keep resending the same information again and again just to
> >communicate a small increment regarding the indicators. Or, I 'hack'
> >the format by associating each campaign with an empty 'standard
> >indicator' which I then reference from new indicators via the
> "RelatedIndicators" element.
> >
> >Apart from data efficiency, there is another aspect to this. Imagine,
> >you have a non-STIX-enabled contributor, whom you want to enable to use
> >STIX for some communicating with you in some well-defined standard
> >use-cases: in effect, you have to look at the data this contributor
> >produces and then provide some kind of tooling into which he throws his
> >data in some agreed format, and the tooling takes care of producing the
> >corresponding STIX data.
> >In this case, it would be a lot easier to agree on mechanisms to
> >communicate the base data about newly identified campaigns once and
> >then refer to that campaign in subsequent updates about incidents,
> >rather than making the tool always produce a campaign object with all
> >information in it: the latter would require much tighter integration of
> >the STIX-tooling into the contributor's tool/data landscape (which is
> >where we may want to end up anyways, but may not be able to get things
> >started with).
> >
> >Any thoughts?
> >
> >Kind regards,
> >
> >Bernd
> >
> >
> >
<BR>_____________________________________________________________
<FONT size=2><BR>
DTCC DISCLAIMER: This email and any files transmitted with it are
confidential and intended solely for the use of the individual or
entity to whom they are addressed. If you have received this email
in error, please notify us immediately and delete the email and any
attachments from your system. The recipient should check this email
and any attachments for the presence of viruses.  The company
accepts no liability for any damage caused by any virus transmitted
by this email.</FONT>
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Johnny Vestergaard
Great response, many thanks to all of you.

After this discussion I realise that i was not too far off with my
original plan in regards to how i would structure the document.

Bernd: Good point with the indicator and human involvement. But, i
guess that without an indicator we cannot give an attack pattern
either? (i could be wrong...). Actually Bernd, any chance that you
could post a sanitized version of one of your STIX incident reports?

Sean: Yes, more than happy, great response!

regards,
johnny


On Mon, Nov 25, 2013 at 8:24 PM, Chernin, Michael A. <[hidden email]> wrote:

> Sean, we are currently implementing the Campaign to Indicator structure. This structure more closely follows what usually happens manually today:
>
> 1) SOC identifies a pattern in indicators/observables
> 2) A new campaign is created
> 3) Indicators are referenced to the Campaign
>
> While I can make this work the other way, where an indicator references a campaign, I had just convinced myself that the campaign to indicato references was the right way to go.
>
> I'd be interested in hearing how this indicator campaign tagging is supposed to work. Why not reference a stub campaign that just has a title or a single name included in the names list?  Seems like we may be moving away from having Campaign as its own construct.
>
> Aharon
>
> DTCC Non-Confidential (White)
> ---------------------------------------------------
> Michael "Aharon" Chernin
> Security Automation
> DTCC Tampa
> [hidden email]
> Office: 813-470-2173
>
>
>
>> -----Original Message-----
>> From: [hidden email] [mailto:owner-stix-
>> [hidden email]] On Behalf Of Barnum, Sean D.
>> Sent: Monday, November 25, 2013 12:07 PM
>> To: Grobauer, Bernd; Johnny Vestergaard; stix-discussion-list Structured
>> Threat Information Expression/ST
>> Subject: Re: Guidance on using STIX for event data.
>>
>> Hey Bernd,
>>
>> I apologize. I thought that I had replied to this but apparently it got lost in the
>> shuffle. Mea culpa.
>>
>> The quick answer is that this issue is already on our radar and we have a
>> proposed change that moves the Campaign-Indicator relationship from the
>> Campaign structure to the Indicator structure and also enables tagging
>> Indicators with campaign names even before a formal Campaign structure
>> instance exists.
>> This proposal should be going out to the list within the next week or two.
>> I will see what I can do about maybe accelerating it a bit.
>>
>> We are definitely interested in your thoughts on the proposed fix once you
>> have a chance to review the proposal.
>>
>> sean
>>
>> On 11/25/13 11:24 AM, "Grobauer, Bernd" <[hidden email]>
>> wrote:
>>
>> >Hi,
>> >
>> >> I love it when the community beats me to answering questions. This is
>> >> the sort of community collaboration we are aiming for and I am sure
>> >> all of you appreciate.
>> >>
>> >> My short answer is that Bernd is correct, it depends on what you are
>> >> wanting to convey.
>> >
>> >While we are talking about the usage of STIX to convey certain use
>> >cases, let me take the opportunity to restate a question from beginning
>> >of October, (see below) which unfortunately elicited no response on the
>> >list. Any kind of response (if only that the use case is not well
>> >thought out and why) would really be appreciated!
>> >
>> >Kind regards,
>> >
>> >Bernd
>> >
>> >---------------
>> >
>> >
>> >Hi,
>> >
>> >when looking at the STIX "class" diagram or into the XML schema
>> >descriptions, one sees that Campaigns and Incidents can reference
>> >related indicators, but indicators cannot reference campaigns or
>> >incidents to which they 'belong'.
>> >Below I talk about campaigns and indicators, but the same holds for
>> >incidents.
>> >
>> >Since STIX is a transport format, shouldn't it be possible, to do
>> >something to the effect of "Here I send you another indicator, which is
>> >associated with campaign XYZ (provided by reference only) that I
>> >informed you about some time ago. (And implicitly: if you don't have
>> >information about XYZ, use the TAXII query mechanism to request it from
>> >me)."
>> >
>> >Currently, the only way I see to inform about new indicators for an
>> >existing campaign is to always send updates of an existing campaign
>> >object/structure in which the "RelatedIndicators" element is updated:
>> >thus,
>> >I keep resending the same information again and again just to
>> >communicate a small increment regarding the indicators. Or, I 'hack'
>> >the format by associating each campaign with an empty 'standard
>> >indicator' which I then reference from new indicators via the
>> "RelatedIndicators" element.
>> >
>> >Apart from data efficiency, there is another aspect to this. Imagine,
>> >you have a non-STIX-enabled contributor, whom you want to enable to use
>> >STIX for some communicating with you in some well-defined standard
>> >use-cases: in effect, you have to look at the data this contributor
>> >produces and then provide some kind of tooling into which he throws his
>> >data in some agreed format, and the tooling takes care of producing the
>> >corresponding STIX data.
>> >In this case, it would be a lot easier to agree on mechanisms to
>> >communicate the base data about newly identified campaigns once and
>> >then refer to that campaign in subsequent updates about incidents,
>> >rather than making the tool always produce a campaign object with all
>> >information in it: the latter would require much tighter integration of
>> >the STIX-tooling into the contributor's tool/data landscape (which is
>> >where we may want to end up anyways, but may not be able to get things
>> >started with).
>> >
>> >Any thoughts?
>> >
>> >Kind regards,
>> >
>> >Bernd
>> >
>> >
>> >
> <BR>_____________________________________________________________
> <FONT size=2><BR>
> DTCC DISCLAIMER: This email and any files transmitted with it are
> confidential and intended solely for the use of the individual or
> entity to whom they are addressed. If you have received this email
> in error, please notify us immediately and delete the email and any
> attachments from your system. The recipient should check this email
> and any attachments for the presence of viruses.  The company
> accepts no liability for any damage caused by any virus transmitted
> by this email.</FONT>
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
Hi Johnny,

please excuse the late reply!

> Bernd: Good point with the indicator and human involvement. But, i
> guess that without an indicator we cannot give an attack pattern

My understanding regarding attack patterns is that this would
be expressed as a "TTP" and associated with the Incident via
"Leveraged_TTP" -- I don't  think you need the Indicator object there.

> either? (i could be wrong...). Actually Bernd, any chance that you
> could post a sanitized version of one of your STIX incident reports?

I will send something to the list in the next few days.
But don't expect too much ... for example, we have not gone
into communicating attack patterns, for example...

Kind regards,

Bernd


</FONT>
Reply | Threaded
Open this post in threaded view
|

Actor / Campaign APIs for python-stix

Foley, Alexander - GIS
In reply to this post by Johnny Vestergaard
Hi,

After getting up to speed on the python-stix package, I wondered if any users in the community had been able to bolt additional APIs onto the package or if this was a development item on the roadmap?  I wasn't sure if there was an opportunity for members of the community to pitch in to help augment the codebase with additional APIs or if that was already being handled elsewhere.  In any case, thanks, and for those confused I was referring to the APIs documented here:

https://github.com/STIXProject/python-stix/wiki/APIs-vs.-Bindings
https://github.com/STIXProject/python-stix/wiki/API-Coverage

To those who celebrate it, Happy Thanksgiving!

Alex

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.
Reply | Threaded
Open this post in threaded view
|

RE: Actor / Campaign APIs for python-stix

Aharon
Happy Thanksgiving Alex. We have completed the python-stix APIs for actor and campaign as part of our implementation. It will be available to you through our package. We are trying to determine timing on when/if we will push back to Mitre's Github.

Aharon

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Foley, Alexander
Sent: Wednesday, November 27, 2013 5:20 PM
To: [hidden email]
Subject: Actor / Campaign APIs for python-stix

Hi,

After getting up to speed on the python-stix package, I wondered if any users in the community had been able to bolt additional APIs onto the package or if this was a development item on the roadmap?  I wasn't sure if there was an opportunity for members of the community to pitch in to help augment the codebase with additional APIs or if that was already being handled elsewhere.  In any case, thanks, and for those confused I was referring to the APIs documented here:

https://github.com/STIXProject/python-stix/wiki/APIs-vs.-Bindings
https://github.com/STIXProject/python-stix/wiki/API-Coverage

To those who celebrate it, Happy Thanksgiving!

Alex

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.
<BR>_____________________________________________________________
<FONT size=2><BR>
DTCC DISCLAIMER: This email and any files transmitted with it are
confidential and intended solely for the use of the individual or
entity to whom they are addressed. If you have received this email
in error, please notify us immediately and delete the email and any
attachments from your system. The recipient should check this email
and any attachments for the presence of viruses.  The company
accepts no liability for any damage caused by any virus transmitted
by this email.</FONT>
Reply | Threaded
Open this post in threaded view
|

Re: Actor / Campaign APIs for python-stix

PAT MARONEY-2
Aharon,
Is this available to us somewhere today?

Patrick Maroney
Cell: (609)841-5104


On Nov 28, 2013, at 8:36 AM, "Chernin, Michael A." <[hidden email]> wrote:

Happy Thanksgiving Alex. We have completed the python-stix APIs for actor and campaign as part of our implementation. It will be available to you through our package. We are trying to determine timing on when/if we will push back to Mitre's Github.

Aharon

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Foley, Alexander
Sent: Wednesday, November 27, 2013 5:20 PM
To: [hidden email]
Subject: Actor / Campaign APIs for python-stix

Hi,

After getting up to speed on the python-stix package, I wondered if any users in the community had been able to bolt additional APIs onto the package or if this was a development item on the roadmap?  I wasn't sure if there was an opportunity for members of the community to pitch in to help augment the codebase with additional APIs or if that was already being handled elsewhere.  In any case, thanks, and for those confused I was referring to the APIs documented here:

https://github.com/STIXProject/python-stix/wiki/APIs-vs.-Bindings
https://github.com/STIXProject/python-stix/wiki/API-Coverage

To those who celebrate it, Happy Thanksgiving!

Alex

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.
<BR>_____________________________________________________________
<FONT size=2><BR>
DTCC DISCLAIMER: This email and any files transmitted with it are
confidential and intended solely for the use of the individual or
entity to whom they are addressed. If you have received this email
in error, please notify us immediately and delete the email and any
attachments from your system. The recipient should check this email
and any attachments for the presence of viruses.  The company
accepts no liability for any damage caused by any virus transmitted
by this email.</FONT>
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Johnny Vestergaard
In reply to this post by Grobauer, Bernd
Hi Bernd,

On Wed, Nov 27, 2013 at 12:02 PM, Grobauer, Bernd
<[hidden email]> wrote:
>> Bernd: Good point with the indicator and human involvement. But, i
>> guess that without an indicator we cannot give an attack pattern
>
> My understanding regarding attack patterns is that this would
> be expressed as a "TTP" and associated with the Incident via
> "Leveraged_TTP" -- I don't  think you need the Indicator object there.

Good point, i did not notice leveraged_ttp until now.

>> either? (i could be wrong...). Actually Bernd, any chance that you
>> could post a sanitized version of one of your STIX incident reports?
>
> I will send something to the list in the next few days.
> But don't expect too much ... for example, we have not gone
> into communicating attack patterns, for example...

Sound great.

Btw, does anyone know if there is work being done adding more entities
to python-stix API?

regards,
Johnny
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
Hi Johnny,

here is the promised use case and some explanation regarding our
design decisions. Feedback by you or anybody else on why some
decisions might better be made differently (or, conversely, support for
the taken decisions) is most welcome.

First of all, the use-case, which is rather small: a submitter
has an IDS listening to DNS traffic and thus notices queries for
known malicious domains. The position of the IDS is such that
it cannot see the requesting clients -- just that malicious
domains were queried.

This information is to be shared with other parties, e.g. such
that are able to investigate further, block the domains, etc.

So, there is not much "hard" data to be shared, really:

- the observed queries
- the reason, why the queried domain is considered
  as malicious

Then there is timing info: when were the queries observed,
but we assume daily reports about these events, and for
the purpose of blocking the domains or further investigation
(e.g. in proxy logs), the information about the 24h slice
in which the requests were observed is sufficient ... and
this information is implicit; come to think of it, we could/should
add timing info in the incident's Time element about first
observation...

We thus want to communicate a single incident that reports
this information (one could also think about one incident
per observed domain, but let's go for one incident for now)
and asks for some course-of-action to be taken.

The whole XML in the attachment; let me go through some parts
of the XML in the email:

- We need to add our namespaces to the package::

    <stix:STIX_Package xmlns:some_cert="cert.some_organization.com" xmlns:submitter="... >

- Whenever a vocabulary is used, we add information, also if it is
  the default vocabulary::

        <stix:Package_Intent  xsi:type="stixVocabs:PackageIntentVocab-1.0">Incident</stix:Package_Intent>

- We try to use references to existing objects whereever possible and do not
  resend objects that are known to be known by the recipient. For example,
  the submitter knows that the recipient has the object describing its
  identity::

        <stix:Information_Source>
            <stixCommon:Identity idref="some_cert:identity-submitter.some.organization.com"/>
        </stix:Information_Source>

  Here we have the first occurrence of an identifier that does not use uuid. I am
  not sure how strict the STIX/Cybox specification is about the shape of identifiers:
  we have use-cases where we'd rather use something else than uuid: a meaningful
  identifier in this case, or a identifier generated by a digest (see below).

- We communicate the observables used in this STIX package and reference them
  later from incidents, indicators, ...::

    <stix:Observables cybox_major_version="2" cybox_minor_version="0" cybox_update_version="1">
        <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
            (...)
        </cybox:Observable>

        <cybox:Observable id="submitter:Observable-6e260711e503a9108541dfd13616ce97">
            (...)
        </cybox:Observable>
    </stix:Observables>

- Two specialities about the identifiers for observable and object embedded in the observable::

        <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
            <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">

  - The identifier of the Object is auto-generated by a
    JSON-2-STIX-converter taking a digest of the "basic facts" from
    which the object is created. We do this here for the following
    reason: the data that is converted to STIX is generated by a
    system that knows nothing of STIX, but simply pushes events in
    JSON-format to the JSON-2-STIX-converter. It is likely to
    push the same domain names again and again (at least for some days) ... and I do not
    want to have the same (very simple) DNS-Query-object under a dozen different
    identifiers from one and the same submitter. Since the JSON-2-STIX converter
    does not hold state between conversions, using a digest as identifier is
    the easiest way to avoid this.

  - Notice that we give an identifier to both the observable and the embedded object.
    The identifier on the observable is required  for referencing it; the identifier
    on the object is required, because our processing tool extracts cybox
    objects as separate entities with associated data type (a DNS-Query object, in this case).

- What to communicate as observable? The bare minimum would be an URI-Object that communicates
  the domain name; on the other end of the scale there would be an object expressing
  all facts about the observed network connection and the embedded protocol. Here,
  we stay closer to the bare minimum, but include the information that a DNS query
  was observed: thus we a have DNS-Query object rather than a URI-Object::

            <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
                <cybox:Properties xsi:type="DNSQueryObject:DNSQueryObjectType">
                    <DNSQueryObj:Question>
                        <DNSQueryObj:QName>
                            <URIObj:Value datatype="anyURI">evil.com</URIObj:Value>
                        </DNSQueryObj:QName>
                    </DNSQueryObj:Question>
                </cybox:Properties>
            </cybox:Object>
 
- We use the "Information Source" element of the relationship to communicate
  the reason why the observed  DNS query really constitutes an incident by
  providing a reference to the IDS rule that was triggered::

         <incident:Related_Observables scope="exclusive">
                <incident:Related_Observable>
                    <stixCommon:Information_Source>
                        <stixCommon:References>
                            <stixCommon:Reference>http://www.snort.org/search/sid/123456</stixCommon:Reference>
                        </stixCommon:References>
                    </stixCommon:Information_Source>
                    <stixCommon:Observable>
                        <cybox:Object idref="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0"/>
                    </stixCommon:Observable>
                </incident:Related_Observable>
               (...)

- Also with the requested course of action, we refer to an existing object known to
  all involved parties rather than resending the same COA object over and over::

     <incident:COA_Requested>
                <incident:Course_Of_Action idref="some_cert:COA-19b68437-6467-4b74-99a9-152c365fe6ad"/>
     </incident:COA_Requested>


So much for now -- I hope, this was helpful in some way!

Kind regards,

Bernd

----
Bernd Grobauer, Siemens CERT

ids_dns_usecase.xml (17K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

Grobauer, Bernd
In reply to this post by Johnny Vestergaard
Hi Johnny,

> Btw, does anyone know if there is work being done adding more entities
> to python-stix API?

We decided not to wait for the extension of the API and have written
code that generates STIX-XML from a Python-dictionary-representation
of STIX. Essentially 'to_dict'/'from_dict' as in the API, but
completely generic by leveraging the 'build'-functions of the
STIX/CybOX-bindings.

The input from which the XML included with my previous email
to this list is attached to this email.

Kind regards,

Bernd

---

Bernd Grobauer, Siemens CERT



stix_python_dict_repr.txt (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

pmaroney
In reply to this post by Grobauer, Bernd
Bernd,

A general comment on the use case:  collection and sharing of the answer(s) to the query, the time(s) of observations, and metadata like TTL, RRI, etc. can all be very valuable presuming one has structured their DNS mitigations to allow the capture of same while still mitigating the threat.   As the "natural" observer, one is the unique position to gather this data without OPSEC exposure.  For example if activity is the result of a targeted attack, adversary will expect activity from that organization.

Great discussions by the way.  We need more of these use cases and examples.

Patrick Maroney
Executive Director
Defense Security Information Exchange (DSIE)
Office: (856)983-0001
Cell: (609)841-5104
[hidden email]
________________________________________
From: [hidden email] <[hidden email]> on behalf of Grobauer, Bernd <[hidden email]>
Sent: Friday, November 29, 2013 8:40:39 AM
To: Johnny Vestergaard
Cc: stix-discussion-list Structured Threat Information Expression/ST
Subject: RE: Guidance on using STIX for event data.

Hi Johnny,

here is the promised use case and some explanation regarding our
design decisions. Feedback by you or anybody else on why some
decisions might better be made differently (or, conversely, support for
the taken decisions) is most welcome.

First of all, the use-case, which is rather small: a submitter
has an IDS listening to DNS traffic and thus notices queries for
known malicious domains. The position of the IDS is such that
it cannot see the requesting clients -- just that malicious
domains were queried.

This information is to be shared with other parties, e.g. such
that are able to investigate further, block the domains, etc.

So, there is not much "hard" data to be shared, really:

- the observed queries
- the reason, why the queried domain is considered
  as malicious

Then there is timing info: when were the queries observed,
but we assume daily reports about these events, and for
the purpose of blocking the domains or further investigation
(e.g. in proxy logs), the information about the 24h slice
in which the requests were observed is sufficient ... and
this information is implicit; come to think of it, we could/should
add timing info in the incident's Time element about first
observation...

We thus want to communicate a single incident that reports
this information (one could also think about one incident
per observed domain, but let's go for one incident for now)
and asks for some course-of-action to be taken.

The whole XML in the attachment; let me go through some parts
of the XML in the email:

- We need to add our namespaces to the package::

    <stix:STIX_Package xmlns:some_cert="cert.some_organization.com" xmlns:submitter="... >

- Whenever a vocabulary is used, we add information, also if it is
  the default vocabulary::

        <stix:Package_Intent  xsi:type="stixVocabs:PackageIntentVocab-1.0">Incident</stix:Package_Intent>

- We try to use references to existing objects whereever possible and do not
  resend objects that are known to be known by the recipient. For example,
  the submitter knows that the recipient has the object describing its
  identity::

        <stix:Information_Source>
            <stixCommon:Identity idref="some_cert:identity-submitter.some.organization.com"/>
        </stix:Information_Source>

  Here we have the first occurrence of an identifier that does not use uuid. I am
  not sure how strict the STIX/Cybox specification is about the shape of identifiers:
  we have use-cases where we'd rather use something else than uuid: a meaningful
  identifier in this case, or a identifier generated by a digest (see below).

- We communicate the observables used in this STIX package and reference them
  later from incidents, indicators, ...::

    <stix:Observables cybox_major_version="2" cybox_minor_version="0" cybox_update_version="1">
        <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
            (...)
        </cybox:Observable>

        <cybox:Observable id="submitter:Observable-6e260711e503a9108541dfd13616ce97">
            (...)
        </cybox:Observable>
    </stix:Observables>

- Two specialities about the identifiers for observable and object embedded in the observable::

        <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
            <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">

  - The identifier of the Object is auto-generated by a
    JSON-2-STIX-converter taking a digest of the "basic facts" from
    which the object is created. We do this here for the following
    reason: the data that is converted to STIX is generated by a
    system that knows nothing of STIX, but simply pushes events in
    JSON-format to the JSON-2-STIX-converter. It is likely to
    push the same domain names again and again (at least for some days) ... and I do not
    want to have the same (very simple) DNS-Query-object under a dozen different
    identifiers from one and the same submitter. Since the JSON-2-STIX converter
    does not hold state between conversions, using a digest as identifier is
    the easiest way to avoid this.

  - Notice that we give an identifier to both the observable and the embedded object.
    The identifier on the observable is required  for referencing it; the identifier
    on the object is required, because our processing tool extracts cybox
    objects as separate entities with associated data type (a DNS-Query object, in this case).

- What to communicate as observable? The bare minimum would be an URI-Object that communicates
  the domain name; on the other end of the scale there would be an object expressing
  all facts about the observed network connection and the embedded protocol. Here,
  we stay closer to the bare minimum, but include the information that a DNS query
  was observed: thus we a have DNS-Query object rather than a URI-Object::

            <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
                <cybox:Properties xsi:type="DNSQueryObject:DNSQueryObjectType">
                    <DNSQueryObj:Question>
                        <DNSQueryObj:QName>
                            <URIObj:Value datatype="anyURI">evil.com</URIObj:Value>
                        </DNSQueryObj:QName>
                    </DNSQueryObj:Question>
                </cybox:Properties>
            </cybox:Object>

- We use the "Information Source" element of the relationship to communicate
  the reason why the observed  DNS query really constitutes an incident by
  providing a reference to the IDS rule that was triggered::

         <incident:Related_Observables scope="exclusive">
                <incident:Related_Observable>
                    <stixCommon:Information_Source>
                        <stixCommon:References>
                            <stixCommon:Reference>http://www.snort.org/search/sid/123456</stixCommon:Reference>
                        </stixCommon:References>
                    </stixCommon:Information_Source>
                    <stixCommon:Observable>
                        <cybox:Object idref="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0"/>
                    </stixCommon:Observable>
                </incident:Related_Observable>
               (...)

- Also with the requested course of action, we refer to an existing object known to
  all involved parties rather than resending the same COA object over and over::

     <incident:COA_Requested>
                <incident:Course_Of_Action idref="some_cert:COA-19b68437-6467-4b74-99a9-152c365fe6ad"/>
     </incident:COA_Requested>


So much for now -- I hope, this was helpful in some way!

Kind regards,

Bernd

----
Bernd Grobauer, Siemens CERT
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Johnny Vestergaard
As a followup to our discussion, here is my current take on a very
basic STIX incident message for traffic to a ConPot honeypot:

https://gist.github.com/johnnykv/7725345

regards,
Johnny

On Fri, Nov 29, 2013 at 4:15 PM, Patrick Maroney <[hidden email]> wrote:

> Bernd,
>
> A general comment on the use case:  collection and sharing of the answer(s) to the query, the time(s) of observations, and metadata like TTL, RRI, etc. can all be very valuable presuming one has structured their DNS mitigations to allow the capture of same while still mitigating the threat.   As the "natural" observer, one is the unique position to gather this data without OPSEC exposure.  For example if activity is the result of a targeted attack, adversary will expect activity from that organization.
>
> Great discussions by the way.  We need more of these use cases and examples.
>
> Patrick Maroney
> Executive Director
> Defense Security Information Exchange (DSIE)
> Office: (856)983-0001
> Cell: (609)841-5104
> [hidden email]
> ________________________________________
> From: [hidden email] <[hidden email]> on behalf of Grobauer, Bernd <[hidden email]>
> Sent: Friday, November 29, 2013 8:40:39 AM
> To: Johnny Vestergaard
> Cc: stix-discussion-list Structured Threat Information Expression/ST
> Subject: RE: Guidance on using STIX for event data.
>
> Hi Johnny,
>
> here is the promised use case and some explanation regarding our
> design decisions. Feedback by you or anybody else on why some
> decisions might better be made differently (or, conversely, support for
> the taken decisions) is most welcome.
>
> First of all, the use-case, which is rather small: a submitter
> has an IDS listening to DNS traffic and thus notices queries for
> known malicious domains. The position of the IDS is such that
> it cannot see the requesting clients -- just that malicious
> domains were queried.
>
> This information is to be shared with other parties, e.g. such
> that are able to investigate further, block the domains, etc.
>
> So, there is not much "hard" data to be shared, really:
>
> - the observed queries
> - the reason, why the queried domain is considered
>   as malicious
>
> Then there is timing info: when were the queries observed,
> but we assume daily reports about these events, and for
> the purpose of blocking the domains or further investigation
> (e.g. in proxy logs), the information about the 24h slice
> in which the requests were observed is sufficient ... and
> this information is implicit; come to think of it, we could/should
> add timing info in the incident's Time element about first
> observation...
>
> We thus want to communicate a single incident that reports
> this information (one could also think about one incident
> per observed domain, but let's go for one incident for now)
> and asks for some course-of-action to be taken.
>
> The whole XML in the attachment; let me go through some parts
> of the XML in the email:
>
> - We need to add our namespaces to the package::
>
>     <stix:STIX_Package xmlns:some_cert="cert.some_organization.com" xmlns:submitter="... >
>
> - Whenever a vocabulary is used, we add information, also if it is
>   the default vocabulary::
>
>         <stix:Package_Intent  xsi:type="stixVocabs:PackageIntentVocab-1.0">Incident</stix:Package_Intent>
>
> - We try to use references to existing objects whereever possible and do not
>   resend objects that are known to be known by the recipient. For example,
>   the submitter knows that the recipient has the object describing its
>   identity::
>
>         <stix:Information_Source>
>             <stixCommon:Identity idref="some_cert:identity-submitter.some.organization.com"/>
>         </stix:Information_Source>
>
>   Here we have the first occurrence of an identifier that does not use uuid. I am
>   not sure how strict the STIX/Cybox specification is about the shape of identifiers:
>   we have use-cases where we'd rather use something else than uuid: a meaningful
>   identifier in this case, or a identifier generated by a digest (see below).
>
> - We communicate the observables used in this STIX package and reference them
>   later from incidents, indicators, ...::
>
>     <stix:Observables cybox_major_version="2" cybox_minor_version="0" cybox_update_version="1">
>         <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>             (...)
>         </cybox:Observable>
>
>         <cybox:Observable id="submitter:Observable-6e260711e503a9108541dfd13616ce97">
>             (...)
>         </cybox:Observable>
>     </stix:Observables>
>
> - Two specialities about the identifiers for observable and object embedded in the observable::
>
>         <cybox:Observable id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>             <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>
>   - The identifier of the Object is auto-generated by a
>     JSON-2-STIX-converter taking a digest of the "basic facts" from
>     which the object is created. We do this here for the following
>     reason: the data that is converted to STIX is generated by a
>     system that knows nothing of STIX, but simply pushes events in
>     JSON-format to the JSON-2-STIX-converter. It is likely to
>     push the same domain names again and again (at least for some days) ... and I do not
>     want to have the same (very simple) DNS-Query-object under a dozen different
>     identifiers from one and the same submitter. Since the JSON-2-STIX converter
>     does not hold state between conversions, using a digest as identifier is
>     the easiest way to avoid this.
>
>   - Notice that we give an identifier to both the observable and the embedded object.
>     The identifier on the observable is required  for referencing it; the identifier
>     on the object is required, because our processing tool extracts cybox
>     objects as separate entities with associated data type (a DNS-Query object, in this case).
>
> - What to communicate as observable? The bare minimum would be an URI-Object that communicates
>   the domain name; on the other end of the scale there would be an object expressing
>   all facts about the observed network connection and the embedded protocol. Here,
>   we stay closer to the bare minimum, but include the information that a DNS query
>   was observed: thus we a have DNS-Query object rather than a URI-Object::
>
>             <cybox:Object id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>                 <cybox:Properties xsi:type="DNSQueryObject:DNSQueryObjectType">
>                     <DNSQueryObj:Question>
>                         <DNSQueryObj:QName>
>                             <URIObj:Value datatype="anyURI">evil.com</URIObj:Value>
>                         </DNSQueryObj:QName>
>                     </DNSQueryObj:Question>
>                 </cybox:Properties>
>             </cybox:Object>
>
> - We use the "Information Source" element of the relationship to communicate
>   the reason why the observed  DNS query really constitutes an incident by
>   providing a reference to the IDS rule that was triggered::
>
>          <incident:Related_Observables scope="exclusive">
>                 <incident:Related_Observable>
>                     <stixCommon:Information_Source>
>                         <stixCommon:References>
>                             <stixCommon:Reference>http://www.snort.org/search/sid/123456</stixCommon:Reference>
>                         </stixCommon:References>
>                     </stixCommon:Information_Source>
>                     <stixCommon:Observable>
>                         <cybox:Object idref="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0"/>
>                     </stixCommon:Observable>
>                 </incident:Related_Observable>
>                (...)
>
> - Also with the requested course of action, we refer to an existing object known to
>   all involved parties rather than resending the same COA object over and over::
>
>      <incident:COA_Requested>
>                 <incident:Course_Of_Action idref="some_cert:COA-19b68437-6467-4b74-99a9-152c365fe6ad"/>
>      </incident:COA_Requested>
>
>
> So much for now -- I hope, this was helpful in some way!
>
> Kind regards,
>
> Bernd
>
> ----
> Bernd Grobauer, Siemens CERT
Reply | Threaded
Open this post in threaded view
|

RE: Guidance on using STIX for event data.

pmaroney
Johnny,

Thanks for sharing this mock schema .

Caveat: One reason for the 'Packet Rat' nickname is a ravenous appetite
for data, and in particular Packet Captures. In my former life we
collected ~800+ TBs of 7x24 packet captures/year.

Your illustrative example leaves me wanting more: Nature of the activity
(times, frequency, volume, method, etc.), impact, subjective analysis, and
most importantly any Raw Data (event logs, netflow, PACKETS!!) wherever
practical.

The "Who, What, When, Where, Why, and How" should be well summarized in
narrative sections of the STIX packages.   Even when we had to walk tens
of miles through the snow to get CSV files with lists of 100's  of
Indicators on a Floppy Disk, many now realize getting raw indicators is
not our main problem.  The timely and efficient inter-exchange of
actionable details on events and related observations is the real gap.

Tag Line for DHS/MITRE Marketing and Hallmark Card Campaigns:

    <<< Every STIX package should tell a story >>>


_____________________________
From: [hidden email]
<[hidden email]> on behalf of Johnny
Vestergaard <[hidden email]>
Sent: Saturday, November 30, 2013 5:25:57 PM
To: stix-discussion-list Structured Threat Information Expression/ST
Subject: Re: Guidance on using STIX for event data.

As a followup to our discussion, here is my current take on a very
basic STIX incident message for traffic to a ConPot honeypot:

https://gist.github.com/johnnykv/7725345

regards,
Johnny

On Fri, Nov 29, 2013 at 4:15 PM, Patrick Maroney <[hidden email]>
wrote:

> Bernd,
>
> A general comment on the use case:  collection and sharing of the
>answer(s) to the query, the time(s) of observations, and metadata like
>TTL, RRI, etc. can all be very valuable presuming one has structured
>their DNS mitigations to allow the capture of same while still mitigating
>the threat.   As the "natural" observer, one is the unique position to
>gather this data without OPSEC exposure.  For example if activity is the
>result of a targeted attack, adversary will expect activity from that
>organization.
>
> Great discussions by the way.  We need more of these use cases and
>examples.
>
> Patrick Maroney
> Executive Director
> Defense Security Information Exchange (DSIE)
> Office: (856)983-0001
> Cell: (609)841-5104
> [hidden email]
> ________________________________________
> From: [hidden email]
><[hidden email]> on behalf of Grobauer, Bernd
><[hidden email]>
> Sent: Friday, November 29, 2013 8:40:39 AM
> To: Johnny Vestergaard
> Cc: stix-discussion-list Structured Threat Information Expression/ST
> Subject: RE: Guidance on using STIX for event data.
>
> Hi Johnny,
>
> here is the promised use case and some explanation regarding our
> design decisions. Feedback by you or anybody else on why some
> decisions might better be made differently (or, conversely, support for
> the taken decisions) is most welcome.
>
> First of all, the use-case, which is rather small: a submitter
> has an IDS listening to DNS traffic and thus notices queries for
> known malicious domains. The position of the IDS is such that
> it cannot see the requesting clients -- just that malicious
> domains were queried.
>
> This information is to be shared with other parties, e.g. such
> that are able to investigate further, block the domains, etc.
>
> So, there is not much "hard" data to be shared, really:
>
> - the observed queries
> - the reason, why the queried domain is considered
>   as malicious
>
> Then there is timing info: when were the queries observed,
> but we assume daily reports about these events, and for
> the purpose of blocking the domains or further investigation
> (e.g. in proxy logs), the information about the 24h slice
> in which the requests were observed is sufficient ... and
> this information is implicit; come to think of it, we could/should
> add timing info in the incident's Time element about first
> observation...
>
> We thus want to communicate a single incident that reports
> this information (one could also think about one incident
> per observed domain, but let's go for one incident for now)
> and asks for some course-of-action to be taken.
>
> The whole XML in the attachment; let me go through some parts
> of the XML in the email:
>
> - We need to add our namespaces to the package::
>
>     <stix:STIX_Package xmlns:some_cert="cert.some_organization.com"
>xmlns:submitter="... >
>
> - Whenever a vocabulary is used, we add information, also if it is
>   the default vocabulary::
>
>         <stix:Package_Intent
>xsi:type="stixVocabs:PackageIntentVocab-1.0">Incident</stix:Package_Intent
>>
>
> - We try to use references to existing objects whereever possible and do
>not
>   resend objects that are known to be known by the recipient. For
>example,
>   the submitter knows that the recipient has the object describing its
>   identity::
>
>         <stix:Information_Source>
>             <stixCommon:Identity
>idref="some_cert:identity-submitter.some.organization.com"/>
>         </stix:Information_Source>
>
>   Here we have the first occurrence of an identifier that does not use
>uuid. I am
>   not sure how strict the STIX/Cybox specification is about the shape of
>identifiers:
>   we have use-cases where we'd rather use something else than uuid: a
>meaningful
>   identifier in this case, or a identifier generated by a digest (see
>below).
>
> - We communicate the observables used in this STIX package and reference
>them
>   later from incidents, indicators, ...::
>
>     <stix:Observables cybox_major_version="2" cybox_minor_version="0"
>cybox_update_version="1">
>         <cybox:Observable
>id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>             (...)
>         </cybox:Observable>
>
>         <cybox:Observable
>id="submitter:Observable-6e260711e503a9108541dfd13616ce97">
>             (...)
>         </cybox:Observable>
>     </stix:Observables>
>
> - Two specialities about the identifiers for observable and object
>embedded in the observable::
>
>         <cybox:Observable
>id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>             <cybox:Object
>id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>
>   - The identifier of the Object is auto-generated by a
>     JSON-2-STIX-converter taking a digest of the "basic facts" from
>     which the object is created. We do this here for the following
>     reason: the data that is converted to STIX is generated by a
>     system that knows nothing of STIX, but simply pushes events in
>     JSON-format to the JSON-2-STIX-converter. It is likely to
>     push the same domain names again and again (at least for some days)
>... and I do not
>     want to have the same (very simple) DNS-Query-object under a dozen
>different
>     identifiers from one and the same submitter. Since the JSON-2-STIX
>converter
>     does not hold state between conversions, using a digest as
>identifier is
>     the easiest way to avoid this.
>
>   - Notice that we give an identifier to both the observable and the
>embedded object.
>     The identifier on the observable is required  for referencing it;
>the identifier
>     on the object is required, because our processing tool extracts cybox
>     objects as separate entities with associated data type (a DNS-Query
>object, in this case).
>
> - What to communicate as observable? The bare minimum would be an
>URI-Object that communicates
>   the domain name; on the other end of the scale there would be an
>object expressing
>   all facts about the observed network connection and the embedded
>protocol. Here,
>   we stay closer to the bare minimum, but include the information that a
>DNS query
>   was observed: thus we a have DNS-Query object rather than a
>URI-Object::
>
>             <cybox:Object
>id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>                 <cybox:Properties
>xsi:type="DNSQueryObject:DNSQueryObjectType">
>                     <DNSQueryObj:Question>
>                         <DNSQueryObj:QName>
>                             <URIObj:Value
>datatype="anyURI">evil.com</URIObj:Value>
>                         </DNSQueryObj:QName>
>                     </DNSQueryObj:Question>
>                 </cybox:Properties>
>             </cybox:Object>
>
> - We use the "Information Source" element of the relationship to
>communicate
>   the reason why the observed  DNS query really constitutes an incident
>by
>   providing a reference to the IDS rule that was triggered::
>
>          <incident:Related_Observables scope="exclusive">
>                 <incident:Related_Observable>
>                     <stixCommon:Information_Source>
>                         <stixCommon:References>
>                  
><stixCommon:Reference>http://www.snort.org/search/sid/123456</stixCommon:R
>eference>
>                         </stixCommon:References>
>                     </stixCommon:Information_Source>
>                     <stixCommon:Observable>
>                         <cybox:Object
>idref="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0"/>
>                     </stixCommon:Observable>
>                 </incident:Related_Observable>
>                (...)
>
> - Also with the requested course of action, we refer to an existing
>object known to
>   all involved parties rather than resending the same COA object over
>and over::
>
>      <incident:COA_Requested>
>                 <incident:Course_Of_Action
>idref="some_cert:COA-19b68437-6467-4b74-99a9-152c365fe6ad"/>
>      </incident:COA_Requested>
>
>
> So much for now -- I hope, this was helpful in some way!
>
> Kind regards,
>
> Bernd
>
> ----
> Bernd Grobauer, Siemens CERT
Reply | Threaded
Open this post in threaded view
|

Re: Guidance on using STIX for event data.

Barnum, Sean D.
In reply to this post by Johnny Vestergaard
Hey Johnny,

Looks like a good first stab.
A few suggestions for you below. Also, attached is a tweaked version of
your example with most of the below suggestions addressed.

- You don't need all of those namespace declarations. Best case, you only
need to include the ones used within that specific instance document.
Worst case, you may want to decide the union of all of those that will
ever be included in your content (can be defined using a STIX Profile) and
include all of those in each and every document which saves the logic of
having to determine at time of creation which ones to include but does not
clutter your content with unneeded ones.

- It is suggested practice to include an id for the STIX_Package so that
it can be unambiguously identified

- There is suggested practice guidance
(https://github.com/STIXProject/schemas/wiki/Suggested-Practices) for
formatting ids on STIX constructs. You don't have to follow them but it
does make using content from various sources much easier.


- You don't need to declare the Incident namespace locally inline when you
have already declared it above

- You need to specify instances of the Incident construct within the
enclosing Incidents. The id and xsi:type declaration go on Incident

- @scope on the Related_Observables construct does not really need to be
specified. This is a very rarely used feature and is definitely not
relevant for cases where only one Related_Observable is specified.

- You need to include an Object construct layer within Observable and a
Properties construct layer within Object declaring its xsi:type (in this
case, NetworkConnectionObjectType) to contain the object properties.

- You don't need to include datatype="string" on the Port Value element or
typically on any of the object property fields (with a couple exceptions
that we are working out).




I specified schema locations in my modified version of your example so it
would be easier to validate but they are not required.


Let me know if any of this does not make sense.


sean


On 11/30/13 5:25 PM, "Johnny Vestergaard" <[hidden email]> wrote:

>As a followup to our discussion, here is my current take on a very
>basic STIX incident message for traffic to a ConPot honeypot:
>
>https://gist.github.com/johnnykv/7725345
>
>regards,
>Johnny
>
>On Fri, Nov 29, 2013 at 4:15 PM, Patrick Maroney <[hidden email]>
>wrote:
>> Bernd,
>>
>> A general comment on the use case:  collection and sharing of the
>>answer(s) to the query, the time(s) of observations, and metadata like
>>TTL, RRI, etc. can all be very valuable presuming one has structured
>>their DNS mitigations to allow the capture of same while still
>>mitigating the threat.   As the "natural" observer, one is the unique
>>position to gather this data without OPSEC exposure.  For example if
>>activity is the result of a targeted attack, adversary will expect
>>activity from that organization.
>>
>> Great discussions by the way.  We need more of these use cases and
>>examples.
>>
>> Patrick Maroney
>> Executive Director
>> Defense Security Information Exchange (DSIE)
>> Office: (856)983-0001
>> Cell: (609)841-5104
>> [hidden email]
>> ________________________________________
>> From: [hidden email]
>><[hidden email]> on behalf of Grobauer,
>>Bernd <[hidden email]>
>> Sent: Friday, November 29, 2013 8:40:39 AM
>> To: Johnny Vestergaard
>> Cc: stix-discussion-list Structured Threat Information Expression/ST
>> Subject: RE: Guidance on using STIX for event data.
>>
>> Hi Johnny,
>>
>> here is the promised use case and some explanation regarding our
>> design decisions. Feedback by you or anybody else on why some
>> decisions might better be made differently (or, conversely, support for
>> the taken decisions) is most welcome.
>>
>> First of all, the use-case, which is rather small: a submitter
>> has an IDS listening to DNS traffic and thus notices queries for
>> known malicious domains. The position of the IDS is such that
>> it cannot see the requesting clients -- just that malicious
>> domains were queried.
>>
>> This information is to be shared with other parties, e.g. such
>> that are able to investigate further, block the domains, etc.
>>
>> So, there is not much "hard" data to be shared, really:
>>
>> - the observed queries
>> - the reason, why the queried domain is considered
>>   as malicious
>>
>> Then there is timing info: when were the queries observed,
>> but we assume daily reports about these events, and for
>> the purpose of blocking the domains or further investigation
>> (e.g. in proxy logs), the information about the 24h slice
>> in which the requests were observed is sufficient ... and
>> this information is implicit; come to think of it, we could/should
>> add timing info in the incident's Time element about first
>> observation...
>>
>> We thus want to communicate a single incident that reports
>> this information (one could also think about one incident
>> per observed domain, but let's go for one incident for now)
>> and asks for some course-of-action to be taken.
>>
>> The whole XML in the attachment; let me go through some parts
>> of the XML in the email:
>>
>> - We need to add our namespaces to the package::
>>
>>     <stix:STIX_Package xmlns:some_cert="cert.some_organization.com"
>>xmlns:submitter="... >
>>
>> - Whenever a vocabulary is used, we add information, also if it is
>>   the default vocabulary::
>>
>>         <stix:Package_Intent
>>xsi:type="stixVocabs:PackageIntentVocab-1.0">Incident</stix:Package_Inten
>>t>
>>
>> - We try to use references to existing objects whereever possible and
>>do not
>>   resend objects that are known to be known by the recipient. For
>>example,
>>   the submitter knows that the recipient has the object describing its
>>   identity::
>>
>>         <stix:Information_Source>
>>             <stixCommon:Identity
>>idref="some_cert:identity-submitter.some.organization.com"/>
>>         </stix:Information_Source>
>>
>>   Here we have the first occurrence of an identifier that does not use
>>uuid. I am
>>   not sure how strict the STIX/Cybox specification is about the shape
>>of identifiers:
>>   we have use-cases where we'd rather use something else than uuid: a
>>meaningful
>>   identifier in this case, or a identifier generated by a digest (see
>>below).
>>
>> - We communicate the observables used in this STIX package and
>>reference them
>>   later from incidents, indicators, ...::
>>
>>     <stix:Observables cybox_major_version="2" cybox_minor_version="0"
>>cybox_update_version="1">
>>         <cybox:Observable
>>id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>>             (...)
>>         </cybox:Observable>
>>
>>         <cybox:Observable
>>id="submitter:Observable-6e260711e503a9108541dfd13616ce97">
>>             (...)
>>         </cybox:Observable>
>>     </stix:Observables>
>>
>> - Two specialities about the identifiers for observable and object
>>embedded in the observable::
>>
>>         <cybox:Observable
>>id="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0">
>>             <cybox:Object
>>id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>>
>>   - The identifier of the Object is auto-generated by a
>>     JSON-2-STIX-converter taking a digest of the "basic facts" from
>>     which the object is created. We do this here for the following
>>     reason: the data that is converted to STIX is generated by a
>>     system that knows nothing of STIX, but simply pushes events in
>>     JSON-format to the JSON-2-STIX-converter. It is likely to
>>     push the same domain names again and again (at least for some days)
>>... and I do not
>>     want to have the same (very simple) DNS-Query-object under a dozen
>>different
>>     identifiers from one and the same submitter. Since the JSON-2-STIX
>>converter
>>     does not hold state between conversions, using a digest as
>>identifier is
>>     the easiest way to avoid this.
>>
>>   - Notice that we give an identifier to both the observable and the
>>embedded object.
>>     The identifier on the observable is required  for referencing it;
>>the identifier
>>     on the object is required, because our processing tool extracts
>>cybox
>>     objects as separate entities with associated data type (a DNS-Query
>>object, in this case).
>>
>> - What to communicate as observable? The bare minimum would be an
>>URI-Object that communicates
>>   the domain name; on the other end of the scale there would be an
>>object expressing
>>   all facts about the observed network connection and the embedded
>>protocol. Here,
>>   we stay closer to the bare minimum, but include the information that
>>a DNS query
>>   was observed: thus we a have DNS-Query object rather than a
>>URI-Object::
>>
>>             <cybox:Object
>>id="submitter:DNSQuery-4cd4dcdbf30f51d43303c9e0359e7fb0">
>>                 <cybox:Properties
>>xsi:type="DNSQueryObject:DNSQueryObjectType">
>>                     <DNSQueryObj:Question>
>>                         <DNSQueryObj:QName>
>>                             <URIObj:Value
>>datatype="anyURI">evil.com</URIObj:Value>
>>                         </DNSQueryObj:QName>
>>                     </DNSQueryObj:Question>
>>                 </cybox:Properties>
>>             </cybox:Object>
>>
>> - We use the "Information Source" element of the relationship to
>>communicate
>>   the reason why the observed  DNS query really constitutes an incident
>>by
>>   providing a reference to the IDS rule that was triggered::
>>
>>          <incident:Related_Observables scope="exclusive">
>>                 <incident:Related_Observable>
>>                     <stixCommon:Information_Source>
>>                         <stixCommon:References>
>>                
>><stixCommon:Reference>http://www.snort.org/search/sid/123456</stixCommon:
>>Reference>
>>                         </stixCommon:References>
>>                     </stixCommon:Information_Source>
>>                     <stixCommon:Observable>
>>                         <cybox:Object
>>idref="submitter:Observable-4cd4dcdbf30f51d43303c9e0359e7fb0"/>
>>                     </stixCommon:Observable>
>>                 </incident:Related_Observable>
>>                (...)
>>
>> - Also with the requested course of action, we refer to an existing
>>object known to
>>   all involved parties rather than resending the same COA object over
>>and over::
>>
>>      <incident:COA_Requested>
>>                 <incident:Course_Of_Action
>>idref="some_cert:COA-19b68437-6467-4b74-99a9-152c365fe6ad"/>
>>      </incident:COA_Requested>
>>
>>
>> So much for now -- I hope, this was helpful in some way!
>>
>> Kind regards,
>>
>> Bernd
>>
>> ----
>> Bernd Grobauer, Siemens CERT


Johnny V Honeypot report example (modified).xml (4K) Download Attachment
12