Defanging/Regular Expression and base property examples

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Defanging/Regular Expression and base property examples

Renne, David

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

 

I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

Re: Defanging/Regular Expression and base property examples

pmaroney
David,

re: "I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended."

Many have tried within various communities and same has been suggested for this community, along with the recognition that we need to accommodate the reality that no common standard is consistently used and therefore we have to adopt "flexible" (aka relatively complicated) transformation schemes.  

All we need is for someone in this community to take point and draft a "Standard" (or leverage an existing defanging/re-fanging ad hoc document), run it through the community for general consensus, and we could at least have a baseline de facto standard.

Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
[hidden email]

From: <Renne>, David <[hidden email]>
Date: Monday, March 17, 2014 4:09 PM
To: cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Defanging/Regular Expression and base property examples

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

 

I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

Re: Defanging/Regular Expression and base property examples

pmaroney
Proposal: We developed a De-Fanging Guideline with DC3/DCISE as part of the DIB Framework Agreement that we could table as a draft.  Perhaps if there is anyone from DC3/DCISE on the list, and no objections to considering this as a draft framework, they can forward to the list?

Patrick Maroney


Office: (856)983-0001
Cell: (609)841-5104
[hidden email]

From: Patrick Maroney <[hidden email]>
Date: Monday, March 17, 2014 5:26 PM
To: "Renne, David" <[hidden email]>, cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Re: Defanging/Regular Expression and base property examples

David,

re: "I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended."

Many have tried within various communities and same has been suggested for this community, along with the recognition that we need to accommodate the reality that no common standard is consistently used and therefore we have to adopt "flexible" (aka relatively complicated) transformation schemes.  

All we need is for someone in this community to take point and draft a "Standard" (or leverage an existing defanging/re-fanging ad hoc document), run it through the community for general consensus, and we could at least have a baseline de facto standard.

Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
[hidden email]

From: <Renne>, David <[hidden email]>
Date: Monday, March 17, 2014 4:09 PM
To: cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Defanging/Regular Expression and base property examples

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

 

I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

Re: Defanging/Regular Expression and base property examples

Barnum, Sean D.
In reply to this post by Renne, David
Hi David.

Comments inline below.

sean

From: <Renne>, David <[hidden email]>
Date: Monday, March 17, 2014 at 4:09 PM
To: cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Defanging/Regular Expression and base property examples

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

[sean]David, did you mean that you wanted an example of a defanged IP address using the AddressObject or a defanged URL using the URIObject? You would not be using an AddressObject to represent a URL. I assume this was just a typo. :-)
[sean]Anyway, one common defanging algorithm used for either IP addresses or URLs is simply to enclose “dots” in “brackets”. For example, “192.168.0.1" would become "192[.]168[.]0[.]1” or “http://www.evil.com” would become "<a href="http://www[.]evil[.]com">http://www[.]evil[.]com”. By doing this, you remove the risk of the URL value or the IP value being inadvertently used in a dynamic action.
So, an example of a CybOX pattern for a URL that has been defanged might look like this:

<cybox:Object id="example:Object-26be6630-b2df-4bf9-8750-3f45ca9e19cf">

            <cybox:Properties xsi:type="URIObject:URIObjectType" type="URL">

                <URIObject:Value is_defanged="true" defanging_algorithm_ref="http://www.justiceleagueofamerica.org/defanging/dot-bracket-defanging.html" condition="Equals">http://example[.]com/index1[.]html</URIObject:Value>

            </cybox:Properties>

 </cybox:Object>


[sean]The reference in the @defanging_algorith_ref is not real but hopefully you get the picture.
[sean]If desired, it would also be possible to specify a @refanging_transform (gives an actual transform that can be applied to the defanged value to refang it to its original state) and @refanging_transform_type (explicitly describe what type of transform content is contained in the @refanging_transform attribute (e.g. Regex). 

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

[sean]Does the above comment explain this for you? You would want to defang things like URLs so that someone does not inadvertently click on or load malicious ones. The specific defanging/refanging transformations simply take them from the original state to a defanged state and then back again.

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

[sean]I intentionally held off on answering this one because I had high confidence Pat would speak up, and he did. :-)

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.


[sean]How’s this? This specifies a CybOX pattern for any file whose name contains the string “Version” immediately followed by a digit (1 to 9). For example, “System Architecture Version5.doc” would match, "System Architecture Version_5.doc” would not match.

<cybox:Object id="example:Object-17e97e7c-d3e6-4139-891b-291576dc5d41">

            <cybox:Properties xsi:type="FileObj:FileObjectType">

                <FileObj:File_Name condition="FitsPattern" pattern_type="Regex”>Version[1-9]</FileObj:File_Name>

            </cybox:Properties>

 </cybox:Object>


[sean] to specify a property field value in CybOX you would typically use the combination of @condition=“FitsPattern” and pattern_type=“Regex”. You would not neet to specify @regex_syntax unless you were explicitly signifying that the flavor of regex used (e.g. Perl) in the specified property field value was other than the default CybOX flavor. You can find details of the default CybOX flavor at http://cybox.mitre.org/language/regular_expression_support.pdf.

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

[sean]I don’t understand what you are asking here. Can you clarify please?

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

[sean]Greg Back will get you examples for these sometime soon. 



I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

Re: Defanging/Regular Expression and base property examples

Barnum, Sean D.
Okay. So, sorry all.
I have been informed that the hokey defanging_algorithm_ref="http://www.justiceleagueofamerica.org/defanging/dot-bracket-defanging.htmlexample that I completely pulled out of my rear in an attempt to be as obviously fake as possible appears to actually resolve to something. How?? I have no idea.
Please know that this was unintentional.
Wow!!!!


Also, another member of the community while agreeing that there really is no consensus “standard” for defanging made practice suggestions of:
  • Replacing “.” with “[.]” rather than “[dot]”
  • Replacing “http” with “hxxp”
  • Replacing “@“ with [@]” rather than “[at]”


sean

From: <Barnum>, "Barnum, Sean D." <[hidden email]>
Date: Tuesday, March 18, 2014 at 1:10 PM
To: "Renne, David" <[hidden email]>, cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Re: Defanging/Regular Expression and base property examples

Hi David.

Comments inline below.

sean

From: <Renne>, David <[hidden email]>
Date: Monday, March 17, 2014 at 4:09 PM
To: cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Defanging/Regular Expression and base property examples

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

[sean]David, did you mean that you wanted an example of a defanged IP address using the AddressObject or a defanged URL using the URIObject? You would not be using an AddressObject to represent a URL. I assume this was just a typo. :-)
[sean]Anyway, one common defanging algorithm used for either IP addresses or URLs is simply to enclose “dots” in “brackets”. For example, “192.168.0.1" would become "192[.]168[.]0[.]1” or “http://www.evil.com” would become "<a href="http://www[.]evil[.]com">http://www[.]evil[.]com”. By doing this, you remove the risk of the URL value or the IP value being inadvertently used in a dynamic action.
So, an example of a CybOX pattern for a URL that has been defanged might look like this:

<cybox:Object id="example:Object-26be6630-b2df-4bf9-8750-3f45ca9e19cf">

            <cybox:Properties xsi:type="URIObject:URIObjectType" type="URL">

                <URIObject:Value is_defanged="true" defanging_algorithm_ref="http://www.justiceleagueofamerica.org/defanging/dot-bracket-defanging.html" condition="Equals"><a href="http://example[.]com/index1[.]html">http://example[.]com/index1[.]html</URIObject:Value>

            </cybox:Properties>

 </cybox:Object>


[sean]The reference in the @defanging_algorith_ref is not real but hopefully you get the picture.
[sean]If desired, it would also be possible to specify a @refanging_transform (gives an actual transform that can be applied to the defanged value to refang it to its original state) and @refanging_transform_type (explicitly describe what type of transform content is contained in the @refanging_transform attribute (e.g. Regex). 

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

[sean]Does the above comment explain this for you? You would want to defang things like URLs so that someone does not inadvertently click on or load malicious ones. The specific defanging/refanging transformations simply take them from the original state to a defanged state and then back again.

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

[sean]I intentionally held off on answering this one because I had high confidence Pat would speak up, and he did. :-)

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.


[sean]How’s this? This specifies a CybOX pattern for any file whose name contains the string “Version” immediately followed by a digit (1 to 9). For example, “System Architecture Version5.doc” would match, "System Architecture Version_5.doc” would not match.

<cybox:Object id="example:Object-17e97e7c-d3e6-4139-891b-291576dc5d41">

            <cybox:Properties xsi:type="FileObj:FileObjectType">

                <FileObj:File_Name condition="FitsPattern" pattern_type="Regex”>Version[1-9]</FileObj:File_Name>

            </cybox:Properties>

 </cybox:Object>


[sean] to specify a property field value in CybOX you would typically use the combination of @condition=“FitsPattern” and pattern_type=“Regex”. You would not neet to specify @regex_syntax unless you were explicitly signifying that the flavor of regex used (e.g. Perl) in the specified property field value was other than the default CybOX flavor. You can find details of the default CybOX flavor at http://cybox.mitre.org/language/regular_expression_support.pdf.

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

[sean]I don’t understand what you are asking here. Can you clarify please?

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

[sean]Greg Back will get you examples for these sometime soon. 



I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

RE: Defanging/Regular Expression and base property examples

Renne, David
In reply to this post by Barnum, Sean D.

Thanks see below. 

 

The thread is resolved until someone wants to submit some examples for is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes.  That would be sweet!

 

No big deal if not.

 

From: Barnum, Sean D. [mailto:[hidden email]]
Sent: Tuesday, March 18, 2014 1:11 PM
To: Renne, David; cybox-discussion-list Cyber Observable Expression/CybOX Discussi
Subject: Re: Defanging/Regular Expression and base property examples

 

Hi David.

 

Comments inline below.

 

sean

 

From: <Renne>, David <[hidden email]>
Date: Monday, March 17, 2014 at 4:09 PM
To: cybox-discussion-list Cyber Observable Expression/CybOX Discussi <[hidden email]>
Subject: Defanging/Regular Expression and base property examples

 

Hi there:

 

Do you guys have examples of BaseObjectPropertyGroup cybox XML which utilizes attribute examples where we have a cybox object that is describing an observable which:

 

1.       Shows a AddressObject which is a defanged URL in which this cybox object URL uses the most common defanging algorithm and has the attribute is_defanged=”true” and of who has a defanging_algorithm_ref link?  I want to see a live example of both defanging and refanging transform attributes.

[sean]David, did you mean that you wanted an example of a defanged IP address using the AddressObject or a defanged URL using the URIObject? You would not be using an AddressObject to represent a URL. I assume this was just a typo. :-)

 

YES

 

[sean]Anyway, one common defanging algorithm used for either IP addresses or URLs is simply to enclose “dots” in “brackets”. For example, “192.168.0.1" would become "192[.]168[.]0[.]1” or “http://www.evil.com” would become "<a href="http://www[.]evil[.]com">http://www[.]evil[.]com”. By doing this, you remove the risk of the URL value or the IP value being inadvertently used in a dynamic action.

So, an example of a CybOX pattern for a URL that has been defanged might look like this:

 

<cybox:Object id="example:Object-26be6630-b2df-4bf9-8750-3f45ca9e19cf">

            <cybox:Properties xsi:type="URIObject:URIObjectType" type="URL">

                <URIObject:Value is_defanged="true" defanging_algorithm_ref="http://www.justiceleagueofamerica.org/defanging/dot-bracket-defanging.html" condition="Equals"><a href="http://example[.]com/index1%5b.%5dhtml%3c/URIObject:Value">http://example[.]com/index1[.]html</URIObject:Value>

            </cybox:Properties>

 </cybox:Object>

 

 

 

[sean]The reference in the @defanging_algorith_ref is not real but hopefully you get the picture.

[sean]If desired, it would also be possible to specify a @refanging_transform (gives an actual transform that can be applied to the defanged value to refang it to its original state) and @refanging_transform_type (explicitly describe what type of transform content is contained in the @refanging_transform attribute (e.g. Regex). 

 

a.       I am really not a security expert, so I don’t know what would be a good use case for the most common algorithm to describe how URLS have been defanged and how to refang them back.

[sean]Does the above comment explain this for you? You would want to defang things like URLs so that someone does not inadvertently click on or load malicious ones. The specific defanging/refanging transformations simply take them from the original state to a defanged state and then back again.

 

b.      I wish it was just a little more simple that you would recommend certain default defanging algorithms that are industry standards instead of making the XSD so open ended.

[sean]I intentionally held off on answering this one because I had high confidence Pat would speak up, and he did. :-)

Thanks Pat

 

2.       Can you show some kind of cybox object which utilizes regular expressions?

a.       The way I am reading the XSD is that if condition=”FitsPattern”, then you can use possibly the regex_syntax attribute or the pattern_type?  I need to be able to visualize an example to see how this might be used in the real world.

 

[sean]How’s this? This specifies a CybOX pattern for any file whose name contains the string “Version” immediately followed by a digit (1 to 9). For example, “System Architecture Version5.doc” would match, "System Architecture Version_5.doc” would not match.

 

<cybox:Object id="example:Object-17e97e7c-d3e6-4139-891b-291576dc5d41">

            <cybox:Properties xsi:type="FileObj:FileObjectType">

                <FileObj:File_Name condition="FitsPattern" pattern_type="Regex”>Version[1-9]</FileObj:File_Name>

            </cybox:Properties>

 </cybox:Object>

 

[sean] to specify a property field value in CybOX you would typically use the combination of @condition=“FitsPattern” and pattern_type=“Regex”. You would not neet to specify @regex_syntax unless you were explicitly signifying that the flavor of regex used (e.g. Perl) in the specified property field value was other than the default CybOX flavor. You can find details of the default CybOX flavor at http://cybox.mitre.org/language/regular_expression_support.pdf.

 

Thanks for the link.  Is there a javascript library that you guys know of that would closely validate the syntax of the default regular expression values entered? 

 

b.      Since regular expressions seem to be only found in attributes, would there be values inside of the node itself if someone wants to express a regular expression?

[sean]I don’t understand what you are asking here. Can you clarify please?

I think I just did not understand where you wanted the actual regEx value.  For some reason I thought it might be stuffed into an attribute value.  But it is just inside the element value.

 

3.       Can you provide real world examples or use cases for a cybox object that needs to use the is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref, appears_random, bit_mask,  or observed_encoding attributes?

[sean]Greg Back will get you examples for these sometime soon. 

 

 

I need to use the information you provide to determine exactly when to show these attributes or which ones to really support out of the box.  Many of these attributes don’t make sense for certain cybox objects and I wish there was a matrix that showed which cybox objects might utilize these attributes since they are core values of several cybox objects.  I realize you extend off of this base group, but in several cases, I cant imagine certain attributes ever making any sense for certain cybox objects.  Why would anyone ever use an is_defanged attribute to a file object or a registry key object?   Many of these do not seem to apply to certain fields on some cybox objects.

 

I am sure in theory, parsing and interpreting cybox can probably become extremely cumbersome if an end user did not know that these additional attributes give new meaning to the XML payload and how to process the data differently depending on if these attributes are being used.

 

Thanks for any information you can provide!


_____________________________________________________________

DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.


_____________________________________________________________
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
Reply | Threaded
Open this post in threaded view
|

RE: Defanging/Regular Expression and base property examples

Back, Greg
David -

Hopefully these are helpful.

Greg


is_case_sensitive
----------------------
The default for this is "true", so it only needs to be specified is when the property should be matched in a case-insensitive way.  Case-insensitivity is the default in some contexts (Windows file names, domain names), so often tools that would process this content will often do The Right Thing anyway, but it helps to be explicit. In some contexts, it might be necessary to say "I want to match a URL that ends with .asp, .ASP, or even .Asp"

<URIObj:URIObjectType>
    <URIObj:Value is_case_sensitive="false" condition="EndsWith">.asp</URIObj:Value>
</URIObj:URIObjectType>


obfuscation
--------------
When sharing threat information, it can be common practice to hide some information that is not strictly necessary to those receiving the information. One common example is victim email addresses, where you might want to indicate the organizations that received the email, but not the individuals. This is typically more useful for CybOX instance data than CybOX patterns.

<EmailMessageObj:EmailMessageObjectType
    <EmailMessageObj:Header>
        <EmailMessageObj:To>
            <EmailMessageObj:Recipient xsi:type="AddressObj:AddressObjectType" category="e-mail">
                <AddressObj:Address_Value obfuscation_algorithm_ref="http://example.com/privacy/howto.html#obscure_victims" is_obfuscated="true">[hidden email]</AddressObj:Address_Value>
            </EmailMessageObj:Recipient>
            <EmailMessageObj:Recipient xsi:type="AddressObj:AddressObjectType" category="e-mail">
                <AddressObj:Address_Value obfuscation_algorithm_ref="http://example.com/privacy/howto.html#obscure_victims" is_obfuscated="true">[hidden email]</AddressObj:Address_Value>
            </EmailMessageObj:Recipient>
            <EmailMessageObj:Recipient xsi:type="AddressObj:AddressObjectType" category="e-mail">
                <AddressObj:Address_Value obfuscation_algorithm_ref="http://example.com/privacy/howto.html#obscure_victims" is_obfuscated="true">[hidden email]</AddressObj:Address_Value>
            </EmailMessageObj:Recipient>
        </EmailMessageObj:To>
    </EmailMessageObj:Header>
</EmailMessageObj:EmailMessageObjectType>


appears_random
----------------------
I think of this as a precursor to an actual pattern (e.g., using a regular expression), especially when you are trying to express a property that you think may not be fixed. For example, a malware sample copies itself to a temporary directory using a randomly-generated filename. You don't want people receiving the CybOX content to assume that it HAS to be that exact value, but don't have enough information to describe an exact pattern.

<FileObj:FileObjectType xsi:type="FileObj:FileObjectType">
    <FileObj:File_Path appears_random="true">C:\WINDOWS\Temp\AR4982.pdf</FileObj:File_Path>
</FileObj:FileObjectType>


observed_encoding
------------------------
The default for XML content is UTF-8, and even if you were to express a different encoding (Latin-1) at the top of the XML file, it may not be possible to have all content in that CybOX document in that encoding. Given an email message with a "raw" subject of the following:

Subject: =?iso-8859-1?q?p=F6stal?=

You would represent it in CybOX as:

<EmailMessageObj:EmailMessageObjectType
    <EmailMessageObj:Header>
        <EmailMessageObj:Subject observed_encoding="iso-8859-1">pöstal</EmailMessageObj:Subject>
    </EmailMessageObj:Header>
</EmailMessageObj:EmailMessageObjectType>

Even though the text is now UTF-8, you know that it was originally sent as Latin-1.

Hope that helps,

Greg

>-----Original Message-----
>From: [hidden email] [mailto:owner-cybox-
>[hidden email]] On Behalf Of Renne, David
>Sent: Tuesday, March 18, 2014 1:37 PM
>To: Barnum, Sean D.; cybox-discussion-list Cyber Observable
>Expression/CybOX Discussi
>Subject: RE: Defanging/Regular Expression and base property examples
>
>Thanks see below.
>
>
>
>The thread is resolved until someone wants to submit some examples for
>is_case_sensitive, is_obfuscated/obfuscation_algorithm_ref,
>appears_random, bit_mask,  or observed_encoding attributes.  That would be
>sweet!
>
>
>
>No big deal if not.
>
>
>