[TAXII] Extended Headers and Status Detail

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[TAXII] Extended Headers and Status Detail

Jordan, Bret
Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 


signature.asc (858 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [TAXII] Extended Headers and Status Detail

Jordan, Bret
I think I get the idea for the Status Detail, just not the Extended Headers, so any examples on that would be helpful…    

I do have a question about why we allow multiple <Detail> lines in the <Status Detail>.  Given the list of data points in the tables and the fact that we can only have a single @status_type, what is the use case for multiple <Detail> lines?  It seems like that would never actually happen…???….???

<taxii_11:Status_Message 
    xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1” 
    message_id="42902” 
    in_response_to="34502” 
    status_type=“PENDING”
>
    <taxii_11:Status_Detail>
<Detail name=“ESTIMATED_WAIT”>60</Detail>
    </taxii_11:Status_Detail>
</taxii_11:Status_Message> 
 



Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Mar 19, 2015, at 1:09 PM, Jordan, Bret <[hidden email]> wrote:

Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 



signature.asc (858 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [TAXII] Extended Headers and Status Detail

mdavidson

Bret,

 

I don’t think I have any examples I can share publicly. They are an extension point in TAXII (just like X-Headers in HTTP or SMTP), so “Official” TAXII doesn’t define specific extended headers or values. As long as you can take in a name (key) and any kind of value (e.g., array, object, literal), you’ll be OK.

 

It may or may not be the best example, but here’s how libtaxii deserializes extended headers: [1] [2]. I do know for a fact that this works for some groups that are using Extended Headers.

 

(The basic answer is: Sorry, I don’t have a great example; you just have to do something sufficiently general)

 

As for Status Detail, you can have multiple detail lines. In fact, the example you picked (Pending) defines three status details (Check out Table 3 of the TAXII Services Spec): Estimated Wait, Result ID, and Will Push. Libtaxii enforces this: if you attempt to create a Status Message with a Status Type of Pending, all three Status Details must be present (because TAXII requires them) or libtaxii will raise an error.

 

# > sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60})

ValueError: status_detail['WILL_PUSH'] is not allowed to be None and the provided value was None

 

vs

 

# > >>> sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60, 'WILL_PUSH': False, 'RESULT_ID': '12345'})

(No error)

 

 

Thank you.

-Mark

 

[1] https://github.com/TAXIIProject/libtaxii/blob/master/libtaxii/messages_11.py#L1124

[2] https://github.com/TAXIIProject/libtaxii/blob/master/libtaxii/messages_11.py#L1150

 

From: Jordan, Bret [mailto:[hidden email]]
Sent: Thursday, March 19, 2015 3:54 PM
To: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: [TAXII] Extended Headers and Status Detail

 

I think I get the idea for the Status Detail, just not the Extended Headers, so any examples on that would be helpful…    

 

I do have a question about why we allow multiple <Detail> lines in the <Status Detail>.  Given the list of data points in the tables and the fact that we can only have a single @status_type, what is the use case for multiple <Detail> lines?  It seems like that would never actually happen…???….???

 

<taxii_11:Status_Message 
    xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1” 
    message_id="42902” 
    in_response_to="34502” 
    status_type=“PENDING”
>
    <taxii_11:Status_Detail>
     <Detail name=“ESTIMATED_WAIT”>60</Detail>
    </taxii_11:Status_Detail>
</taxii_11:Status_Message> 
 

 

 

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

On Mar 19, 2015, at 1:09 PM, Jordan, Bret <[hidden email]> wrote:

 

Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: [TAXII] Extended Headers and Status Detail

Jordan, Bret
So in reading through the TAXII documents, in regards to the <Status Detail>  i did not come to the conclusion that all three <Detail>s were required for that field.  I only understood the documentation to mean that one of them was required..   We might want to add some verbiage to the docs.

In regards to the extended headers, you do not have anything that has been done before that you can share.  But based on your knowledge of what has been done, can you fabricate something that is based on a potentially real use case and share that?


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Mar 20, 2015, at 5:34 AM, Davidson II, Mark S <[hidden email]> wrote:

Bret,
 
I don’t think I have any examples I can share publicly. They are an extension point in TAXII (just like X-Headers in HTTP or SMTP), so “Official” TAXII doesn’t define specific extended headers or values. As long as you can take in a name (key) and any kind of value (e.g., array, object, literal), you’ll be OK.
 
It may or may not be the best example, but here’s how libtaxii deserializes extended headers: [1] [2]. I do know for a fact that this works for some groups that are using Extended Headers.
 
(The basic answer is: Sorry, I don’t have a great example; you just have to do something sufficiently general)
 
As for Status Detail, you can have multiple detail lines. In fact, the example you picked (Pending) defines three status details (Check out Table 3 of the TAXII Services Spec): Estimated Wait, Result ID, and Will Push. Libtaxii enforces this: if you attempt to create a Status Message with a Status Type of Pending, all three Status Details must be present (because TAXII requires them) or libtaxii will raise an error.
 
# > sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60})
ValueError: status_detail['WILL_PUSH'] is not allowed to be None and the provided value was None
 
vs
 
# > >>> sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60, 'WILL_PUSH': False, 'RESULT_ID': '12345'})
(No error)
 
 
Thank you.
-Mark
 
 
From: Jordan, Bret [[hidden email]] 
Sent: Thursday, March 19, 2015 3:54 PM
To: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: [TAXII] Extended Headers and Status Detail
 
I think I get the idea for the Status Detail, just not the Extended Headers, so any examples on that would be helpful…    
 
I do have a question about why we allow multiple <Detail> lines in the <Status Detail>.  Given the list of data points in the tables and the fact that we can only have a single @status_type, what is the use case for multiple <Detail> lines?  It seems like that would never actually happen…???….???
 
<taxii_11:Status_Message 
    xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1” 
    message_id="42902” 
    in_response_to="34502” 
    status_type=“PENDING”
>
    <taxii_11:Status_Detail>
     <Detail name=“ESTIMATED_WAIT”>60</Detail>
    </taxii_11:Status_Detail>
</taxii_11:Status_Message> 
 
 
 
 
Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
 
On Mar 19, 2015, at 1:09 PM, Jordan, Bret <[hidden email]> wrote:
 
Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.

 

Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 


signature.asc (858 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [TAXII] Extended Headers and Status Detail

mdavidson

Bret,

 

You’ve hit on one of the areas that could be improved in TAXII. The general line of thinking is that the TAXII Services Specification outlays the concepts for TAXII Messages, and a TAXII Message Binding maps those concepts to a representation. One nuance here is that some representations (e.g., XML) can define default values for fields that are not present while others cannot. Therefore, it is up to the Message Binding to explicitly call out which fields are required, optional, and not used.

 

To that end, Table 2 of the TAXII XML Message Binding 1.1 defines which of those fields are required for TAXII XML.

 

The unintended negative consequence is that a developer needs to cross-reference two tables from two different documents in order to implement TAXII correctly. Having worked on TAXII for so long, I sometimes forget exactly which information is in which document and mentally map it as a requirement of TAXII.

 

(This might be a bit esoteric – read on if you like that sort of thing)

This is to some degree a natural consequence of permitting multiple message bindings, since permitting multiple message bindings necessarily leaves optionality of fields up to the message binding. The only thing the Service Spec can do is require that information to be conceptually present (whether through presence of, for instance, an XML element/attribute or by defining the meaning that element/attribute’s absence). That said, the conceptual presence of information is a hard requirement to develop against.

 

Part of this is that we just don’t have practice writing multiple message binding specs and we are going to learn about some unpleasant nuances and nits along the way.

 

Thank you.

-Mark

 

From: Jordan, Bret [mailto:[hidden email]]
Sent: Friday, March 20, 2015 10:53 AM
To: Davidson II, Mark S
Cc: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: Extended Headers and Status Detail

 

So in reading through the TAXII documents, in regards to the <Status Detail>  i did not come to the conclusion that all three <Detail>s were required for that field.  I only understood the documentation to mean that one of them was required..   We might want to add some verbiage to the docs.

 

In regards to the extended headers, you do not have anything that has been done before that you can share.  But based on your knowledge of what has been done, can you fabricate something that is based on a potentially real use case and share that?

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

On Mar 20, 2015, at 5:34 AM, Davidson II, Mark S <[hidden email]> wrote:

 

Bret,

 

I don’t think I have any examples I can share publicly. They are an extension point in TAXII (just like X-Headers in HTTP or SMTP), so “Official” TAXII doesn’t define specific extended headers or values. As long as you can take in a name (key) and any kind of value (e.g., array, object, literal), you’ll be OK.

 

It may or may not be the best example, but here’s how libtaxii deserializes extended headers: [1] [2]. I do know for a fact that this works for some groups that are using Extended Headers.

 

(The basic answer is: Sorry, I don’t have a great example; you just have to do something sufficiently general)

 

As for Status Detail, you can have multiple detail lines. In fact, the example you picked (Pending) defines three status details (Check out Table 3 of the TAXII Services Spec): Estimated Wait, Result ID, and Will Push. Libtaxii enforces this: if you attempt to create a Status Message with a Status Type of Pending, all three Status Details must be present (because TAXII requires them) or libtaxii will raise an error.

 

# > sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60})

ValueError: status_detail['WILL_PUSH'] is not allowed to be None and the provided value was None

 

vs

 

# > >>> sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60, 'WILL_PUSH': False, 'RESULT_ID': '12345'})

(No error)

 

 

Thank you.

-Mark

 

 

From: Jordan, Bret [[hidden email]] 
Sent: Thursday, March 19, 2015 3:54 PM
To: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: [TAXII] Extended Headers and Status Detail

 

I think I get the idea for the Status Detail, just not the Extended Headers, so any examples on that would be helpful…    

 

I do have a question about why we allow multiple <Detail> lines in the <Status Detail>.  Given the list of data points in the tables and the fact that we can only have a single @status_type, what is the use case for multiple <Detail> lines?  It seems like that would never actually happen…???….???

 

<taxii_11:Status_Message 
    xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1” 
    message_id="42902” 
    in_response_to="34502” 
    status_type=“PENDING”
>
    <taxii_11:Status_Detail>
     <Detail name=“ESTIMATED_WAIT”>60</Detail>
    </taxii_11:Status_Detail>
</taxii_11:Status_Message> 
 

 

 

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

On Mar 19, 2015, at 1:09 PM, Jordan, Bret <[hidden email]> wrote:

 

Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

Reply | Threaded
Open this post in threaded view
|

Re: [TAXII] Extended Headers and Status Detail

Jordan, Bret
I think a simple statement in the XML binding document that all three of the PENDING statuses are required would be sufficient.  Now, regardless of what I am working on I would have had the same question, and I have read both documents and have both open and cross reference back and forth all day long.  

I hope that by working on a new XML API (not Python) and working on the JSON version, we can improve the XML version and the documentation.  I have often found that by doing a second implementation in a different programming language and/or a different protocol often helps bring light problems that we did not see before.

I may be wrong here, but I think I am the first person, out side of MITRE, to start writing a new set of APIs.  I believe everyone else is just using the Python APIs or modified versions of them.  So as I go through and write new APIs for XML STIX and future JSON STIX, I have a lot of questions.  

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Mar 20, 2015, at 9:12 AM, Davidson II, Mark S <[hidden email]> wrote:

Bret,
 
You’ve hit on one of the areas that could be improved in TAXII. The general line of thinking is that the TAXII Services Specification outlays the concepts for TAXII Messages, and a TAXII Message Binding maps those concepts to a representation. One nuance here is that some representations (e.g., XML) can define default values for fields that are not present while others cannot. Therefore, it is up to the Message Binding to explicitly call out which fields are required, optional, and not used.
 
To that end, Table 2 of the TAXII XML Message Binding 1.1 defines which of those fields are required for TAXII XML.
 
The unintended negative consequence is that a developer needs to cross-reference two tables from two different documents in order to implement TAXII correctly. Having worked on TAXII for so long, I sometimes forget exactly which information is in which document and mentally map it as a requirement of TAXII.
 
(This might be a bit esoteric – read on if you like that sort of thing)
This is to some degree a natural consequence of permitting multiple message bindings, since permitting multiple message bindings necessarily leaves optionality of fields up to the message binding. The only thing the Service Spec can do is require that information to be conceptually present (whether through presence of, for instance, an XML element/attribute or by defining the meaning that element/attribute’s absence). That said, the conceptual presence of information is a hard requirement to develop against.
 
Part of this is that we just don’t have practice writing multiple message binding specs and we are going to learn about some unpleasant nuances and nits along the way.
 
Thank you.
-Mark
 
From: Jordan, Bret [[hidden email]] 
Sent: Friday, March 20, 2015 10:53 AM
To: Davidson II, Mark S
Cc: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: Extended Headers and Status Detail
 
So in reading through the TAXII documents, in regards to the <Status Detail>  i did not come to the conclusion that all three <Detail>s were required for that field.  I only understood the documentation to mean that one of them was required..   We might want to add some verbiage to the docs.
 
In regards to the extended headers, you do not have anything that has been done before that you can share.  But based on your knowledge of what has been done, can you fabricate something that is based on a potentially real use case and share that?

 

Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
 
On Mar 20, 2015, at 5:34 AM, Davidson II, Mark S <[hidden email]> wrote:
 
Bret,
 
I don’t think I have any examples I can share publicly. They are an extension point in TAXII (just like X-Headers in HTTP or SMTP), so “Official” TAXII doesn’t define specific extended headers or values. As long as you can take in a name (key) and any kind of value (e.g., array, object, literal), you’ll be OK.
 
It may or may not be the best example, but here’s how libtaxii deserializes extended headers: [1] [2]. I do know for a fact that this works for some groups that are using Extended Headers.
 
(The basic answer is: Sorry, I don’t have a great example; you just have to do something sufficiently general)
 
As for Status Detail, you can have multiple detail lines. In fact, the example you picked (Pending) defines three status details (Check out Table 3 of the TAXII Services Spec): Estimated Wait, Result ID, and Will Push. Libtaxii enforces this: if you attempt to create a Status Message with a Status Type of Pending, all three Status Details must be present (because TAXII requires them) or libtaxii will raise an error.
 
# > sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60})
ValueError: status_detail['WILL_PUSH'] is not allowed to be None and the provided value was None
 
vs
 
# > >>> sm = tm11.StatusMessage(message_id='1', in_response_to='0', status_type='PENDING', status_detail={'ESTIMATED_WAIT': 60, 'WILL_PUSH': False, 'RESULT_ID': '12345'})
(No error)
 
 
Thank you.
-Mark
 
 
From: Jordan, Bret [[hidden email]] 
Sent: Thursday, March 19, 2015 3:54 PM
To: taxii-discussion-list Trusted Automated Exchange of Indicator In
Subject: Re: [TAXII] Extended Headers and Status Detail
 
I think I get the idea for the Status Detail, just not the Extended Headers, so any examples on that would be helpful…    
 
I do have a question about why we allow multiple <Detail> lines in the <Status Detail>.  Given the list of data points in the tables and the fact that we can only have a single @status_type, what is the use case for multiple <Detail> lines?  It seems like that would never actually happen…???….???
 
<taxii_11:Status_Message 
    xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1” 
    message_id="42902” 
    in_response_to="34502” 
    status_type=“PENDING”
>
    <taxii_11:Status_Detail>
     <Detail name=“ESTIMATED_WAIT”>60</Detail>
    </taxii_11:Status_Detail>
</taxii_11:Status_Message> 
 
 
 
 
Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
 
On Mar 19, 2015, at 1:09 PM, Jordan, Bret <[hidden email]> wrote:
 
Can I get some examples of these, in XML or even in text form.  I just want to make sure I understand what they are used for and what they would actually look like.

 

Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 


signature.asc (858 bytes) Download Attachment