[STIX] STIX 1.2 Multiple Descriptions - JSON Options

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
  "stix_package" : {
    "indicators" : [
      {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
  {
    "txt" : "some interesting TLP green text",
    "html" : "some interesting TLP green html text"
  },
  {
    "txt" : "some interesting TLP red text",
    "md" : "some interesting TLP red markdown text"
  }
],
      }
    ]
  }
}

2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
  "stix_package" : {
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
descriptions_structuring_format" : “txt",
"descriptions" : [
  "some interesting TLP green text",
  "some interesting TLP red text"
],
      }
    ]
  }
}


Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 


signature.asc (859 bytes) Download Attachment
JA
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

JA
I think 2) would be safer.
Would need to investigate a bit more, but I think about descriptions in XML or JSON format, in multiple languages (e.g.: French, Japanese, Arabic...), something that could potentially break the JSON when embedded.
While more restrictive, with a Guidance document and examples, like it MUST be in UTF-8 or encoded (i.e. hexa), this should avoid common issues.
Does this make sense?

On Saturday, June 6, 2015, Jordan, Bret <[hidden email]> wrote:
All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
  "stix_package" : {
    "indicators" : [
      {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
  {
    "txt" : "some interesting TLP green text",
    "html" : "some interesting TLP green html text"
  },
  {
    "txt" : "some interesting TLP red text",
    "md" : "some interesting TLP red markdown text"
  }
],
      }
    ]
  }
}

2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
  "stix_package" : {
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
descriptions_structuring_format" : “txt",
"descriptions" : [
  "some interesting TLP green text",
  "some interesting TLP red text"
],
      }
    ]
  }
}


Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
That is a very good point, language support.  This is currently a limitation in the spec and needs to be addressed.. 

A question for the community, would you send a single Indicator with descriptions in multiple languages?  Or would you send multiple Indicators, one for each language that you wanted to support?  This simple question can have radical implications on the underlying encoding.

I really like your comment Jerome about requiring some sort of base64, hex, UTF-8 or something.. The question is what should that something be?

Examples:

1) If sent as a single indicator, it could be done like:

{
  "stix_package" : {
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
"descriptions" : [
          {
            “format” : “txt”,
            “language” : “en_us”,
            “description” : "some interesting TLP green text in English"
          },
          {
            “format” : “txt”,
            “language” : “zh-cn”,
            “description” : "some interesting TLP green text in Chinese"
          },
          {
            “format” : “txt”,
            “language” : “fr”,
            “description” : "some interesting TLP green text in French"
          }
]
      }
    ]
  }
}


2) If sent as multiple Indicators it could look like

{
  "stix_package" : {
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
        “descriptions_format” : “txt”,
        “descriptions_language” : “en_us”,
"descriptions" : [
          "some interesting TLP green text in English”,
          "some interesting TLP red text in English”
]
      }
    ]
  }
}

I am curious to know what the community would like to see and how they would like it to work.


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 5, 2015, at 22:11, Jerome Athias <[hidden email]> wrote:

I think 2) would be safer.
Would need to investigate a bit more, but I think about descriptions in XML or JSON format, in multiple languages (e.g.: French, Japanese, Arabic...), something that could potentially break the JSON when embedded.
While more restrictive, with a Guidance document and examples, like it MUST be in UTF-8 or encoded (i.e. hexa), this should avoid common issues.
Does this make sense?

On Saturday, June 6, 2015, Jordan, Bret <[hidden email]> wrote:
All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
  "stix_package" : {
    "indicators" : [
      {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
  {
    "txt" : "some interesting TLP green text",
    "html" : "some interesting TLP green html text"
  },
  {
    "txt" : "some interesting TLP red text",
    "md" : "some interesting TLP red markdown text"
  }
],
      }
    ]
  }
}

2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
  "stix_package" : {
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
descriptions_structuring_format" : “txt",
"descriptions" : [
  "some interesting TLP green text",
  "some interesting TLP red text"
],
      }
    ]
  }
}


Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 



signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Dave Dittrich
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just a quick thought (that's all I have time for ATM).

On 6/6/15 9:40 AM, Jordan, Bret wrote:
> "descriptions" : [
>           "some interesting TLP green text in English”,
>           "some interesting TLP red text in English”
> ]

I think you need more structure here. How is a program to
know how to filter or insert *just* the text for a particular
TLP level? Does the text have to include "TLP green" and
be selected based on a regular expression? What if someone
inserts "TLP_GREEN" or "TLP Green" in the text? If it
isn't easy to parse and select with a program, it won't
make our lives much easier.

- --
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iKYEARECAGYFAlVzjdRfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5N0I0RENCQkYxNkUxRDhBMDZDNzUxMkE3
NTFDODBBRDE1RUUwNzkACgkQp1HICtFe4HnxxgCfRTx1+wqZqWRKGR+V3mH6N97y
LwUAnRxBHInXAZAa0upnrGJ2G2s2B9Eg
=UYbx
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Terry MacDonald-2
Hi Bret,

I like option 1. It gives the most flexibility, and allows the structure to cover with different languages which I think is very elegant. Option2 conflates these together and would require multiple versions of Indicators to supply different languages which would is not ideal. 

Dave - 'TLP' description is handled by the Handling/DataMarking Objects described here: http://stixproject.github.io/documentation/concepts/data-markings/ . Bret was just putting in text into the descriptions as a quick demonstration of the sort of data that could go into the fields. I think it was a quick sketch to help us decide which path to go down. 


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant




Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just a quick thought (that's all I have time for ATM).

On 6/6/15 9:40 AM, Jordan, Bret wrote:
>       "descriptions" : [
>           "some interesting TLP green text in English”,
>           "some interesting TLP red text in English”
>       ]

I think you need more structure here. How is a program to
know how to filter or insert *just* the text for a particular
TLP level? Does the text have to include "TLP green" and
be selected based on a regular expression? What if someone
inserts "TLP_GREEN" or "TLP Green" in the text? If it
isn't easy to parse and select with a program, it won't
make our lives much easier.

- --
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iKYEARECAGYFAlVzjdRfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5N0I0RENCQkYxNkUxRDhBMDZDNzUxMkE3
NTFDODBBRDE1RUUwNzkACgkQp1HICtFe4HnxxgCfRTx1+wqZqWRKGR+V3mH6N97y
LwUAnRxBHInXAZAa0upnrGJ2G2s2B9Eg
=UYbx
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Dave Dittrich
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]> wro
te:

>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

- --
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iKYEARECAGYFAlVzsQtfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5N0I0RENCQkYxNkUxRDhBMDZDNzUxMkE3
NTFDODBBRDE1RUUwNzkACgkQp1HICtFe4Hm8VgCgvugj6fM0E0/8+kt9iCJmX9ts
uZsAoKVvfYB6kW1VfV9F9zxIcwDm806j
=R4Nf
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
Dave,

You bring up a really good point, one that I started to tackle a few months ago and was going to present on at one of our community calls, but then the OASIS thing happened.  

Given the issues with the way marking is currently built and how it is very XML centric, I am leaning to the following structure for JSON and other encodings (think binary like cap’n photo)…. Note, I will add in the marking/data handling as I currently see it working and use shortened UUIDs just for brevity and easy of reading.  

This markings would have an inheritance model, so you could set it at the indicator level and it would effect everything downstream until it hit a change.  This would allow you to do really creative and easily consumable things in code.  And the whole purpose of my work is to make things easy for developers, vendors, and groups to adopt this technology and put it in code, specifically UI code.

{
  "stix_package" : {
    “markings” : [
      {
        “id” : “marking-UUID-1234”,
        ## Some additional data handling fields ##
      },
      {
        “id” : “marking-UUID-2222”,
        ## Some additional data handling fields ##
      },
      {
        “id” : “marking-UUID-5678”,
        ## Some additional data handling fields ##
      }
    ],
    "indicators" : [
      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
"descriptions" : [
          {
            “marking” : “marking-UUID-1234”,
            “format” : “txt”,
            “language” : “en_us”,
            “description” : "some interesting TLP red text in English"
          },
          {
            “marking” : “marking-UUID-2222”,
            “format” : “txt”,
            “language” : “zh-cn”,
            “description” : "some interesting TLP green text in Chinese"
          },
          {
            “marking” : “marking-UUID-5678”,
            “format” : “txt”,
            “language” : “fr”,
            “description” : "some interesting TLP green text in French"
          }
]
      }
    ]
  }
}




Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 6, 2015, at 20:48, Dave Dittrich <[hidden email]> wrote:

Signed PGP part
On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]> wro
te:

>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

--
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079



signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Terry MacDonald-2
In reply to this post by Dave Dittrich
Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
  </marking:Marking>
  <marking:Marking>
    <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>



The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant




Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]> wro
te:
>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

- --
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iKYEARECAGYFAlVzsQtfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5N0I0RENCQkYxNkUxRDhBMDZDNzUxMkE3
NTFDODBBRDE1RUUwNzkACgkQp1HICtFe4Hm8VgCgvugj6fM0E0/8+kt9iCJmX9ts
uZsAoKVvfYB6kW1VfV9F9zxIcwDm806j
=R4Nf
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

pmaroney
[This comment is directed at TLP and Data marking in general.]

re:  I don't recall seeing any objects with partial TLP marking at all so far in the wild.

Have the real-world use cases/challenges (primarily as artifacts of fusion of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings of same, with divergent TLP definitions and additional constructs like Source “Releasability”.  Haven’t sorted out practical solutions from an implementation perspective.   In our World View we need to retain Source referential integrity, transform TLP mappings, and respect all Data Marking and Handling instructions for all Data and Metadata.  We are currently sticking with the core concept that we will not re-distribute anything we don’t Source directly which works reasonably well in the role of an individual consumer/provider of CTI. However, this work-around does not scale well to Consortiums that need selective dissemination within their community and releasability to other communities, while respecting all original Source Data Markings and passing same on with the data.

To paraphrase Dr. Burger (again), the current models don’t seem to align well with his astute observation that TLP Transforms and Data Marking/Handling need to be “Butt Simple”.

I posit Terry's simple example below as evidence (in other words try to envision what a large STIX package containing fused intelligence from commingled Sources, with heterogeneous TLPs, and Data Markings would look like relative to “Butt Simple”.

Curious if any others are struggling with these challenges.


Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104

From: Terry MacDonald <[hidden email]>
Reply-To: Terry MacDonald <[hidden email]>
Date: Sunday, June 7, 2015 at 3:08 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
  </marking:Marking>
  <marking:Marking>
    <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>



The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant




Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich <[hidden email]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]> wro
te:
>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

- --
Dave Dittrich
[hidden email]
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iKYEARECAGYFAlVzsQtfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5N0I0RENCQkYxNkUxRDhBMDZDNzUxMkE3
NTFDODBBRDE1RUUwNzkACgkQp1HICtFe4Hm8VgCgvugj6fM0E0/8+kt9iCJmX9ts
uZsAoKVvfYB6kW1VfV9F9zxIcwDm806j
=R4Nf
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
Patrick,

Good points, as always.   I think in small niche eco-systems you can manage this headache through external contracts.  However, as you said, this all comes crumbling down when we need to actually share with a broader community - which is the whole purpose - or if you do not fully trust people in your niche eco-system to follow the out-of-band agreements.  

The other problem I have with xpath markings is they are weird, non-intuitive, and IMHO very cumbersome to implement in code.  And if it is hard to implement in code, people will not do it.  

This is why I have been pushing for a simple reference design that uses an inheritance model.  Imagine if you could do something like what I spelled out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I am using it here as a place holder for bigger things.

Marking 1234 
TLP Yellow+Extra stuff
A bunch of rules and values
Marking 5678
TLP Red+Extra stuff
A bunch of rules and values

Six_Package
Indictor FOO
Marking 1234
a bunch of data
Description 1
some yellow description (note this falls under Marking 1234)
Description 2
Marking 5678
some red description (note this falls under Marking 5678)


There are many reasons why I like this, obviously it is easy to understand, easy to implement in code, and offers inheritance.  I could also see a time when markings from certain groups will get to a steady state.  This means you could easily reference them over and over and over.

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:

[This comment is directed at TLP and Data marking in general.]

re:  I don't recall seeing any objects with partial TLP marking at all so far in the wild.

Have the real-world use cases/challenges (primarily as artifacts of fusion of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings of same, with divergent TLP definitions and additional constructs like Source “Releasability”.  Haven’t sorted out practical solutions from an implementation perspective.   In our World View we need to retain Source referential integrity, transform TLP mappings, and respect all Data Marking and Handling instructions for all Data and Metadata.  We are currently sticking with the core concept that we will not re-distribute anything we don’t Source directly which works reasonably well in the role of an individual consumer/provider of CTI. However, this work-around does not scale well to Consortiums that need selective dissemination within their community and releasability to other communities, while respecting all original Source Data Markings and passing same on with the data.

To paraphrase Dr. Burger (again), the current models don’t seem to align well with his astute observation that TLP Transforms and Data Marking/Handling need to be “Butt Simple”.

I posit Terry's simple example below as evidence (in other words try to envision what a large STIX package containing fused intelligence from commingled Sources, with heterogeneous TLPs, and Data Markings would look like relative to “Butt Simple”.

Curious if any others are struggling with these challenges.


Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
Email: [hidden email]

From: Terry MacDonald <[hidden email]<[hidden email]>>
Reply-To: Terry MacDonald <[hidden email]<[hidden email]>>
Date: Sunday, June 7, 2015 at 3:08 AM
To: "[hidden email]<[hidden email]>" <[hidden email]<[hidden email]>>
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
  </marking:Marking>
  <marking:Marking>
    <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>


The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: [hidden email]<[hidden email]>
W: www.threatloop.com<https://www.threatloop.com>

[https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich <[hidden email]<[hidden email]>> wrote:
Signed PGP part
On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]<[hidden email]>> wro
te:

>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

--
Dave Dittrich
[hidden email]<[hidden email]>
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079


signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Terry MacDonald

Bret,

Just to clarify, are you proposing a model where there are some 'well known' GUIDs that refer to the TLP model definitions (which explain the rules), and everyone uses those same shared definitions to identify how they want specific data handled? If so I quite like that idea.

Combined with the inheritance idea it is quite powerful and I like it. But I do see a couple of issues with it...

1. How do we cope with by reference relationships with inheritance?

When observables are defined 'inside' an indicator it is easy to see the inheritance, so that if the indicator is assigned a TLP RED then it's easy to see that the observables will be too. When the observables are separate, and the Indicator includes them by reference, then this gets a bit tricker. We could mandate that the the inheritance doesn't propagate through references, meaning that any referenced objects need to either explicitly labelled or covered by the respective inheritance applied higher 'up the tree'.

2. STIX Package is now just a container

With STIXPackage now being a container, and effectively only used to collect together some objects to be sent together, we may want to determine where we want the root of the inheritance to begin. We could in the future say that the inheritance has to be applied to the top-level objects within the the STIX package. In this way we could actually send two separate reports with separate objects in a single STIX package and be sure that each objects are labelled appropriately. Or we could say that the inheritance is exactly as it is now - with the ability of planting a blanket TLP level across the STIX package, effectively assuming that two separate reports will be send in two separate STIX Packages.  I'm personally leaning towards the former case, which labels the top-level objects, and uses inheritance within each object, as that seems to give more flexibility to the STIX package, allowing for the streaming use-case that we discussed late last year.

Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: [hidden email]
W: www.threatloop.com

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.


On 8 Jun 2015 7:22 am, "Jordan, Bret" <[hidden email]> wrote:
Patrick,

Good points, as always.   I think in small niche eco-systems you can manage this headache through external contracts.  However, as you said, this all comes crumbling down when we need to actually share with a broader community - which is the whole purpose - or if you do not fully trust people in your niche eco-system to follow the out-of-band agreements.  

The other problem I have with xpath markings is they are weird, non-intuitive, and IMHO very cumbersome to implement in code.  And if it is hard to implement in code, people will not do it.  

This is why I have been pushing for a simple reference design that uses an inheritance model.  Imagine if you could do something like what I spelled out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I am using it here as a place holder for bigger things.

Marking 1234 
TLP Yellow+Extra stuff
A bunch of rules and values
Marking 5678
TLP Red+Extra stuff
A bunch of rules and values

Six_Package
Indictor FOO
Marking 1234
a bunch of data
Description 1
some yellow description (note this falls under Marking 1234)
Description 2
Marking 5678
some red description (note this falls under Marking 5678)


There are many reasons why I like this, obviously it is easy to understand, easy to implement in code, and offers inheritance.  I could also see a time when markings from certain groups will get to a steady state.  This means you could easily reference them over and over and over.

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:

[This comment is directed at TLP and Data marking in general.]

re:  I don't recall seeing any objects with partial TLP marking at all so far in the wild.

Have the real-world use cases/challenges (primarily as artifacts of fusion of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings of same, with divergent TLP definitions and additional constructs like Source “Releasability”.  Haven’t sorted out practical solutions from an implementation perspective.   In our World View we need to retain Source referential integrity, transform TLP mappings, and respect all Data Marking and Handling instructions for all Data and Metadata.  We are currently sticking with the core concept that we will not re-distribute anything we don’t Source directly which works reasonably well in the role of an individual consumer/provider of CTI. However, this work-around does not scale well to Consortiums that need selective dissemination within their community and releasability to other communities, while respecting all original Source Data Markings and passing same on with the data.

To paraphrase Dr. Burger (again), the current models don’t seem to align well with his astute observation that TLP Transforms and Data Marking/Handling need to be “Butt Simple”.

I posit Terry's simple example below as evidence (in other words try to envision what a large STIX package containing fused intelligence from commingled Sources, with heterogeneous TLPs, and Data Markings would look like relative to “Butt Simple”.

Curious if any others are struggling with these challenges.


Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
Email: [hidden email]

From: Terry MacDonald <[hidden email]<[hidden email]>>
Reply-To: Terry MacDonald <[hidden email]<[hidden email]>>
Date: Sunday, June 7, 2015 at 3:08 AM
To: "[hidden email]<[hidden email]>" <[hidden email]<[hidden email]>>
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
  </marking:Marking>
  <marking:Marking>
    <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>


The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: <a href="tel:%2B61-407-203-026" value="+61407203026" target="_blank">+61-407-203-026
E: [hidden email]<[hidden email]>
W: www.threatloop.com<https://www.threatloop.com>

[https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich <[hidden email]<[hidden email]>> wrote:
Signed PGP part
On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]<[hidden email]>> wro
te:

>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

--
Dave Dittrich
[hidden email]<[hidden email]>
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jason Keirstead
In reply to this post by Jordan, Bret

Can I suggest instead of having "txt" and "html" for the format, you use standard RFC 6838 mime types (IE "text/plain", "text/html"), as it leaves the door open for future data formats while also avoiding making a STIX-specific enumeration (of which I feel there are already too many in the standard).

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Inactive hide details for "Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like"Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like what I did for TAXII and have a question

From: "Jordan, Bret" <[hidden email]>
To: <[hidden email]>
Date: 2015/06/05 07:36 PM
Subject: [STIX] STIX 1.2 Multiple Descriptions - JSON Options





All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
 "stix_package" : {
   "indicators" : [
     {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
 {
   "txt" : "some interesting TLP green text",
   "html" : "some interesting TLP green html text"
 },
 {
   "txt" : "some interesting TLP red text",
   "md" : "some interesting TLP red markdown text"
 }
],
     }
   ]
 }
}


2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
 "stix_package" : {
   "indicators" : [

      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
“descriptions_structuring_format" : “txt",
"descriptions" : [
 "some interesting TLP green text",
 "some interesting TLP red text"
],
     }
   ]
 }
}



Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
[attachment "signature.asc" deleted by Jason Keirstead/CanEast/IBM]

Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Terry MacDonald
Great idea Jason.

Cheers
Terry MacDonald


On 8 June 2015 at 22:19, Jason Keirstead <[hidden email]> wrote:

Can I suggest instead of having "txt" and "html" for the format, you use standard RFC 6838 mime types (IE "text/plain", "text/html"), as it leaves the door open for future data formats while also avoiding making a STIX-specific enumeration (of which I feel there are already too many in the standard).

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Inactive hide details for "Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like"Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like what I did for TAXII and have a question

From: "Jordan, Bret" <[hidden email]>
To: <[hidden email]>
Date: 2015/06/05 07:36 PM
Subject: [STIX] STIX 1.2 Multiple Descriptions - JSON Options





All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
 "stix_package" : {
   "indicators" : [
     {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
 {
   "txt" : "some interesting TLP green text",
   "html" : "some interesting TLP green html text"
 },
 {
   "txt" : "some interesting TLP red text",
   "md" : "some interesting TLP red markdown text"
 }
],
     }
   ]
 }
}


2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
 "stix_package" : {
   "indicators" : [

      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
“descriptions_structuring_format" : “txt",
"descriptions" : [
 "some interesting TLP green text",
 "some interesting TLP red text"
],
     }
   ]
 }
}



Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
[attachment "signature.asc" deleted by Jason Keirstead/CanEast/IBM]


JA
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

JA
In reply to this post by Jason Keirstead
+1

2015-06-08 15:19 GMT+03:00 Jason Keirstead <[hidden email]>:

Can I suggest instead of having "txt" and "html" for the format, you use standard RFC 6838 mime types (IE "text/plain", "text/html"), as it leaves the door open for future data formats while also avoiding making a STIX-specific enumeration (of which I feel there are already too many in the standard).

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Inactive hide details for "Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like"Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like what I did for TAXII and have a question

From: "Jordan, Bret" <[hidden email]>
To: <[hidden email]>
Date: 2015/06/05 07:36 PM
Subject: [STIX] STIX 1.2 Multiple Descriptions - JSON Options





All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
 "stix_package" : {
   "indicators" : [
     {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
 {
   "txt" : "some interesting TLP green text",
   "html" : "some interesting TLP green html text"
 },
 {
   "txt" : "some interesting TLP red text",
   "md" : "some interesting TLP red markdown text"
 }
],
     }
   ]
 }
}


2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
 "stix_package" : {
   "indicators" : [

      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
“descriptions_structuring_format" : “txt",
"descriptions" : [
 "some interesting TLP green text",
 "some interesting TLP red text"
],
     }
   ]
 }
}



Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
[attachment "signature.asc" deleted by Jason Keirstead/CanEast/IBM]


JA
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

JA
In reply to this post by Jordan, Bret
Just for information, the described mechanism 'strangely' reminds me
the mechanisms used to build OVAL Definitions (but maybe I'm alone to
see this :p)

2015-06-08 0:21 GMT+03:00 Jordan, Bret <[hidden email]>:

> Patrick,
>
> Good points, as always.   I think in small niche eco-systems you can manage
> this headache through external contracts.  However, as you said, this all
> comes crumbling down when we need to actually share with a broader community
> - which is the whole purpose - or if you do not fully trust people in your
> niche eco-system to follow the out-of-band agreements.
>
> The other problem I have with xpath markings is they are weird,
> non-intuitive, and IMHO very cumbersome to implement in code.  And if it is
> hard to implement in code, people will not do it.
>
> This is why I have been pushing for a simple reference design that uses an
> inheritance model.  Imagine if you could do something like what I spelled
> out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I
> am using it here as a place holder for bigger things.
>
> Marking 1234
> TLP Yellow+Extra stuff
> A bunch of rules and values
> Marking 5678
> TLP Red+Extra stuff
> A bunch of rules and values
>
> Six_Package
> Indictor FOO
> Marking 1234
> a bunch of data
> Description 1
> some yellow description (note this falls under Marking 1234)
> Description 2
> Marking 5678
> some red description (note this falls under Marking 5678)
>
>
> There are many reasons why I like this, obviously it is easy to understand,
> easy to implement in code, and offers inheritance.  I could also see a time
> when markings from certain groups will get to a steady state.  This means
> you could easily reference them over and over and over.
>
> Thanks,
>
> Bret
>
>
>
> Bret Jordan CISSP
> Director of Security Architecture and Standards | Office of the CTO
> Blue Coat Systems
> PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
> "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can
> not be unscrambled is an egg."
>
> On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:
>
> [This comment is directed at TLP and Data marking in general.]
>
> re:  I don't recall seeing any objects with partial TLP marking at all so
> far in the wild.
>
> Have the real-world use cases/challenges (primarily as artifacts of fusion
> of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings
> of same, with divergent TLP definitions and additional constructs like
> Source “Releasability”.  Haven’t sorted out practical solutions from an
> implementation perspective.   In our World View we need to retain Source
> referential integrity, transform TLP mappings, and respect all Data Marking
> and Handling instructions for all Data and Metadata.  We are currently
> sticking with the core concept that we will not re-distribute anything we
> don’t Source directly which works reasonably well in the role of an
> individual consumer/provider of CTI. However, this work-around does not
> scale well to Consortiums that need selective dissemination within their
> community and releasability to other communities, while respecting all
> original Source Data Markings and passing same on with the data.
>
> To paraphrase Dr. Burger (again), the current models don’t seem to align
> well with his astute observation that TLP Transforms and Data
> Marking/Handling need to be “Butt Simple”.
>
> I posit Terry's simple example below as evidence (in other words try to
> envision what a large STIX package containing fused intelligence from
> commingled Sources, with heterogeneous TLPs, and Data Markings would look
> like relative to “Butt Simple”.
>
> Curious if any others are struggling with these challenges.
>
>
> Patrick Maroney
> Office: (856)983-0001
> Cell: (609)841-5104
> Email: [hidden email]
>
> From: Terry MacDonald
> <[hidden email]<mailto:[hidden email]>>
> Reply-To: Terry MacDonald
> <[hidden email]<mailto:[hidden email]>>
> Date: Sunday, June 7, 2015 at 3:08 AM
> To:
> "[hidden email]<mailto:[hidden email]>"
> <[hidden email]<mailto:[hidden email]>>
> Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options
>
> Based on the guidance on the STIX website, you would two TLP marking
> structures, with one as TLP green applying to the main part of the doc, and
> then use an XPath selector to assign the other bits you want to be TLP red.
> I've listed an example below but it may be a little wrong as my Xpath
> querying is pretty poor :).
>
> <stix:Handling>
>   <marking:Marking>
>     <marking:Controlled_Structure>//node() |
> //@*</marking:Controlled_Structure>
>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
> tlp:color="GREEN" />
>   </marking:Marking>
>   <marking:Marking>
>
> <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node()
> |
> ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
> tlp:color="RED" />
>   </marking:Marking>
> </stix:Handling>
>
>
> The other option is to just make the indicator completely TLP red, or TLP
> green rather than applying the Marking Structure at such a low level. I
> don't recall seeing any objects with partial TLP marking at all so far in
> the wild.
>
>
> Cheers
>
> Terry MacDonald | STIX, TAXII, CybOX Consultant
>
> M: +61-407-203-026
> E: [hidden email]<mailto:[hidden email]>
> W: www.threatloop.com<https://www.threatloop.com>
>
> [https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>
>
> Disclaimer: The opinions expressed within this email do not represent the
> sentiment of any other party except my own. My views do not necessarily
> reflect those of my employers.
>
> On 7 June 2015 at 12:48, Dave Dittrich
> <[hidden email]<mailto:[hidden email]>> wrote:
> Signed PGP part
> On 6/6/15 5:56 PM, Terry MacDonald wrote:
>
>> Dave - 'TLP' description is handled by the Handling/DataMarking Object
> s
>> described here:
>> http://stixproject.github.io/documentation/concepts/data-markings/ . B
> ret
>> was just putting in text into the descriptions as a quick demonstratio
> n of
>> the sort of data that could go into the fields. I think it was a quick
>> sketch to help us decide which path to go down.
>
> OK, but that reference still doesn't answer my concern.
> How does this (modified from the example on the page above):
>
> <stix:Handling>
>   <marking:Marking>
>     <marking:Controlled_Structure>//node() |
> //@*</marking:Controlled_Structure>
>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
> tlp:color="RED" />
>   </marking:Marking>
> </stix:Handling>
>
> Map to one (and only one, and the *right* one) of these:
>
> [ "some interesting TLP green text in English”,
> "some interesting TLP red text in English”]
>
> Am I clearly illustrating the problem?
>
> Dave
>
>> On 7 June 2015 at 10:18, Dave Dittrich
>> <[hidden email]<mailto:[hidden email]>> wro
> te:
>>
>> Just a quick thought (that's all I have time for ATM).
>>
>> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>>       "descriptions" : [
>>>>>           "some interesting TLP green text in English”,
>>>>>           "some interesting TLP red text in English”
>>>>>       ]
>>
>> I think you need more structure here. How is a program to
>> know how to filter or insert *just* the text for a particular
>> TLP level? Does the text have to include "TLP green" and
>> be selected based on a regular expression? What if someone
>> inserts "TLP_GREEN" or "TLP Green" in the text? If it
>> isn't easy to parse and select with a program, it won't
>> make our lives much easier.
>>
>>>
>>
>
> --
> Dave Dittrich
> [hidden email]<mailto:[hidden email]>
> http://staff.washington.edu/dittrich
>
> PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
> Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
>
>
JA
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

JA
In reply to this post by Terry MacDonald
what we would need ultimately is being able to track the context,
including the Information_Source, meaning <Entities> and their @roles
(e.g.: creator, aggregator, distributor) for each objects, and their
characteristics-values couples. But that, IMHO, would require more
maturity.

2015-06-08 10:33 GMT+03:00 Terry MacDonald <[hidden email]>:

> Bret,
>
> Just to clarify, are you proposing a model where there are some 'well known'
> GUIDs that refer to the TLP model definitions (which explain the rules), and
> everyone uses those same shared definitions to identify how they want
> specific data handled? If so I quite like that idea.
>
> Combined with the inheritance idea it is quite powerful and I like it. But I
> do see a couple of issues with it...
>
> 1. How do we cope with by reference relationships with inheritance?
>
> When observables are defined 'inside' an indicator it is easy to see the
> inheritance, so that if the indicator is assigned a TLP RED then it's easy
> to see that the observables will be too. When the observables are separate,
> and the Indicator includes them by reference, then this gets a bit tricker.
> We could mandate that the the inheritance doesn't propagate through
> references, meaning that any referenced objects need to either explicitly
> labelled or covered by the respective inheritance applied higher 'up the
> tree'.
>
> 2. STIX Package is now just a container
>
> With STIXPackage now being a container, and effectively only used to collect
> together some objects to be sent together, we may want to determine where we
> want the root of the inheritance to begin. We could in the future say that
> the inheritance has to be applied to the top-level objects within the the
> STIX package. In this way we could actually send two separate reports with
> separate objects in a single STIX package and be sure that each objects are
> labelled appropriately. Or we could say that the inheritance is exactly as
> it is now - with the ability of planting a blanket TLP level across the STIX
> package, effectively assuming that two separate reports will be send in two
> separate STIX Packages.  I'm personally leaning towards the former case,
> which labels the top-level objects, and uses inheritance within each object,
> as that seems to give more flexibility to the STIX package, allowing for the
> streaming use-case that we discussed late last year.
>
> Cheers
>
> Terry MacDonald | STIX, TAXII, CybOX Consultant
>
> M: +61-407-203-026
> E: [hidden email]
> W: www.threatloop.com
>
> Disclaimer: The opinions expressed within this email do not represent the
> sentiment of any other party except my own. My views do not necessarily
> reflect those of my employers.
>
>
> On 8 Jun 2015 7:22 am, "Jordan, Bret" <[hidden email]> wrote:
>>
>> Patrick,
>>
>> Good points, as always.   I think in small niche eco-systems you can
>> manage this headache through external contracts.  However, as you said, this
>> all comes crumbling down when we need to actually share with a broader
>> community - which is the whole purpose - or if you do not fully trust people
>> in your niche eco-system to follow the out-of-band agreements.
>>
>> The other problem I have with xpath markings is they are weird,
>> non-intuitive, and IMHO very cumbersome to implement in code.  And if it is
>> hard to implement in code, people will not do it.
>>
>> This is why I have been pushing for a simple reference design that uses an
>> inheritance model.  Imagine if you could do something like what I spelled
>> out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I
>> am using it here as a place holder for bigger things.
>>
>> Marking 1234
>> TLP Yellow+Extra stuff
>> A bunch of rules and values
>> Marking 5678
>> TLP Red+Extra stuff
>> A bunch of rules and values
>>
>> Six_Package
>> Indictor FOO
>> Marking 1234
>> a bunch of data
>> Description 1
>> some yellow description (note this falls under Marking 1234)
>> Description 2
>> Marking 5678
>> some red description (note this falls under Marking 5678)
>>
>>
>> There are many reasons why I like this, obviously it is easy to
>> understand, easy to implement in code, and offers inheritance.  I could also
>> see a time when markings from certain groups will get to a steady state.
>> This means you could easily reference them over and over and over.
>>
>> Thanks,
>>
>> Bret
>>
>>
>>
>> Bret Jordan CISSP
>> Director of Security Architecture and Standards | Office of the CTO
>> Blue Coat Systems
>> PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
>> "Without cryptography vihv vivc ce xhrnrw, however, the only thing that
>> can not be unscrambled is an egg."
>>
>> On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:
>>
>> [This comment is directed at TLP and Data marking in general.]
>>
>> re:  I don't recall seeing any objects with partial TLP marking at all so
>> far in the wild.
>>
>> Have the real-world use cases/challenges (primarily as artifacts of fusion
>> of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings
>> of same, with divergent TLP definitions and additional constructs like
>> Source “Releasability”.  Haven’t sorted out practical solutions from an
>> implementation perspective.   In our World View we need to retain Source
>> referential integrity, transform TLP mappings, and respect all Data Marking
>> and Handling instructions for all Data and Metadata.  We are currently
>> sticking with the core concept that we will not re-distribute anything we
>> don’t Source directly which works reasonably well in the role of an
>> individual consumer/provider of CTI. However, this work-around does not
>> scale well to Consortiums that need selective dissemination within their
>> community and releasability to other communities, while respecting all
>> original Source Data Markings and passing same on with the data.
>>
>> To paraphrase Dr. Burger (again), the current models don’t seem to align
>> well with his astute observation that TLP Transforms and Data
>> Marking/Handling need to be “Butt Simple”.
>>
>> I posit Terry's simple example below as evidence (in other words try to
>> envision what a large STIX package containing fused intelligence from
>> commingled Sources, with heterogeneous TLPs, and Data Markings would look
>> like relative to “Butt Simple”.
>>
>> Curious if any others are struggling with these challenges.
>>
>>
>> Patrick Maroney
>> Office: (856)983-0001
>> Cell: (609)841-5104
>> Email: [hidden email]
>>
>> From: Terry MacDonald
>> <[hidden email]<mailto:[hidden email]>>
>> Reply-To: Terry MacDonald
>> <[hidden email]<mailto:[hidden email]>>
>> Date: Sunday, June 7, 2015 at 3:08 AM
>> To:
>> "[hidden email]<mailto:[hidden email]>"
>> <[hidden email]<mailto:[hidden email]>>
>> Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options
>>
>> Based on the guidance on the STIX website, you would two TLP marking
>> structures, with one as TLP green applying to the main part of the doc, and
>> then use an XPath selector to assign the other bits you want to be TLP red.
>> I've listed an example below but it may be a little wrong as my Xpath
>> querying is pretty poor :).
>>
>> <stix:Handling>
>>   <marking:Marking>
>>     <marking:Controlled_Structure>//node() |
>> //@*</marking:Controlled_Structure>
>>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
>> tlp:color="GREEN" />
>>   </marking:Marking>
>>   <marking:Marking>
>>
>> <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node()
>> |
>> ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
>>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
>> tlp:color="RED" />
>>   </marking:Marking>
>> </stix:Handling>
>>
>>
>> The other option is to just make the indicator completely TLP red, or TLP
>> green rather than applying the Marking Structure at such a low level. I
>> don't recall seeing any objects with partial TLP marking at all so far in
>> the wild.
>>
>>
>> Cheers
>>
>> Terry MacDonald | STIX, TAXII, CybOX Consultant
>>
>> M: +61-407-203-026
>> E: [hidden email]<mailto:[hidden email]>
>> W: www.threatloop.com<https://www.threatloop.com>
>>
>>
>> [https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>
>>
>> Disclaimer: The opinions expressed within this email do not represent the
>> sentiment of any other party except my own. My views do not necessarily
>> reflect those of my employers.
>>
>> On 7 June 2015 at 12:48, Dave Dittrich
>> <[hidden email]<mailto:[hidden email]>> wrote:
>> Signed PGP part
>> On 6/6/15 5:56 PM, Terry MacDonald wrote:
>>
>> > Dave - 'TLP' description is handled by the Handling/DataMarking Object
>> s
>> > described here:
>> > http://stixproject.github.io/documentation/concepts/data-markings/ . B
>> ret
>> > was just putting in text into the descriptions as a quick demonstratio
>> n of
>> > the sort of data that could go into the fields. I think it was a quick
>> > sketch to help us decide which path to go down.
>>
>> OK, but that reference still doesn't answer my concern.
>> How does this (modified from the example on the page above):
>>
>> <stix:Handling>
>>   <marking:Marking>
>>     <marking:Controlled_Structure>//node() |
>> //@*</marking:Controlled_Structure>
>>     <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
>> tlp:color="RED" />
>>   </marking:Marking>
>> </stix:Handling>
>>
>> Map to one (and only one, and the *right* one) of these:
>>
>> [ "some interesting TLP green text in English”,
>> "some interesting TLP red text in English”]
>>
>> Am I clearly illustrating the problem?
>>
>> Dave
>>
>> > On 7 June 2015 at 10:18, Dave Dittrich
>> > <[hidden email]<mailto:[hidden email]>> wro
>> te:
>> >
>> > Just a quick thought (that's all I have time for ATM).
>> >
>> > On 6/6/15 9:40 AM, Jordan, Bret wrote:
>> >>>>       "descriptions" : [
>> >>>>           "some interesting TLP green text in English”,
>> >>>>           "some interesting TLP red text in English”
>> >>>>       ]
>> >
>> > I think you need more structure here. How is a program to
>> > know how to filter or insert *just* the text for a particular
>> > TLP level? Does the text have to include "TLP green" and
>> > be selected based on a regular expression? What if someone
>> > inserts "TLP_GREEN" or "TLP Green" in the text? If it
>> > isn't easy to parse and select with a program, it won't
>> > make our lives much easier.
>> >
>> >>
>> >
>>
>> --
>> Dave Dittrich
>> [hidden email]<mailto:[hidden email]>
>> http://staff.washington.edu/dittrich
>>
>> PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
>> Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
In reply to this post by Terry MacDonald
Terry, 

For item #1, I think your idea of having inheritance NOT flow through an reference is the way to go.  The reason why is, I may send the observable and say it is TLP GREEN.  You may reference the observable with extra context and say your extra context is TLP RED.

For item #2, I was also thinking that top level objects would get a field called “marking” or marking_id”.  What ever we want to call it.  And ideally it would be the first object in the container.  This would allow parsers to read it first and decide if they should go on.  

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 8, 2015, at 01:33, Terry MacDonald <[hidden email]> wrote:

Bret,

Just to clarify, are you proposing a model where there are some 'well known' GUIDs that refer to the TLP model definitions (which explain the rules), and everyone uses those same shared definitions to identify how they want specific data handled? If so I quite like that idea.

Combined with the inheritance idea it is quite powerful and I like it. But I do see a couple of issues with it...

1. How do we cope with by reference relationships with inheritance?

When observables are defined 'inside' an indicator it is easy to see the inheritance, so that if the indicator is assigned a TLP RED then it's easy to see that the observables will be too. When the observables are separate, and the Indicator includes them by reference, then this gets a bit tricker. We could mandate that the the inheritance doesn't propagate through references, meaning that any referenced objects need to either explicitly labelled or covered by the respective inheritance applied higher 'up the tree'.

2. STIX Package is now just a container

With STIXPackage now being a container, and effectively only used to collect together some objects to be sent together, we may want to determine where we want the root of the inheritance to begin. We could in the future say that the inheritance has to be applied to the top-level objects within the the STIX package. In this way we could actually send two separate reports with separate objects in a single STIX package and be sure that each objects are labelled appropriately. Or we could say that the inheritance is exactly as it is now - with the ability of planting a blanket TLP level across the STIX package, effectively assuming that two separate reports will be send in two separate STIX Packages.  I'm personally leaning towards the former case, which labels the top-level objects, and uses inheritance within each object, as that seems to give more flexibility to the STIX package, allowing for the streaming use-case that we discussed late last year.

Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: [hidden email]
W: www.threatloop.com

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.


On 8 Jun 2015 7:22 am, "Jordan, Bret" <[hidden email]> wrote:
Patrick,

Good points, as always.   I think in small niche eco-systems you can manage this headache through external contracts.  However, as you said, this all comes crumbling down when we need to actually share with a broader community - which is the whole purpose - or if you do not fully trust people in your niche eco-system to follow the out-of-band agreements.  

The other problem I have with xpath markings is they are weird, non-intuitive, and IMHO very cumbersome to implement in code.  And if it is hard to implement in code, people will not do it.  

This is why I have been pushing for a simple reference design that uses an inheritance model.  Imagine if you could do something like what I spelled out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I am using it here as a place holder for bigger things.

Marking 1234 
TLP Yellow+Extra stuff
A bunch of rules and values
Marking 5678
TLP Red+Extra stuff
A bunch of rules and values

Six_Package
Indictor FOO
Marking 1234
a bunch of data
Description 1
some yellow description (note this falls under Marking 1234)
Description 2
Marking 5678
some red description (note this falls under Marking 5678)


There are many reasons why I like this, obviously it is easy to understand, easy to implement in code, and offers inheritance.  I could also see a time when markings from certain groups will get to a steady state.  This means you could easily reference them over and over and over.

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:

[This comment is directed at TLP and Data marking in general.]

re:  I don't recall seeing any objects with partial TLP marking at all so far in the wild.

Have the real-world use cases/challenges (primarily as artifacts of fusion of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings of same, with divergent TLP definitions and additional constructs like Source “Releasability”.  Haven’t sorted out practical solutions from an implementation perspective.   In our World View we need to retain Source referential integrity, transform TLP mappings, and respect all Data Marking and Handling instructions for all Data and Metadata.  We are currently sticking with the core concept that we will not re-distribute anything we don’t Source directly which works reasonably well in the role of an individual consumer/provider of CTI. However, this work-around does not scale well to Consortiums that need selective dissemination within their community and releasability to other communities, while respecting all original Source Data Markings and passing same on with the data.

To paraphrase Dr. Burger (again), the current models don’t seem to align well with his astute observation that TLP Transforms and Data Marking/Handling need to be “Butt Simple”.

I posit Terry's simple example below as evidence (in other words try to envision what a large STIX package containing fused intelligence from commingled Sources, with heterogeneous TLPs, and Data Markings would look like relative to “Butt Simple”.

Curious if any others are struggling with these challenges.


Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
Email: [hidden email]

From: Terry MacDonald <[hidden email]<[hidden email]>>
Reply-To: Terry MacDonald <[hidden email]<[hidden email]>>
Date: Sunday, June 7, 2015 at 3:08 AM
To: "[hidden email]<[hidden email]>" <[hidden email]<[hidden email]>>
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
  </marking:Marking>
  <marking:Marking>
    <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>


The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: <a href="tel:%2B61-407-203-026" value="+61407203026" target="_blank" class="">+61-407-203-026
E: [hidden email]<[hidden email]>
W: www.threatloop.com<https://www.threatloop.com>

[https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich <[hidden email]<[hidden email]>> wrote:
Signed PGP part
On 6/6/15 5:56 PM, Terry MacDonald wrote:

> Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
> described here:
> http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
> was just putting in text into the descriptions as a quick demonstratio
n of
> the sort of data that could go into the fields. I think it was a quick
> sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
  <marking:Marking>
    <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
    <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
  </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

> On 7 June 2015 at 10:18, Dave Dittrich <[hidden email]<[hidden email]>> wro
te:

>
> Just a quick thought (that's all I have time for ATM).
>
> On 6/6/15 9:40 AM, Jordan, Bret wrote:
>>>>       "descriptions" : [
>>>>           "some interesting TLP green text in English”,
>>>>           "some interesting TLP red text in English”
>>>>       ]
>
> I think you need more structure here. How is a program to
> know how to filter or insert *just* the text for a particular
> TLP level? Does the text have to include "TLP green" and
> be selected based on a regular expression? What if someone
> inserts "TLP_GREEN" or "TLP Green" in the text? If it
> isn't easy to parse and select with a program, it won't
> make our lives much easier.
>
>>
>

--
Dave Dittrich
[hidden email]<[hidden email]>
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079



signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
In reply to this post by Jason Keirstead
Absolutely.  Thanks for pointing that out.  I will make sure I update that in my code and all future example I send out.  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 8, 2015, at 06:19, Jason Keirstead <[hidden email]> wrote:

Can I suggest instead of having "txt" and "html" for the format, you use standard RFC 6838 mime types (IE "text/plain", "text/html"), as it leaves the door open for future data formats while also avoiding making a STIX-specific enumeration (of which I feel there are already too many in the standard).

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


<graycol.gif>"Jordan, Bret" ---2015/06/05 07:36:33 PM---All, I am working on a JSON binding document for STIX like what I did for TAXII and have a question

From: "Jordan, Bret" <[hidden email]>
To: <[hidden email]>
Date: 2015/06/05 07:36 PM
Subject: [STIX] STIX 1.2 Multiple Descriptions - JSON Options





All,

I am working on a JSON binding document for STIX like what I did for TAXII and have a question about the new 1.2 feature for multiple descriptions and multiple short_descriptions.  Both of these allow for a flag called structuring_format.  My question is how best to represent this in JSON…  

1) One option is to use an array of objects and encod the structuring_format as the key in the key/value pair.  This obviously allows for some very elaborate description options.

{
 "stix_package" : {
   "indicators" : [
     {
"id": "example:indicator-6215fe51-770a-456d-a559-b23519908157",
"descriptions" : [
 {
   "txt" : "some interesting TLP green text",
   "html" : "some interesting TLP green html text"
 },
 {
   "txt" : "some interesting TLP red text",
   "md" : "some interesting TLP red markdown text"
 }
],
     }
   ]
 }
}


2) Another option would be something more simple…  This is obviously a lot easier to consume but is more restrictive as each description would need to be of the same format.  

{
 "stix_package" : {
   "indicators" : [

      {
        "id": "example:indicator-6215fe51-770a-456d-a559-b23519908157”,
“descriptions_structuring_format" : “txt",
"descriptions" : [
 "some interesting TLP green text",
 "some interesting TLP red text"
],
     }
   ]
 }
}



Please let me know which one you prefer or propose a different option…  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
[attachment "signature.asc" deleted by Jason Keirstead/CanEast/IBM]



signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jordan, Bret
In reply to this post by JA
Interesting ideas Jerome…  Can you give some non-markup examples of what you are talking about?  


Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On Jun 8, 2015, at 06:50, Jerome Athias <[hidden email]> wrote:

what we would need ultimately is being able to track the context,
including the Information_Source, meaning <Entities> and their @roles
(e.g.: creator, aggregator, distributor) for each objects, and their
characteristics-values couples. But that, IMHO, would require more
maturity.

2015-06-08 10:33 GMT+03:00 Terry MacDonald <[hidden email]>:
Bret,

Just to clarify, are you proposing a model where there are some 'well known'
GUIDs that refer to the TLP model definitions (which explain the rules), and
everyone uses those same shared definitions to identify how they want
specific data handled? If so I quite like that idea.

Combined with the inheritance idea it is quite powerful and I like it. But I
do see a couple of issues with it...

1. How do we cope with by reference relationships with inheritance?

When observables are defined 'inside' an indicator it is easy to see the
inheritance, so that if the indicator is assigned a TLP RED then it's easy
to see that the observables will be too. When the observables are separate,
and the Indicator includes them by reference, then this gets a bit tricker.
We could mandate that the the inheritance doesn't propagate through
references, meaning that any referenced objects need to either explicitly
labelled or covered by the respective inheritance applied higher 'up the
tree'.

2. STIX Package is now just a container

With STIXPackage now being a container, and effectively only used to collect
together some objects to be sent together, we may want to determine where we
want the root of the inheritance to begin. We could in the future say that
the inheritance has to be applied to the top-level objects within the the
STIX package. In this way we could actually send two separate reports with
separate objects in a single STIX package and be sure that each objects are
labelled appropriately. Or we could say that the inheritance is exactly as
it is now - with the ability of planting a blanket TLP level across the STIX
package, effectively assuming that two separate reports will be send in two
separate STIX Packages.  I'm personally leaning towards the former case,
which labels the top-level objects, and uses inheritance within each object,
as that seems to give more flexibility to the STIX package, allowing for the
streaming use-case that we discussed late last year.

Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: [hidden email]
W: www.threatloop.com

Disclaimer: The opinions expressed within this email do not represent the
sentiment of any other party except my own. My views do not necessarily
reflect those of my employers.


On 8 Jun 2015 7:22 am, "Jordan, Bret" <[hidden email]> wrote:

Patrick,

Good points, as always.   I think in small niche eco-systems you can
manage this headache through external contracts.  However, as you said, this
all comes crumbling down when we need to actually share with a broader
community - which is the whole purpose - or if you do not fully trust people
in your niche eco-system to follow the out-of-band agreements.

The other problem I have with xpath markings is they are weird,
non-intuitive, and IMHO very cumbersome to implement in code.  And if it is
hard to implement in code, people will not do it.

This is why I have been pushing for a simple reference design that uses an
inheritance model.  Imagine if you could do something like what I spelled
out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I
am using it here as a place holder for bigger things.

Marking 1234
TLP Yellow+Extra stuff
A bunch of rules and values
Marking 5678
TLP Red+Extra stuff
A bunch of rules and values

Six_Package
Indictor FOO
Marking 1234
a bunch of data
Description 1
some yellow description (note this falls under Marking 1234)
Description 2
Marking 5678
some red description (note this falls under Marking 5678)


There are many reasons why I like this, obviously it is easy to
understand, easy to implement in code, and offers inheritance.  I could also
see a time when markings from certain groups will get to a steady state.
This means you could easily reference them over and over and over.

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that
can not be unscrambled is an egg."

On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:

[This comment is directed at TLP and Data marking in general.]

re:  I don't recall seeing any objects with partial TLP marking at all so
far in the wild.

Have the real-world use cases/challenges (primarily as artifacts of fusion
of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings
of same, with divergent TLP definitions and additional constructs like
Source “Releasability”.  Haven’t sorted out practical solutions from an
implementation perspective.   In our World View we need to retain Source
referential integrity, transform TLP mappings, and respect all Data Marking
and Handling instructions for all Data and Metadata.  We are currently
sticking with the core concept that we will not re-distribute anything we
don’t Source directly which works reasonably well in the role of an
individual consumer/provider of CTI. However, this work-around does not
scale well to Consortiums that need selective dissemination within their
community and releasability to other communities, while respecting all
original Source Data Markings and passing same on with the data.

To paraphrase Dr. Burger (again), the current models don’t seem to align
well with his astute observation that TLP Transforms and Data
Marking/Handling need to be “Butt Simple”.

I posit Terry's simple example below as evidence (in other words try to
envision what a large STIX package containing fused intelligence from
commingled Sources, with heterogeneous TLPs, and Data Markings would look
like relative to “Butt Simple”.

Curious if any others are struggling with these challenges.


Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104
Email: [hidden email]

From: Terry MacDonald
<[hidden email]<[hidden email]>>
Reply-To: Terry MacDonald
<[hidden email]<[hidden email]>>
Date: Sunday, June 7, 2015 at 3:08 AM
To:
"[hidden email]<[hidden email]>"
<[hidden email]<[hidden email]>>
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Based on the guidance on the STIX website, you would two TLP marking
structures, with one as TLP green applying to the main part of the doc, and
then use an XPath selector to assign the other bits you want to be TLP red.
I've listed an example below but it may be a little wrong as my Xpath
querying is pretty poor :).

<stix:Handling>
 <marking:Marking>
   <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
   <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="GREEN" />
 </marking:Marking>
 <marking:Marking>

<marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node()
|
../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
   <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
 </marking:Marking>
</stix:Handling>


The other option is to just make the indicator completely TLP red, or TLP
green rather than applying the Marking Structure at such a low level. I
don't recall seeing any objects with partial TLP marking at all so far in
the wild.


Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: [hidden email]<[hidden email]>
W: www.threatloop.com<https://www.threatloop.com>


[https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>

Disclaimer: The opinions expressed within this email do not represent the
sentiment of any other party except my own. My views do not necessarily
reflect those of my employers.

On 7 June 2015 at 12:48, Dave Dittrich
<[hidden email]<[hidden email]>> wrote:
Signed PGP part
On 6/6/15 5:56 PM, Terry MacDonald wrote:

Dave - 'TLP' description is handled by the Handling/DataMarking Object
s
described here:
http://stixproject.github.io/documentation/concepts/data-markings/ . B
ret
was just putting in text into the descriptions as a quick demonstratio
n of
the sort of data that could go into the fields. I think it was a quick
sketch to help us decide which path to go down.

OK, but that reference still doesn't answer my concern.
How does this (modified from the example on the page above):

<stix:Handling>
 <marking:Marking>
   <marking:Controlled_Structure>//node() |
//@*</marking:Controlled_Structure>
   <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
tlp:color="RED" />
 </marking:Marking>
</stix:Handling>

Map to one (and only one, and the *right* one) of these:

[ "some interesting TLP green text in English”,
"some interesting TLP red text in English”]

Am I clearly illustrating the problem?

Dave

On 7 June 2015 at 10:18, Dave Dittrich
<[hidden email]<[hidden email]>> wro
te:

Just a quick thought (that's all I have time for ATM).

On 6/6/15 9:40 AM, Jordan, Bret wrote:
     "descriptions" : [
         "some interesting TLP green text in English”,
         "some interesting TLP red text in English”
     ]

I think you need more structure here. How is a program to
know how to filter or insert *just* the text for a particular
TLP level? Does the text have to include "TLP green" and
be selected based on a regular expression? What if someone
inserts "TLP_GREEN" or "TLP Green" in the text? If it
isn't easy to parse and select with a program, it won't
make our lives much easier.




--
Dave Dittrich
[hidden email]<[hidden email]>
http://staff.washington.edu/dittrich

PGP key:     http://staff.washington.edu/dittrich/pgpkey.txt
Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079





signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options

Jason Keirstead
In reply to this post by Terry MacDonald

There is something I have never really understood about the TLP marking mechanisms in the context of STIX, and that is how MITRE intended them to be used in combination with TAXII. I feel this needs some good thought, and also work on the standard, because this is where the TLP marking concept breaks down as soon as you breach that "nieche ecosystem" barrier.

To be honest, I have always felt like TLP is woefully insufficient for the job at hand. The problem with TLP markings in the standard is they only make sense in the context of "an organization". They don't make sense in the context of multiple, nested levels of organizations, unless there is some agreement for all levels of those organization that "this color means X". If MITRE is just assuming these agreements will exist, and not make them part of the standard itself, I am not sure how tools are supposed to work with the data reliably.

Here is an example to illustrate the problem (I love examples)..

- Assume there is some widely used future threat sharing platform that leverages STIX/TAXII (call it "the platform" from now on)

- Assume that Corporation A and Corporation B are both members of the platform, and each has many consumer groups and producer groups of STIX data (many accounts)

- Assume that Corporation A and Corporation B are also both members of some industry threat sharing group (call it Group A) that uses "the platform", and that there is a sub-community for this group (it has it's own TAXII feed)

There are now a slew of questions/unknowns, all of which would need to be understood to write the tools to work with "the platform", as well as to write "the platform" itself.

- What does TLP:RED mean when Corporation A shares indicators using them? Does it only stay with the producer who posted it? Or does it go out to all of Corporation A?

- What does TLP:AMBER mean when Corporation A shares indicators using them? Does it only stay with the Corporation A or does it go out to all of Group A?

- What does TLP:GREEN mean when Corporation A shares indicators using them? Does it only stay with the Corporation A or does it go out to all of Group A?

- What happens if Corporation A shares a STIX document in it that has one observable flagged TLP:RED in it, but many others that are flagged TLP:GREEN. When this document is consumed by Corporation B, is that observable just not present at all? Is it present but empty?

- In either case, does it keep the same ID if it the document doesn't contain the original submitted data? Should the consumer of a STIX package know "there is more information here, but you are not allowed to see it" ? Should they be alerted that they have an incomplete indicator?

Anyway - you can see where I am going here. I find that the TLP mechanism is not even close to fully fleshed out in the standard.

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Inactive hide details for Terry MacDonald ---2015/06/08 04:34:28 AM---Bret, Just to clarify, are you proposing a model where thTerry MacDonald ---2015/06/08 04:34:28 AM---Bret, Just to clarify, are you proposing a model where there are some 'well

From: Terry MacDonald <[hidden email]>
To: <[hidden email]>
Date: 2015/06/08 04:34 AM
Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options





Bret,

Just to clarify, are you proposing a model where there are some 'well known' GUIDs that refer to the TLP model definitions (which explain the rules), and everyone uses those same shared definitions to identify how they want specific data handled? If so I quite like that idea.

Combined with the inheritance idea it is quite powerful and I like it. But I do see a couple of issues with it...

1. How do we cope with by reference relationships with inheritance?

When observables are defined 'inside' an indicator it is easy to see the inheritance, so that if the indicator is assigned a TLP RED then it's easy to see that the observables will be too. When the observables are separate, and the Indicator includes them by reference, then this gets a bit tricker. We could mandate that the the inheritance doesn't propagate through references, meaning that any referenced objects need to either explicitly labelled or covered by the respective inheritance applied higher 'up the tree'.

2. STIX Package is now just a container

With STIXPackage now being a container, and effectively only used to collect together some objects to be sent together, we may want to determine where we want the root of the inheritance to begin. We could in the future say that the inheritance has to be applied to the top-level objects within the the STIX package. In this way we could actually send two separate reports with separate objects in a single STIX package and be sure that each objects are labelled appropriately. Or we could say that the inheritance is exactly as it is now - with the ability of planting a blanket TLP level across the STIX package, effectively assuming that two separate reports will be send in two separate STIX Packages.  I'm personally leaning towards the former case, which labels the top-level objects, and uses inheritance within each object, as that seems to give more flexibility to the STIX package, allowing for the streaming use-case that we discussed late last year.

Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E:
[hidden email]
W:
www.threatloop.com

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 8 Jun 2015 7:22 am, "Jordan, Bret" <[hidden email]> wrote:

    Patrick,

    Good points, as always.   I think in small niche eco-systems you can manage this headache through external contracts.  However, as you said, this all comes crumbling down when we need to actually share with a broader community - which is the whole purpose - or if you do not fully trust people in your niche eco-system to follow the out-of-band agreements.  

    The other problem I have with xpath markings is they are weird, non-intuitive, and IMHO very cumbersome to implement in code.  And if it is hard to implement in code, people will not do it.  

    This is why I have been pushing for a simple reference design that uses an inheritance model.  Imagine if you could do something like what I spelled out in JSON yesterday.  Yes, I know TLP is weak and overly simplistic, but I am using it here as a place holder for bigger things.

    Marking 1234 
    TLP Yellow+Extra stuff
    A bunch of rules and values
    Marking 5678
    TLP Red+Extra stuff
    A bunch of rules and values

    Six_Package
    Indictor FOO
    Marking 1234
    a bunch of data
    Description 1
    some yellow description (note this falls under Marking 1234)
    Description 2
    Marking 5678
    some red description (note this falls under Marking 5678)


    There are many reasons why I like this, obviously it is easy to understand, easy to implement in code, and offers inheritance.  I could also see a time when markings from certain groups will get to a steady state.  This means you could easily reference them over and over and over.

    Thanks,

    Bret



    Bret Jordan CISSP
    Director of Security Architecture and Standards | Office of the CTO
    Blue Coat Systems
    PGP Fingerprint: 62A6 5999 0F7D 0D61 4C66 D59C 2DB5 111D 63BC A303
    "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
      On Jun 7, 2015, at 06:51, Patrick Maroney <[hidden email]> wrote:

      [This comment is directed at TLP and Data marking in general.]

      re:  I don't recall seeing any objects with partial TLP marking at all so far in the wild.


      Have the real-world use cases/challenges (primarily as artifacts of fusion of CTI from Multiple Sources/Communities/Sensitivities, Multiple Sightings of same, with divergent TLP definitions and additional constructs like Source “Releasability”.  Haven’t sorted out practical solutions from an implementation perspective.   In our World View we need to retain Source referential integrity, transform TLP mappings, and respect all Data Marking and Handling instructions for all Data and Metadata.  We are currently sticking with the core concept that we will not re-distribute anything we don’t Source directly which works reasonably well in the role of an individual consumer/provider of CTI. However, this work-around does not scale well to Consortiums that need selective dissemination within their community and releasability to other communities, while respecting all original Source Data Markings and passing same on with the data.


      To paraphrase Dr. Burger (again), the current models don’t seem to align well with his astute observation that TLP Transforms and Data Marking/Handling need to be “Butt Simple”.


      I posit Terry's simple example below as evidence (in other words try to envision what a large STIX package containing fused intelligence from commingled Sources, with heterogeneous TLPs, and Data Markings would look like relative to “Butt Simple”.


      Curious if any others are struggling with these challenges.



      Patrick Maroney
      Office: (856)983-0001
      Cell: (609)841-5104
      Email: 
      [hidden email]

      From: Terry MacDonald <
      [hidden email]<[hidden email]>>
      Reply-To: Terry MacDonald <
      [hidden email]<[hidden email]>>
      Date: Sunday, June 7, 2015 at 3:08 AM
      To: "
      [hidden email]<[hidden email]>" <[hidden email]<[hidden email]>>
      Subject: Re: [STIX] STIX 1.2 Multiple Descriptions - JSON Options


      Based on the guidance on the STIX website, you would two TLP marking structures, with one as TLP green applying to the main part of the doc, and then use an XPath selector to assign the other bits you want to be TLP red. I've listed an example below but it may be a little wrong as my Xpath querying is pretty poor :).


      <stix:Handling>
        <marking:Marking>
          <marking:Controlled_Structure>//node() |
      //@*</marking:Controlled_Structure>
          <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
      tlp:color="GREEN" />
        </marking:Marking>
        <marking:Marking>
          <marking:Controlled_Structure>../../../indicator:Description[last()]/descendant-or-self::node() | ../../../indicator:Description[last()]/descendant-or-self::node()/@*</marking:Controlled_Structure>
          <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
      tlp:color="RED" />
        </marking:Marking>
      </stix:Handling>



      The other option is to just make the indicator completely TLP red, or TLP green rather than applying the Marking Structure at such a low level. I don't recall seeing any objects with partial TLP marking at all so far in the wild.



      Cheers


      Terry MacDonald | STIX, TAXII, CybOX Consultant


      M:
      <a href="tel:%2B61-407-203-026" target="_blank">+61-407-203-026
      E: 
      [hidden email]<[hidden email]>
      W: 
      www.threatloop.com<https://www.threatloop.com>

      [
      https://docs.google.com/uc?export=download&id=0BylsXX2Xx8A4bDdwWjNnaFNwcHM&revid=0BylsXX2Xx8A4TlRQNldhaU1qZ0ZPWitqa2ZyNG0yOUxpN0RZPQ]<https://www.threatloop.com>

      Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.


      On 7 June 2015 at 12:48, Dave Dittrich <
      [hidden email]<[hidden email]>> wrote:
      Signed PGP part

      On 6/6/15 5:56 PM, Terry MacDonald wrote:

      > Dave - 'TLP' description is handled by the Handling/DataMarking Object
      s
      > described here:
      http://stixproject.github.io/documentation/concepts/data-markings/ . B
      ret
      > was just putting in text into the descriptions as a quick demonstratio
      n of
      > the sort of data that could go into the fields. I think it was a quick
      > sketch to help us decide which path to go down.

      OK, but that reference still doesn't answer my concern.
      How does this (modified from the example on the page above):

      <stix:Handling>
        <marking:Marking>
          <marking:Controlled_Structure>//node() |
      //@*</marking:Controlled_Structure>
          <marking:Marking_Structure xsi:type="tlp:TLPMarkingStructureType"
      tlp:color="RED" />
        </marking:Marking>
      </stix:Handling>

      Map to one (and only one, and the *right* one) of these:

      [ "some interesting TLP green text in English”,
      "some interesting TLP red text in English”]

      Am I clearly illustrating the problem?

      Dave

      > On 7 June 2015 at 10:18, Dave Dittrich <
      [hidden email]<[hidden email]>> wro
      te:
      >
      > Just a quick thought (that's all I have time for ATM).
      >
      > On 6/6/15 9:40 AM, Jordan, Bret wrote:
      >>>>       "descriptions" : [
      >>>>           "some interesting TLP green text in English”,
      >>>>           "some interesting TLP red text in English”
      >>>>       ]
      >
      > I think you need more structure here. How is a program to
      > know how to filter or insert *just* the text for a particular
      > TLP level? Does the text have to include "TLP green" and
      > be selected based on a regular expression? What if someone
      > inserts "TLP_GREEN" or "TLP Green" in the text? If it
      > isn't easy to parse and select with a program, it won't
      > make our lives much easier.
      >
      >>
      >

      --
      Dave Dittrich

      [hidden email]<[hidden email]>
      http://staff.washington.edu/dittrich

      PGP key:     
      http://staff.washington.edu/dittrich/pgpkey.txt
      Fingerprint: 097B 4DCB BF16 E1D8 A06C  7512 A751 C80A D15E E079


12