Partial Matching in OVAL

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Partial Matching in OVAL

Michael Chisholm
Hello,

The following was prepared to address what we call "partial matches" in
OVAL.  We hope this addresses peoples' continuing confusion over when to
create items (e.g. see the recent thread with the subject "When to
create an item?").  It is not only a presentation and discussion of the
idea and surrounding issues, but also a proposal to remove them from the
language if the community does not see a need for them.  So any and all
feedback on this is welcome!

Thanks,
Andy



What Is a Partial Match?

Simply put, a partial match is an item with status equal to “does not
exist”.  Normally, if no matches for an object are found on a system,
you would just not create any items.  But under some circumstances, it
was envisioned that you might want to create some items anyway, with
partial information.  The content of these items is left completely up
to interpreter authors, they would be purely optional and informational,
and they would not affect the evaluation of a test.  Most likely,
partial information would come in the form of a set of item entities
which matched some but not all of the object entities.  The containing
item would receive a status value of “does not exist” to indicate that
the item did not actually match the object.


Why Have Partial Matches?

The purpose of a partial match is to assist content authors and
consumers of OVAL content in determining why a search did not find the
expected items.  An example which is frequently used relates to file
searches: if an object was intended to test for the existence of one
particular file, and the file did not exist but the directory did, then
a partial match item could be created which contained the directory but
not the filename, which would indicate that fact.


What’s the Problem?

The aforementioned partial match idea sounds very well and good, but
there is one feature of OVAL which turns out to be incompatible with it:
the “all_exist” value of the “check_existence” attribute of tests. This
value as defined causes interpreters to use items whose status is “does
not exist”.  In particular, if all items collected for an object have a
status of “exists”, then the existence check “all_exist” is satisfied.
If at least one item has a status of “does not exist”, it is not.  This
makes partial match items significant to the evaluation of a test,
directly contradicting the original intent which was that partial
matches are purely optional and informational and do not affect test
evaluation.  So “all_exist” is unusable if we wish to keep partial
matches optional.

What if we decided to keep that enumerator and make partial matches
significant?  Could that work?  Probably not.  The reasons for this go a
bit deeper: “all_exist” really is trying to get at a content author’s
intentions, which is essentially mind reading.  It supposes you can
glean from an object that the author intends that a particular set of
items /should/ exist, which means you can determine whether they all
actually did exist, and therefore make the “all_exist” determination.
But gleaning intent from an object is fraught with difficulties.  It is
tempting to think that if all object entities used operation=”equals”
with no variable references, then the intent was that a single item with
entity values equal to the object entity values should exist.  But in
general it is impossible to tell whether only one item would have
matched, since items are identified by all of their entity values [1],
not just the values of entities shared with the object.  So if you found
a matching item, you wouldn’t be able to tell whether you got all or
only one of many items that should have matched.  Anyway, are there
other situations in which an author’s intent may be considered to be
evident in the object?  What if we allowed variable references where
each variable has exactly one value and var_check is “all”?
Multi-valued variables with var_check=”at least one”?  Operation=”not
equal” and var_check=”none satisfy” (which means none of the variable
values are not-equal to the item entity value, or in other words,
they’re all equal)?  Even if we decide that only the first style of
object, with all operations being “equals” and no variable references,
constitutes evident author intent, do we really want to complicate the
specification with this notion of an object “style” and author intent?
Do we want the validity of other parts of a definition to depend on this?

This seems to add unnecessary complexity to the language.  Unless the
“all_exist” enumerator is really needed, it is probably preferable to
avoid these issues and keep the specification simple.


What Should We Do About It?

If any matching items are found, let the only valid conclusion be that
some items were found, nothing more.  To remain backward-compatible, the
“all_exist” existence check should be defined to pass whenever an item
with a status of “exists” is found, and ignore any other items.  This
makes it equivalent to “at_least_one_exists”.  Partial match items, if
they are deemed to be necessary, should be truly ignored during
definition evaluation.  Perhaps in a future version of OVAL, “all_exist”
can be deprecated.  Other miscellaneous required changes to the language
include:

• The “DE” column from all tables in the ExistenceEnumeration schema
documentation should be removed, and the tables re-evaluated to ensure
they still make sense.

• Schematron rules which enforce properties of items should be pared
down to the minimum, when the item’s status is “does not exist”, to give
content authors maximum control over the form of their partial matches.
  There is at least one we know of which should be removed, associated
with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
partial match item entities with status=”does not exist” from having a
value.

In our experience developing ovaldi, we found that partial match support
added a significant amount of complexity to the code.  Also, based on a
limited tool survey, we found that partial matching is not widely
supported.  If partial matching is made truly optional, then tool
implementers are free to add as little or as much complexity to their
code as they like to support it.  But if it is not deemed to be
necessary, partial matching should be removed from the language, to
simplify the language and avoid future problems.


End Notes:

1. See the OVAL Language Specification, section 5.2.4.2.

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
Just to make this somewhat more concrete, if you have the following file_object and the file "mylog.txt" doesn't exist in "/var/log".

<file_object id="oval:test:obj:1">
  <path>/var/log</path>
  <filename>mylog.txt</filename>
</file_object>

And your tool supports partial matching, your system-characteristics might look something like the following.

<oval_system_characteristics>
...
  <collected_objects>
    <object flag="does not exist" id="oval:test:obj:1">
      <reference item_ref="1"/>
    </object>
  </collected_objects>

  <system_data>
    <file_item id="1" status="does not exist">
      <path status="exists">/var/log</path>
      <filename status="does not exist">mylog.txt</filename>
    </file_item>
  </system_data>
</oval_system_characteristics>

Are there vendors who have added support for partial matching in their tools?  If so, do you have any thoughts on this?  Did you run into challenges implementing it?  Are your customers asking for this type of capability?

Content authors, do you see this as a valuable feature in OVAL?  Or, is this something that you could live without?

I should also mention that rather than using partial matching, tools could utilize OVAL Messages to convey this type of information such as in the following example.  Of course, the use of OVAL Messages is completely optional in the OVAL Language.

<oval_system_characteristics>
...
  <collected_objects>
    <object flag="does not exist" id="oval:test:obj:1">
         <oval-sc:message>The file mylog.txt was not found in /var/log.</oval-sc:message>
    </object>
  </collected_objects>
  <system_data/>
</oval_system_characteristics>

Thanks,

Danny

>-----Original Message-----
>From: Michael Chisholm [mailto:[hidden email]]
>Sent: Tuesday, July 03, 2012 5:17 PM
>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>Subject: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>
>Hello,
>
>The following was prepared to address what we call "partial matches" in
>OVAL.  We hope this addresses peoples' continuing confusion over when to
>create items (e.g. see the recent thread with the subject "When to
>create an item?").  It is not only a presentation and discussion of the
>idea and surrounding issues, but also a proposal to remove them from the
>language if the community does not see a need for them.  So any and all
>feedback on this is welcome!
>
>Thanks,
>Andy
>
>
>
>What Is a Partial Match?
>
>Simply put, a partial match is an item with status equal to "does not
>exist".  Normally, if no matches for an object are found on a system,
>you would just not create any items.  But under some circumstances, it
>was envisioned that you might want to create some items anyway, with
>partial information.  The content of these items is left completely up
>to interpreter authors, they would be purely optional and informational,
>and they would not affect the evaluation of a test.  Most likely,
>partial information would come in the form of a set of item entities
>which matched some but not all of the object entities.  The containing
>item would receive a status value of "does not exist" to indicate that
>the item did not actually match the object.
>
>
>Why Have Partial Matches?
>
>The purpose of a partial match is to assist content authors and
>consumers of OVAL content in determining why a search did not find the
>expected items.  An example which is frequently used relates to file
>searches: if an object was intended to test for the existence of one
>particular file, and the file did not exist but the directory did, then
>a partial match item could be created which contained the directory but
>not the filename, which would indicate that fact.
>
>
>What's the Problem?
>
>The aforementioned partial match idea sounds very well and good, but
>there is one feature of OVAL which turns out to be incompatible with it:
>the "all_exist" value of the "check_existence" attribute of tests. This
>value as defined causes interpreters to use items whose status is "does
>not exist".  In particular, if all items collected for an object have a
>status of "exists", then the existence check "all_exist" is satisfied.
>If at least one item has a status of "does not exist", it is not.  This
>makes partial match items significant to the evaluation of a test,
>directly contradicting the original intent which was that partial
>matches are purely optional and informational and do not affect test
>evaluation.  So "all_exist" is unusable if we wish to keep partial
>matches optional.
>
>What if we decided to keep that enumerator and make partial matches
>significant?  Could that work?  Probably not.  The reasons for this go a
>bit deeper: "all_exist" really is trying to get at a content author's
>intentions, which is essentially mind reading.  It supposes you can
>glean from an object that the author intends that a particular set of
>items /should/ exist, which means you can determine whether they all
>actually did exist, and therefore make the "all_exist" determination.
>But gleaning intent from an object is fraught with difficulties.  It is
>tempting to think that if all object entities used operation="equals"
>with no variable references, then the intent was that a single item with
>entity values equal to the object entity values should exist.  But in
>general it is impossible to tell whether only one item would have
>matched, since items are identified by all of their entity values [1],
>not just the values of entities shared with the object.  So if you found
>a matching item, you wouldn't be able to tell whether you got all or
>only one of many items that should have matched.  Anyway, are there
>other situations in which an author's intent may be considered to be
>evident in the object?  What if we allowed variable references where
>each variable has exactly one value and var_check is "all"?
>Multi-valued variables with var_check="at least one"?  Operation="not
>equal" and var_check="none satisfy" (which means none of the variable
>values are not-equal to the item entity value, or in other words,
>they're all equal)?  Even if we decide that only the first style of
>object, with all operations being "equals" and no variable references,
>constitutes evident author intent, do we really want to complicate the
>specification with this notion of an object "style" and author intent?
>Do we want the validity of other parts of a definition to depend on this?
>
>This seems to add unnecessary complexity to the language.  Unless the
>"all_exist" enumerator is really needed, it is probably preferable to
>avoid these issues and keep the specification simple.
>
>
>What Should We Do About It?
>
>If any matching items are found, let the only valid conclusion be that
>some items were found, nothing more.  To remain backward-compatible, the
>"all_exist" existence check should be defined to pass whenever an item
>with a status of "exists" is found, and ignore any other items.  This
>makes it equivalent to "at_least_one_exists".  Partial match items, if
>they are deemed to be necessary, should be truly ignored during
>definition evaluation.  Perhaps in a future version of OVAL, "all_exist"
>can be deprecated.  Other miscellaneous required changes to the language
>include:
>
>* The "DE" column from all tables in the ExistenceEnumeration schema
>documentation should be removed, and the tables re-evaluated to ensure
>they still make sense.
>
>* Schematron rules which enforce properties of items should be pared
>down to the minimum, when the item's status is "does not exist", to give
>content authors maximum control over the form of their partial matches.
>  There is at least one we know of which should be removed, associated
>with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
>partial match item entities with status="does not exist" from having a
>value.
>
>In our experience developing ovaldi, we found that partial match support
>added a significant amount of complexity to the code.  Also, based on a
>limited tool survey, we found that partial matching is not widely
>supported.  If partial matching is made truly optional, then tool
>implementers are free to add as little or as much complexity to their
>code as they like to support it.  But if it is not deemed to be
>necessary, partial matching should be removed from the language, to
>simplify the language and avoid future problems.
>
>
>End Notes:
>
>1. See the OVAL Language Specification, section 5.2.4.2.
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST
>in the BODY of the message.  If you have difficulties, write to OVAL-DEVELOPER-
>[hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
I just wanted to follow up with this thread (it's been quite a while) and see if there were any other thoughts about the need for partial matches in the OVAL Language?  

So far, we have only received one response and that was in favor of removing partial matches.

Thanks,

Danny

>-----Original Message-----
>From: Haynes, Dan [mailto:[hidden email]]
>Sent: Wednesday, July 18, 2012 4:19 PM
>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>
>Just to make this somewhat more concrete, if you have the following
>file_object and the file "mylog.txt" doesn't exist in "/var/log".
>
><file_object id="oval:test:obj:1">
>  <path>/var/log</path>
>  <filename>mylog.txt</filename>
></file_object>
>
>And your tool supports partial matching, your system-characteristics might
>look something like the following.
>
><oval_system_characteristics>
>...
>  <collected_objects>
>    <object flag="does not exist" id="oval:test:obj:1">
>      <reference item_ref="1"/>
>    </object>
>  </collected_objects>
>
>  <system_data>
>    <file_item id="1" status="does not exist">
>      <path status="exists">/var/log</path>
>      <filename status="does not exist">mylog.txt</filename>
>    </file_item>
>  </system_data>
></oval_system_characteristics>
>
>Are there vendors who have added support for partial matching in their
>tools?  If so, do you have any thoughts on this?  Did you run into challenges
>implementing it?  Are your customers asking for this type of capability?
>
>Content authors, do you see this as a valuable feature in OVAL?  Or, is this
>something that you could live without?
>
>I should also mention that rather than using partial matching, tools could
>utilize OVAL Messages to convey this type of information such as in the
>following example.  Of course, the use of OVAL Messages is completely
>optional in the OVAL Language.
>
><oval_system_characteristics>
>...
>  <collected_objects>
>    <object flag="does not exist" id="oval:test:obj:1">
>         <oval-sc:message>The file mylog.txt was not found in /var/log.</oval-
>sc:message>
>    </object>
>  </collected_objects>
>  <system_data/>
></oval_system_characteristics>
>
>Thanks,
>
>Danny
>
>>-----Original Message-----
>>From: Michael Chisholm [mailto:[hidden email]]
>>Sent: Tuesday, July 03, 2012 5:17 PM
>>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>Subject: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>>
>>Hello,
>>
>>The following was prepared to address what we call "partial matches" in
>>OVAL.  We hope this addresses peoples' continuing confusion over when to
>>create items (e.g. see the recent thread with the subject "When to
>>create an item?").  It is not only a presentation and discussion of the
>>idea and surrounding issues, but also a proposal to remove them from the
>>language if the community does not see a need for them.  So any and all
>>feedback on this is welcome!
>>
>>Thanks,
>>Andy
>>
>>
>>
>>What Is a Partial Match?
>>
>>Simply put, a partial match is an item with status equal to "does not
>>exist".  Normally, if no matches for an object are found on a system,
>>you would just not create any items.  But under some circumstances, it
>>was envisioned that you might want to create some items anyway, with
>>partial information.  The content of these items is left completely up
>>to interpreter authors, they would be purely optional and informational,
>>and they would not affect the evaluation of a test.  Most likely,
>>partial information would come in the form of a set of item entities
>>which matched some but not all of the object entities.  The containing
>>item would receive a status value of "does not exist" to indicate that
>>the item did not actually match the object.
>>
>>
>>Why Have Partial Matches?
>>
>>The purpose of a partial match is to assist content authors and
>>consumers of OVAL content in determining why a search did not find the
>>expected items.  An example which is frequently used relates to file
>>searches: if an object was intended to test for the existence of one
>>particular file, and the file did not exist but the directory did, then
>>a partial match item could be created which contained the directory but
>>not the filename, which would indicate that fact.
>>
>>
>>What's the Problem?
>>
>>The aforementioned partial match idea sounds very well and good, but
>>there is one feature of OVAL which turns out to be incompatible with it:
>>the "all_exist" value of the "check_existence" attribute of tests. This
>>value as defined causes interpreters to use items whose status is "does
>>not exist".  In particular, if all items collected for an object have a
>>status of "exists", then the existence check "all_exist" is satisfied.
>>If at least one item has a status of "does not exist", it is not.  This
>>makes partial match items significant to the evaluation of a test,
>>directly contradicting the original intent which was that partial
>>matches are purely optional and informational and do not affect test
>>evaluation.  So "all_exist" is unusable if we wish to keep partial
>>matches optional.
>>
>>What if we decided to keep that enumerator and make partial matches
>>significant?  Could that work?  Probably not.  The reasons for this go a
>>bit deeper: "all_exist" really is trying to get at a content author's
>>intentions, which is essentially mind reading.  It supposes you can
>>glean from an object that the author intends that a particular set of
>>items /should/ exist, which means you can determine whether they all
>>actually did exist, and therefore make the "all_exist" determination.
>>But gleaning intent from an object is fraught with difficulties.  It is
>>tempting to think that if all object entities used operation="equals"
>>with no variable references, then the intent was that a single item with
>>entity values equal to the object entity values should exist.  But in
>>general it is impossible to tell whether only one item would have
>>matched, since items are identified by all of their entity values [1],
>>not just the values of entities shared with the object.  So if you found
>>a matching item, you wouldn't be able to tell whether you got all or
>>only one of many items that should have matched.  Anyway, are there
>>other situations in which an author's intent may be considered to be
>>evident in the object?  What if we allowed variable references where
>>each variable has exactly one value and var_check is "all"?
>>Multi-valued variables with var_check="at least one"?  Operation="not
>>equal" and var_check="none satisfy" (which means none of the variable
>>values are not-equal to the item entity value, or in other words,
>>they're all equal)?  Even if we decide that only the first style of
>>object, with all operations being "equals" and no variable references,
>>constitutes evident author intent, do we really want to complicate the
>>specification with this notion of an object "style" and author intent?
>>Do we want the validity of other parts of a definition to depend on this?
>>
>>This seems to add unnecessary complexity to the language.  Unless the
>>"all_exist" enumerator is really needed, it is probably preferable to
>>avoid these issues and keep the specification simple.
>>
>>
>>What Should We Do About It?
>>
>>If any matching items are found, let the only valid conclusion be that
>>some items were found, nothing more.  To remain backward-compatible, the
>>"all_exist" existence check should be defined to pass whenever an item
>>with a status of "exists" is found, and ignore any other items.  This
>>makes it equivalent to "at_least_one_exists".  Partial match items, if
>>they are deemed to be necessary, should be truly ignored during
>>definition evaluation.  Perhaps in a future version of OVAL, "all_exist"
>>can be deprecated.  Other miscellaneous required changes to the language
>>include:
>>
>>* The "DE" column from all tables in the ExistenceEnumeration schema
>>documentation should be removed, and the tables re-evaluated to ensure
>>they still make sense.
>>
>>* Schematron rules which enforce properties of items should be pared
>>down to the minimum, when the item's status is "does not exist", to give
>>content authors maximum control over the form of their partial matches.
>>  There is at least one we know of which should be removed, associated
>>with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
>>partial match item entities with status="does not exist" from having a
>>value.
>>
>>In our experience developing ovaldi, we found that partial match support
>>added a significant amount of complexity to the code.  Also, based on a
>>limited tool survey, we found that partial matching is not widely
>>supported.  If partial matching is made truly optional, then tool
>>implementers are free to add as little or as much complexity to their
>>code as they like to support it.  But if it is not deemed to be
>>necessary, partial matching should be removed from the language, to
>>simplify the language and avoid future problems.
>>
>>
>>End Notes:
>>
>>1. See the OVAL Language Specification, section 5.2.4.2.
>>
>>To unsubscribe, send an email message to [hidden email] with
>>SIGNOFF OVAL-DEVELOPER-LIST
>>in the BODY of the message.  If you have difficulties, write to OVAL-
>DEVELOPER-
>>[hidden email].
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST
>in the BODY of the message.  If you have difficulties, write to OVAL-
>[hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

lutton
I'm not sure I understand the question properly...  I'm thinking that
partial matches cause a schematron violation on the results.  Is that true?

If so, are partial matches really, now, part of the OVAL language?

Or were you perhaps asking about removing the partial matches from OVALdi's
implementation?

Thanks in advance,
Bill L.


-----Original Message-----
From: Haynes, Dan [mailto:[hidden email]]
Sent: Monday, January 28, 2013 10:00 AM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

I just wanted to follow up with this thread (it's been quite a while) and
see if there were any other thoughts about the need for partial matches in
the OVAL Language?  

So far, we have only received one response and that was in favor of removing
partial matches.

Thanks,

Danny

>-----Original Message-----
>From: Haynes, Dan [mailto:[hidden email]]
>Sent: Wednesday, July 18, 2012 4:19 PM
>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>
>Just to make this somewhat more concrete, if you have the following
>file_object and the file "mylog.txt" doesn't exist in "/var/log".
>
><file_object id="oval:test:obj:1">
>  <path>/var/log</path>
>  <filename>mylog.txt</filename>
></file_object>
>
>And your tool supports partial matching, your system-characteristics
>might look something like the following.
>
><oval_system_characteristics>
>...
>  <collected_objects>
>    <object flag="does not exist" id="oval:test:obj:1">
>      <reference item_ref="1"/>
>    </object>
>  </collected_objects>
>
>  <system_data>
>    <file_item id="1" status="does not exist">
>      <path status="exists">/var/log</path>
>      <filename status="does not exist">mylog.txt</filename>
>    </file_item>
>  </system_data>
></oval_system_characteristics>
>
>Are there vendors who have added support for partial matching in their
>tools?  If so, do you have any thoughts on this?  Did you run into
>challenges implementing it?  Are your customers asking for this type of
capability?

>
>Content authors, do you see this as a valuable feature in OVAL?  Or, is
>this something that you could live without?
>
>I should also mention that rather than using partial matching, tools
>could utilize OVAL Messages to convey this type of information such as
>in the following example.  Of course, the use of OVAL Messages is
>completely optional in the OVAL Language.
>
><oval_system_characteristics>
>...
>  <collected_objects>
>    <object flag="does not exist" id="oval:test:obj:1">
>         <oval-sc:message>The file mylog.txt was not found in
>/var/log.</oval- sc:message>
>    </object>
>  </collected_objects>
>  <system_data/>
></oval_system_characteristics>
>
>Thanks,
>
>Danny
>
>>-----Original Message-----
>>From: Michael Chisholm [mailto:[hidden email]]
>>Sent: Tuesday, July 03, 2012 5:17 PM
>>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>Subject: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>>
>>Hello,
>>
>>The following was prepared to address what we call "partial matches"
>>in OVAL.  We hope this addresses peoples' continuing confusion over
>>when to create items (e.g. see the recent thread with the subject
>>"When to create an item?").  It is not only a presentation and
>>discussion of the idea and surrounding issues, but also a proposal to
>>remove them from the language if the community does not see a need for
>>them.  So any and all feedback on this is welcome!
>>
>>Thanks,
>>Andy
>>
>>
>>
>>What Is a Partial Match?
>>
>>Simply put, a partial match is an item with status equal to "does not
>>exist".  Normally, if no matches for an object are found on a system,
>>you would just not create any items.  But under some circumstances, it
>>was envisioned that you might want to create some items anyway, with
>>partial information.  The content of these items is left completely up
>>to interpreter authors, they would be purely optional and
>>informational, and they would not affect the evaluation of a test.  
>>Most likely, partial information would come in the form of a set of
>>item entities which matched some but not all of the object entities.  
>>The containing item would receive a status value of "does not exist"
>>to indicate that the item did not actually match the object.
>>
>>
>>Why Have Partial Matches?
>>
>>The purpose of a partial match is to assist content authors and
>>consumers of OVAL content in determining why a search did not find the
>>expected items.  An example which is frequently used relates to file
>>searches: if an object was intended to test for the existence of one
>>particular file, and the file did not exist but the directory did,
>>then a partial match item could be created which contained the
>>directory but not the filename, which would indicate that fact.
>>
>>
>>What's the Problem?
>>
>>The aforementioned partial match idea sounds very well and good, but
>>there is one feature of OVAL which turns out to be incompatible with it:
>>the "all_exist" value of the "check_existence" attribute of tests.
>>This value as defined causes interpreters to use items whose status is
>>"does not exist".  In particular, if all items collected for an object
>>have a status of "exists", then the existence check "all_exist" is
satisfied.

>>If at least one item has a status of "does not exist", it is not.  
>>This makes partial match items significant to the evaluation of a
>>test, directly contradicting the original intent which was that
>>partial matches are purely optional and informational and do not
>>affect test evaluation.  So "all_exist" is unusable if we wish to keep
>>partial matches optional.
>>
>>What if we decided to keep that enumerator and make partial matches
>>significant?  Could that work?  Probably not.  The reasons for this go
>>a bit deeper: "all_exist" really is trying to get at a content
>>author's intentions, which is essentially mind reading.  It supposes
>>you can glean from an object that the author intends that a particular
>>set of items /should/ exist, which means you can determine whether
>>they all actually did exist, and therefore make the "all_exist"
determination.

>>But gleaning intent from an object is fraught with difficulties.  It
>>is tempting to think that if all object entities used operation="equals"
>>with no variable references, then the intent was that a single item
>>with entity values equal to the object entity values should exist.  
>>But in general it is impossible to tell whether only one item would
>>have matched, since items are identified by all of their entity values
>>[1], not just the values of entities shared with the object.  So if
>>you found a matching item, you wouldn't be able to tell whether you
>>got all or only one of many items that should have matched.  Anyway,
>>are there other situations in which an author's intent may be
>>considered to be evident in the object?  What if we allowed variable
>>references where each variable has exactly one value and var_check is
"all"?

>>Multi-valued variables with var_check="at least one"?  Operation="not
>>equal" and var_check="none satisfy" (which means none of the variable
>>values are not-equal to the item entity value, or in other words,
>>they're all equal)?  Even if we decide that only the first style of
>>object, with all operations being "equals" and no variable references,
>>constitutes evident author intent, do we really want to complicate the
>>specification with this notion of an object "style" and author intent?
>>Do we want the validity of other parts of a definition to depend on this?
>>
>>This seems to add unnecessary complexity to the language.  Unless the
>>"all_exist" enumerator is really needed, it is probably preferable to
>>avoid these issues and keep the specification simple.
>>
>>
>>What Should We Do About It?
>>
>>If any matching items are found, let the only valid conclusion be that
>>some items were found, nothing more.  To remain backward-compatible,
>>the "all_exist" existence check should be defined to pass whenever an
>>item with a status of "exists" is found, and ignore any other items.  
>>This makes it equivalent to "at_least_one_exists".  Partial match
>>items, if they are deemed to be necessary, should be truly ignored
>>during definition evaluation.  Perhaps in a future version of OVAL,
"all_exist"

>>can be deprecated.  Other miscellaneous required changes to the
>>language
>>include:
>>
>>* The "DE" column from all tables in the ExistenceEnumeration schema
>>documentation should be removed, and the tables re-evaluated to ensure
>>they still make sense.
>>
>>* Schematron rules which enforce properties of items should be pared
>>down to the minimum, when the item's status is "does not exist", to
>>give content authors maximum control over the form of their partial
matches.

>>  There is at least one we know of which should be removed, associated
>>with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
>>partial match item entities with status="does not exist" from having a
>>value.
>>
>>In our experience developing ovaldi, we found that partial match
>>support added a significant amount of complexity to the code.  Also,
>>based on a limited tool survey, we found that partial matching is not
>>widely supported.  If partial matching is made truly optional, then
>>tool implementers are free to add as little or as much complexity to
>>their code as they like to support it.  But if it is not deemed to be
>>necessary, partial matching should be removed from the language, to
>>simplify the language and avoid future problems.
>>
>>
>>End Notes:
>>
>>1. See the OVAL Language Specification, section 5.2.4.2.
>>
>>To unsubscribe, send an email message to [hidden email] with
>>SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
>>difficulties, write to OVAL-
>DEVELOPER-
>>[hidden email].
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
>difficulties, write to OVAL- [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
Hi Bill,

>I'm not sure I understand the question properly...  I'm thinking that
>partial matches cause a schematron violation on the results.  Is that true?

Yes, that is one problem.  There is a schematron rule which says if an entity has a status="does not exist" it should not have a value.   This causes partial matching items to be invalid.

>If so, are partial matches really, now, part of the OVAL language?
>
>Or were you perhaps asking about removing the partial matches from
>OVALdi's
>implementation?

We are up in the air as to what to do with them as far as what to do with them in the language.  From Andy Chisholm's original message which can be found here:

http://making-security-measurable.1364806.n2.nabble.com/Partial-Matching-in-OVAL-tp7578506.html

Here are some options:

-----------------------------------------------------------------------------------------------------------------------
...

What Should We Do About It?

If any matching items are found, let the only valid conclusion be that
some items were found, nothing more.  To remain backward-compatible, the
"all_exist" existence check should be defined to pass whenever an item
with a status of "exists" is found, and ignore any other items.  This
makes it equivalent to "at_least_one_exists".  Partial match items, if
they are deemed to be necessary, should be truly ignored during
definition evaluation.  Perhaps in a future version of OVAL, "all_exist"
can be deprecated.  Other miscellaneous required changes to the language
include:

* The "DE" column from all tables in the ExistenceEnumeration schema
documentation should be removed, and the tables re-evaluated to ensure
they still make sense.

* Schematron rules which enforce properties of items should be pared
down to the minimum, when the item's status is "does not exist", to give
content authors maximum control over the form of their partial matches.
  There is at least one we know of which should be removed, associated
with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
partial match item entities with status="does not exist" from having a
value.

In our experience developing ovaldi, we found that partial match support
added a significant amount of complexity to the code.  Also, based on a
limited tool survey, we found that partial matching is not widely
supported.  If partial matching is made truly optional, then tool
implementers are free to add as little or as much complexity to their
code as they like to support it.  But if it is not deemed to be
necessary, partial matching should be removed from the language, to
simplify the language and avoid future problems.
-----------------------------------------------------------------------------------------------------------------------

Supporting partial matching has been a challenge for us in OVALDI and we really need to hear from the community what you think about them and if they truly provide value to those in the community.

Thanks,

Danny


>-----Original Message-----
>From: Haynes, Dan [mailto:[hidden email]]
>Sent: Monday, January 28, 2013 10:00 AM
>To: [hidden email]
>Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>
>I just wanted to follow up with this thread (it's been quite a while) and
>see if there were any other thoughts about the need for partial matches in
>the OVAL Language?
>
>So far, we have only received one response and that was in favor of removing
>partial matches.
>
>Thanks,
>
>Danny
>
>>-----Original Message-----
>>From: Haynes, Dan [mailto:[hidden email]]
>>Sent: Wednesday, July 18, 2012 4:19 PM
>>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>>
>>Just to make this somewhat more concrete, if you have the following
>>file_object and the file "mylog.txt" doesn't exist in "/var/log".
>>
>><file_object id="oval:test:obj:1">
>>  <path>/var/log</path>
>>  <filename>mylog.txt</filename>
>></file_object>
>>
>>And your tool supports partial matching, your system-characteristics
>>might look something like the following.
>>
>><oval_system_characteristics>
>>...
>>  <collected_objects>
>>    <object flag="does not exist" id="oval:test:obj:1">
>>      <reference item_ref="1"/>
>>    </object>
>>  </collected_objects>
>>
>>  <system_data>
>>    <file_item id="1" status="does not exist">
>>      <path status="exists">/var/log</path>
>>      <filename status="does not exist">mylog.txt</filename>
>>    </file_item>
>>  </system_data>
>></oval_system_characteristics>
>>
>>Are there vendors who have added support for partial matching in their
>>tools?  If so, do you have any thoughts on this?  Did you run into
>>challenges implementing it?  Are your customers asking for this type of
>capability?
>>
>>Content authors, do you see this as a valuable feature in OVAL?  Or, is
>>this something that you could live without?
>>
>>I should also mention that rather than using partial matching, tools
>>could utilize OVAL Messages to convey this type of information such as
>>in the following example.  Of course, the use of OVAL Messages is
>>completely optional in the OVAL Language.
>>
>><oval_system_characteristics>
>>...
>>  <collected_objects>
>>    <object flag="does not exist" id="oval:test:obj:1">
>>         <oval-sc:message>The file mylog.txt was not found in
>>/var/log.</oval- sc:message>
>>    </object>
>>  </collected_objects>
>>  <system_data/>
>></oval_system_characteristics>
>>
>>Thanks,
>>
>>Danny
>>
>>>-----Original Message-----
>>>From: Michael Chisholm [mailto:[hidden email]]
>>>Sent: Tuesday, July 03, 2012 5:17 PM
>>>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>Subject: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL
>>>
>>>Hello,
>>>
>>>The following was prepared to address what we call "partial matches"
>>>in OVAL.  We hope this addresses peoples' continuing confusion over
>>>when to create items (e.g. see the recent thread with the subject
>>>"When to create an item?").  It is not only a presentation and
>>>discussion of the idea and surrounding issues, but also a proposal to
>>>remove them from the language if the community does not see a need for
>>>them.  So any and all feedback on this is welcome!
>>>
>>>Thanks,
>>>Andy
>>>
>>>
>>>
>>>What Is a Partial Match?
>>>
>>>Simply put, a partial match is an item with status equal to "does not
>>>exist".  Normally, if no matches for an object are found on a system,
>>>you would just not create any items.  But under some circumstances, it
>>>was envisioned that you might want to create some items anyway, with
>>>partial information.  The content of these items is left completely up
>>>to interpreter authors, they would be purely optional and
>>>informational, and they would not affect the evaluation of a test.
>>>Most likely, partial information would come in the form of a set of
>>>item entities which matched some but not all of the object entities.
>>>The containing item would receive a status value of "does not exist"
>>>to indicate that the item did not actually match the object.
>>>
>>>
>>>Why Have Partial Matches?
>>>
>>>The purpose of a partial match is to assist content authors and
>>>consumers of OVAL content in determining why a search did not find the
>>>expected items.  An example which is frequently used relates to file
>>>searches: if an object was intended to test for the existence of one
>>>particular file, and the file did not exist but the directory did,
>>>then a partial match item could be created which contained the
>>>directory but not the filename, which would indicate that fact.
>>>
>>>
>>>What's the Problem?
>>>
>>>The aforementioned partial match idea sounds very well and good, but
>>>there is one feature of OVAL which turns out to be incompatible with it:
>>>the "all_exist" value of the "check_existence" attribute of tests.
>>>This value as defined causes interpreters to use items whose status is
>>>"does not exist".  In particular, if all items collected for an object
>>>have a status of "exists", then the existence check "all_exist" is
>satisfied.
>>>If at least one item has a status of "does not exist", it is not.
>>>This makes partial match items significant to the evaluation of a
>>>test, directly contradicting the original intent which was that
>>>partial matches are purely optional and informational and do not
>>>affect test evaluation.  So "all_exist" is unusable if we wish to keep
>>>partial matches optional.
>>>
>>>What if we decided to keep that enumerator and make partial matches
>>>significant?  Could that work?  Probably not.  The reasons for this go
>>>a bit deeper: "all_exist" really is trying to get at a content
>>>author's intentions, which is essentially mind reading.  It supposes
>>>you can glean from an object that the author intends that a particular
>>>set of items /should/ exist, which means you can determine whether
>>>they all actually did exist, and therefore make the "all_exist"
>determination.
>>>But gleaning intent from an object is fraught with difficulties.  It
>>>is tempting to think that if all object entities used operation="equals"
>>>with no variable references, then the intent was that a single item
>>>with entity values equal to the object entity values should exist.
>>>But in general it is impossible to tell whether only one item would
>>>have matched, since items are identified by all of their entity values
>>>[1], not just the values of entities shared with the object.  So if
>>>you found a matching item, you wouldn't be able to tell whether you
>>>got all or only one of many items that should have matched.  Anyway,
>>>are there other situations in which an author's intent may be
>>>considered to be evident in the object?  What if we allowed variable
>>>references where each variable has exactly one value and var_check is
>"all"?
>>>Multi-valued variables with var_check="at least one"?  Operation="not
>>>equal" and var_check="none satisfy" (which means none of the variable
>>>values are not-equal to the item entity value, or in other words,
>>>they're all equal)?  Even if we decide that only the first style of
>>>object, with all operations being "equals" and no variable references,
>>>constitutes evident author intent, do we really want to complicate the
>>>specification with this notion of an object "style" and author intent?
>>>Do we want the validity of other parts of a definition to depend on this?
>>>
>>>This seems to add unnecessary complexity to the language.  Unless the
>>>"all_exist" enumerator is really needed, it is probably preferable to
>>>avoid these issues and keep the specification simple.
>>>
>>>
>>>What Should We Do About It?
>>>
>>>If any matching items are found, let the only valid conclusion be that
>>>some items were found, nothing more.  To remain backward-compatible,
>>>the "all_exist" existence check should be defined to pass whenever an
>>>item with a status of "exists" is found, and ignore any other items.
>>>This makes it equivalent to "at_least_one_exists".  Partial match
>>>items, if they are deemed to be necessary, should be truly ignored
>>>during definition evaluation.  Perhaps in a future version of OVAL,
>"all_exist"
>>>can be deprecated.  Other miscellaneous required changes to the
>>>language
>>>include:
>>>
>>>* The "DE" column from all tables in the ExistenceEnumeration schema
>>>documentation should be removed, and the tables re-evaluated to ensure
>>>they still make sense.
>>>
>>>* Schematron rules which enforce properties of items should be pared
>>>down to the minimum, when the item's status is "does not exist", to
>>>give content authors maximum control over the form of their partial
>matches.
>>>  There is at least one we know of which should be removed, associated
>>>with oval-sc:EntityAttributeGroup.  The rule erroneously prevents
>>>partial match item entities with status="does not exist" from having a
>>>value.
>>>
>>>In our experience developing ovaldi, we found that partial match
>>>support added a significant amount of complexity to the code.  Also,
>>>based on a limited tool survey, we found that partial matching is not
>>>widely supported.  If partial matching is made truly optional, then
>>>tool implementers are free to add as little or as much complexity to
>>>their code as they like to support it.  But if it is not deemed to be
>>>necessary, partial matching should be removed from the language, to
>>>simplify the language and avoid future problems.
>>>
>>>
>>>End Notes:
>>>
>>>1. See the OVAL Language Specification, section 5.2.4.2.
>>>
>>>To unsubscribe, send an email message to [hidden email] with
>>>SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
>>>difficulties, write to OVAL-
>>DEVELOPER-
>>>[hidden email].
>>
>>To unsubscribe, send an email message to [hidden email] with
>>SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
>>difficulties, write to OVAL- [hidden email].
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message.  If you have
>difficulties, write to [hidden email].
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST
>in the BODY of the message.  If you have difficulties, write to OVAL-
>[hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

joval
Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:
If partial matching is made truly optional


--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [mailto:[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

joval
Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download



--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

John Ulmer

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

sprabhu
Hello,

Late into this conversation,  I was observing the community decision on this, 
and we were in discussion with my team over here, to draw some decision on this.

According to me Partial matches are not required. Obviously one has got nothing to
do with items collected flagged as "Does Not Exist", except for debugging purpose,
the usage of oval-sc messages would also help the same way here.
So I opt for not to consider the partial matches, and would like to make the
evaluation based on the 1st option  that David suggested.

i,e out of the three the 1st one.

**1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

I think that is how jOVAL is currently functioning.


But, it seems at one place where "partial match", or "DNE" object items have caught up with evaluation of the object and it is @ OVAL Object_component.
The ERROR Flag Enumeration in specification "5.3.5.3.2.2.1 Determining the Flag Value" says,

FlagEnumeration                                      Description
Value

error            This flag value must be used when the value does not conform to the specified
                    datatype as defined in the oval:DatatypeEnumeration.

                    This flag value must be used if the OVAL Object does not return any OVAL Items.

                    This flag value must be used if an entity is not found with a name that matches
                    the value of the item_field property.

                    This flag value must be used if a field is not found with a name that matches the
                    value of the record_field property.


Except the first scenario under description, the rest of the 3 scenarios seem to be dependent on the partial matches or "DNE" object items.
I have elaborated each one of these in specific, in the mail continued next.

(Sorry for breaking the mail in two parts, I had to do this, to clearly separate my view on Partial Matching and issue that I'm expecting.)

-- 
Thanks !!
Prabhu S A



On 01/29/2013 07:49 PM, Ulmer, John R. wrote:

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].


To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

sprabhu
Hello,

This is a continuation of the issue that I was expecting, "object_component caught up with "partial matches" or "DNE" object items".

*** This flag value must be used if the OVAL Object does not return any OVAL Items.
=================================================================

    As the creation of the items itself is interpreter authors decision, therefore this case holds good for only those objects that evaluates "Does Not Exist" without creating any item-refs, which evaluate to partial matches. In case of below object collection data, the object_component evaluation is set not to throw ERROR, because it has item, instead it should be considered as "DNE".


    <object comment="Registry key HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer!ENU_GUID (32 bit)" flag="does not exist" id="oval:org.mitre.oval:obj:6736" version="2">
     <reference item_ref="1"/>
    </object>

    <registry_item id="1" status="does not exist" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <hive>HKEY_LOCAL_MACHINE</hive>
      <key status="does not exist">SOFTWARE\Adobe\Acrobat Reader\9.0\Installer</key>
    </registry_item>

So this means that interpreter is not evaluating the whether a collected object which is flagged to "Does not exist" has collected items or not, immaterial of what flags they have. Instead the object_component just takes the collected_objects object flag attribute to heart, and sees "does not exist" and decides it ERROR, as it is referred by variable, like the below obj.

    <object comment="This object represents a file_object with a path that matches the regular expression '/tmp/support/txt($|/.*)'." flag="does not exist" id="oval:org.mitre.oval.test:obj:142" version="1"/>


This is bug with interpreter, for not functioning as per the specification?


*** This flag value must be used if an entity is not found with a name that matches the value of the item_field property.
============================================================================================

    Obviously item_filed property appears, only when the object that object_component refers has item, let it be in complete collection or partial match, for ex : item_filed "result".

         Does not exist object case :


    <object comment="WMI querry that gets all the running services." flag="complete" id="oval:org.secpod.devel.oval:obj:20" version="0">
        <reference item_ref="1"/>
    </object>

    <wmi57_item id="1" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <namespace>root\cimv2</namespace>
      <wql>Select Name from Win32_Service where State="NOT Running"</wql>
      <result datatype="record" status="does not exist"/>
    </wmi57_item>

        So why to flag ERROR for a non existing entity ? This is as good as not found, but not an ERROR. ERROR is when, if the specified Namespace is invalid, or insufficient permissions to execute the query, and so on, w.r.t above example, and same in case of registry_test or file related tests well.

        Complete Status case:


    <object comment="WMI query that gets all the running services." flag="complete" id="oval:org.secpod.devel.oval:obj:20" version="0">
      <reference item_ref="1"/>
    </object>

    <wmi57_item id="1" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <namespace>root\cimv2</namespace>
      <wql>Select Name from Win32_Service where State="Running"</wql>
      <result datatype="record">
        <oval-sc:field name="name">AeLookupSvc</oval-sc:field>
      </result>
      <result datatype="record">
        <oval-sc:field name="name">Alerter</oval-sc:field>
      </result>
        </wmi57_item>

*** This flag value must be used if a field is not found with a name that matches the value of the record_field property.
============================================================================================

    Here "record_field" is collected during the COMPLETE status of object, as in the above object with "Complete Status". i,e record_field "name" is collected only when the object is evaluated to successful i.e flagged to "COMPLETE", including item_filed status not set to "does not exist".

    The scenario of not found with "name that matches the value of record_field" is equivalent to wrong comparison or invalid comparison, but clearly not ERROR.


Bottom-line :
=========

B'coz the object_component treats the, "DNE" objects as error which it should not,  this is impacting the OVAL Functions as well, and evaluating to ERROR.

The "Resulting Flag" for ERROR instances in specification "5.3.5.3.1.4 Determining the Flag Value" of OVAL function says :

Resulting            Number of OVAL Components with the Specified Flag
     Flag          error      complete     incomplete   does not          not         not
                                                                              exist        collected   applicable

error               1+           0+               0+                  0+             0+          0+

does not         0             0+               0+                  1+              0            0
exist


 


But, if there is any oval_component on which the function operates (object_component in my case), not flagged to ERROR but Flagged to "Does Not Exist" should be evaluated to "DNE", as per the "does not exist" Resulting Flag in the above table says.


Let me know, what is your feedback on this and correct me if I interpreted it wrongly.



-- 
Thanks !!
Prabhu S A


On 02/05/2013 12:39 PM, Prabhu S Angadi wrote:
Hello,

Late into this conversation,  I was observing the community decision on this, 
and we were in discussion with my team over here, to draw some decision on this.

According to me Partial matches are not required. Obviously one has got nothing to
do with items collected flagged as "Does Not Exist", except for debugging purpose,
the usage of oval-sc messages would also help the same way here.
So I opt for not to consider the partial matches, and would like to make the
evaluation based on the 1st option  that David suggested.

i,e out of the three the 1st one.

**1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

I think that is how jOVAL is currently functioning.


But, it seems at one place where "partial match", or "DNE" object items have caught up with evaluation of the object and it is @ OVAL Object_component.
The ERROR Flag Enumeration in specification "5.3.5.3.2.2.1 Determining the Flag Value" says,

FlagEnumeration                                      Description
Value

error            This flag value must be used when the value does not conform to the specified
                    datatype as defined in the oval:DatatypeEnumeration.

                    This flag value must be used if the OVAL Object does not return any OVAL Items.

                    This flag value must be used if an entity is not found with a name that matches
                    the value of the item_field property.

                    This flag value must be used if a field is not found with a name that matches the
                    value of the record_field property.


Except the first scenario under description, the rest of the 3 scenarios seem to be dependent on the partial matches or "DNE" object items.
I have elaborated each one of these in specific, in the mail continued next.

(Sorry for breaking the mail in two parts, I had to do this, to clearly separate my view on Partial Matching and issue that I'm expecting.)

-- 
Thanks !!
Prabhu S A



On 01/29/2013 07:49 PM, Ulmer, John R. wrote:

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].


To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].


  

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].


	
	
	
	
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
In reply to this post by John Ulmer

Hi John,

Good question.

 

Since empty string is a valid match to the sub-expression "(.*)", having subexpression entity with status="exists" and an empty string value makes sense. 

 

Similarly, a subexpression entity with a status="does not exist" makes sense when no sub-expression is specified in the pattern entity because you attempted to find a sub-expression and one was not found.


Does that help clarify your situations?


Thanks,

Danny

 

From: Ulmer, John R. [mailto:[hidden email]]
Sent: Tuesday, January 29, 2013 9:19 AM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

smime.p7s (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
In reply to this post by joval

Hi David,

 

>1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)

According to the specification (Section 5.3.2.4.2) and the schema documentation in oval-results-schema.xsd (oval-res:TestType), when the collected_object flag="does not exist", the test result should be based solely on the result of the Check Existence Evaluation.

 

>2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)

Some places where you may need to rely on looking at the item statuses is when there is no collected_object section present (and no flag values) or if you do run into an object where you do something like taking the union of an object with a flag="complete" and an object with a flag="does not exist" seeing that the resulting object with have a flag="complete" (see oval-def:SetOperatorEnumeration).

 

The problem here (and what I think Andy was getting at in his original message) is that you could get different results depending on if a tool decides to implement partial matching.  For example, consider the following test where Tool A implements partial matching and Tool B does not implement partial matching.

 

….

<file_test check_existence="all_exist" id="tst:1">

  <object object_ref=”obj:1”/>

</file_test>

 

<file_object id="obj:1">

  <path>C:\dir1</path>

  <filename>file.txt</filename>

</file_object>

 

<file_object id="obj:2">

  <path>C:\dir2</path>

  <filename>file.txt</filename>

</file_object>

 

<file_object id="obj:3">

  <set set_operator="UNION">
    <object_reference> obj:1</object_reference>
    <object_reference> obj:2</object_reference>

  </set>

</file_object>

 

And your file system looked like this:

 

+C:\

  +C:\dir1

     -file.txt

  +C:\dir2

     -test.txt

 

In the system-characterstics, for Tool A, you would get:

 

<collected_objects>

  <object flag="complete" id="obj:1">
      <reference item_ref="1"/>
    </object>
    <object flag="does not exist" id="obj:2">
      <reference item_ref="2"/>
    </object>
    <object flag="complete" id="obj:3">
      <reference item_ref="1"/>
      <reference item_ref="2"/>
    </object>
</collected_objects>

 

 

<system_data>
  <file_item id="1">
    <path>C:\dir1</path>
    <filename>file.txt</filename>
   …

  </file_item>


  <file_item id="2" status="does not exist">
    <path>C:\dir2</path>
    <filename status="does not exist"/>

    …

 </file_item>

</system_data>

 

 

In the system-characterstics, for Tool B, you would get:

 

<collected_objects>

  <object flag="complete" id="obj:1">
      <reference item_ref="1"/>
    </object>
    <object flag="does not exist" id="obj:2"/>
    <object flag="complete" id="obj:3">
      <reference item_ref="1"/>
    </object>
</collected_objects>

 

 

<system_data>
  <file_item id="1">
    <path>C:\dir1</path>
    <filename>file.txt</filename>
   …

  </file_item>

</system_data>

 

 

Then to evaluate the test, you would follow the oval:ExistenceEnumeration.

 

                      ||  item status value count  ||
  attr value    ||                                           || existence piece is
                      ||  EX  |  DE  |  ER  |  NC       ||
    ---------------||-------------------------------------||------------------
                      ||  1+  |  0   |  0     |  0           ||  True
                      ||  0    |  0   |  0     |  0           ||  False
                      ||  0+  |  1+ |  0+   |  0+         ||  False 
   all_exist     ||  0+  |  0   |  1+   |  0+         ||  Error
                      ||  0+  |  0   |  0     |  1+         ||  Unknown
                      ||  --   |  --   |  --    |  --           ||  Not Evaluated
                      ||  --   |  --   |  --    |  --           ||  Not Applicable
    ---------------||-------------------------------------||------------------

 

For Tool A, obj:3 has collected two items:

*obj:1 status="exists"

*obj:2 status="does not exist"

 

This would evaluate to result="false" (because of row 3).


For Tool B, obj:3 has collected one item:

*obj:1 status="exists"

 

This would evaluate to result="true" (because of row 1).

 

As a result, partial matching cannot be optionally implemented in tools (as currently defined) because it allows for tools to arrive at different results for the same content.  Given this, we were trying to find out if we should fix partial matching or remove it from the language all together.  In either case, from a maintenance perspective, we definitely would like to remove it from OVALDI as it has caused us quite a few headaches.

 

>3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

 

If you are ignoring items based on (1) flag="does not exist" and (2) item status="does not exist", item entity status should only influence state evaluation when the item as a whole exists (i.e. object entities exist) on the system.  Otherwise, the result could be determined just looking at the Check Existence Evaluation.

 

Does this make sense?

 

>Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

 

Yes, I am sure there are some inconsistencies here and that is not a problem with the language.

 

Thanks,

Danny

 

From: David Solin [mailto:[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: Haynes, Dan
Cc: oval-developer-list OVAL Developer List/Closed Public Discussion
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
In reply to this post by sprabhu

Hi Prabhu,

 

>According to me Partial matches are not required. Obviously one has got nothing to do with items collected flagged as "Does Not Exist", except for debugging purpose, the usage of oval-sc >messages would also help the same way here. So I opt for not to consider the partial matches, and would like to make the evaluation based on the 1st option  that David suggested.

 

Thanks for weighing in here.

 

>The ERROR Flag Enumeration in specification "5.3.5.3.2.2.1 Determining the Flag Value" says,

> 

>error This flag value must be used when the value does not conform to the specified datatype as defined in the oval:DatatypeEnumeration.
>

>This flag value must be used if the OVAL Object does not return any OVAL Items.
>

>This flag value must be used if an entity is not found with a name that matches the value of the item_field property.
>

>This flag value must be used if a field is not found with a name that matches the value of the record_field property.

> 

>Except the first scenario under description, the rest of the 3 scenarios seem to be dependent on the partial matches or "DNE" object items.  I have elaborated each one of these in specific, in >the mail continued next.

 

Yes, the last 3 scenarios are affected by the flag attribute.

 

I am following up with your continuation message separately.

 

Thanks,

Danny

 

From: Prabhu S Angadi [mailto:[hidden email]]
Sent: Tuesday, February 05, 2013 2:10 AM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hello,

Late into this conversation,  I was observing the community decision on this, 
and we were in discussion with my team over here, to draw some decision on this.

According to me Partial matches are not required. Obviously one has got nothing to
do with items collected flagged as "Does Not Exist", except for debugging purpose,
the usage of oval-sc messages would also help the same way here.
So I opt for not to consider the partial matches, and would like to make the
evaluation based on the 1st option  that David suggested.

i,e out of the three the 1st one.

**1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

I think that is how jOVAL is currently functioning.


But, it seems at one place where "partial match", or "DNE" object items have caught up with evaluation of the object and it is @ OVAL Object_component.
The ERROR Flag Enumeration in specification "5.3.5.3.2.2.1 Determining the Flag Value" says,

FlagEnumeration                                      Description
Value

error            This flag value must be used when the value does not conform to the specified
                    datatype as defined in the oval:DatatypeEnumeration.

                    This flag value must be used if the OVAL Object does not return any OVAL Items.

                    This flag value must be used if an entity is not found with a name that matches
                    the value of the item_field property.

                    This flag value must be used if a field is not found with a name that matches the
                    value of the record_field property.


Except the first scenario under description, the rest of the 3 scenarios seem to be dependent on the partial matches or "DNE" object items.
I have elaborated each one of these in specific, in the mail continued next.

(Sorry for breaking the mail in two parts, I had to do this, to clearly separate my view on Partial Matching and issue that I'm expecting.)


-- 
Thanks !!
Prabhu S A




On 01/29/2013 07:49 PM, Ulmer, John R. wrote:

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

 

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

joval
In reply to this post by Danny Haynes
On 2/7/2013 8:55 AM, Haynes, Dan wrote:

Does this make sense?


Yes indeed!  In short, the existence_check truth table for "all_exist" exposes the problem, which is that the result will differ based on whether non-existent items were associated with the object.

Now the question is, what does "all_exist" really mean?  For it to remain meaningful, we would need absolute rules on when to create non-existent items.



--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Danny Haynes
Administrator
In reply to this post by sprabhu

Hi Prabhu,

 

>This is bug with interpreter, for not functioning as per the specification?

From the specification and schema documentation, the intent of evaluating to error, when an OVAL Object does not return any OVAL Items, is to report an error when zero matching items are collected by the referenced object.  While the example with partial matches has items, I think an error should still be reported because the items do not actually match the specified object because they do not exist on the system.  So in this case, I think the OVAL Interpreter is functioning according to the specification as currently written.

 

>Bottom-line :
>=========
>
>B'coz the object_component treats the, "DNE" objects as error which it should not,  this is impacting the OVAL Functions as well, and evaluating to ERROR.
>
>The "Resulting Flag" for ERROR instances in specification "5.3.5.3.1.4 Determining the Flag Value" of OVAL function says :
>
>Resulting            Number of OVAL Components with the Specified Flag
>     Flag          error      complete     incomplete   does not  not            not
>                                                                               exist        collected   applicable
>
>error               1+           0+               0+                  0+             0+          0+
>
>does not         0             0+               0+                  1+              0            0
>exist
>
>
>But, if there is any oval_component on which the function operates (object_component in my case), not flagged to ERROR but Flagged to "Does Not Exist" should be evaluated to "DNE", as per >the "does not exist" Resulting Flag in the above table says.

I think I see what you are getting at here.  Components should be able to work with zero values rather than immediately reporting errors causing issues upstream with functions, etc.  I think this issue is aligned with the thread that you started on the count function (http://making-security-measurable.1364806.n2.nabble.com/count-function-against-component-evaluating-to-zero-values-tp7304434.html) reporting an error rather than returning a value of 0.  Please correct me if I am wrong on this ;).  Unfortunately, we are stuck with this behavior because it is part of the current specification.

 

However, there is an open 5.11 tracker (http://oval.mitre.org/language/version5.11/#27633) to consider these issues that affect the use of variables, functions, and components and see how we can further specify how they should work and whether or not error conditions, like above, should be handled at a higher level construct (e.g. entities that reference the variables).  Some of this came out of a presentation that we gave at Spring 2011 developer days (http://scap.nist.gov/events/2011/saddsp/presentations/Danny_Haynes-Error_Handling_in_Variables.pdf). 

 

This is definitely an area that I would like to see us work on as part of the 5.11 release as it needs a good amount of work.


Thanks,

Danny

 

From: Prabhu S Angadi [mailto:[hidden email]]
Sent: Tuesday, February 05, 2013 2:13 AM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hello,

This is a continuation of the issue that I was expecting, "object_component caught up with "partial matches" or "DNE" object items".

*** This flag value must be used if the OVAL Object does not return any OVAL Items.
=================================================================

    As the creation of the items itself is interpreter authors decision, therefore this case holds good for only those objects that evaluates "Does Not Exist" without creating any item-refs, which evaluate to partial matches. In case of below object collection data, the object_component evaluation is set not to throw ERROR, because it has item, instead it should be considered as "DNE".


    <object comment="Registry key HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer!ENU_GUID (32 bit)" flag="does not exist" id="oval:org.mitre.oval:obj:6736" version="2">
     <reference item_ref="1"/>
    </object>

    <registry_item id="1" status="does not exist" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <hive>HKEY_LOCAL_MACHINE</hive>
      <key status="does not exist">SOFTWARE\Adobe\Acrobat Reader\9.0\Installer</key>
    </registry_item>

So this means that interpreter is not evaluating the whether a collected object which is flagged to "Does not exist" has collected items or not, immaterial of what flags they have. Instead the object_component just takes the collected_objects object flag attribute to heart, and sees "does not exist" and decides it ERROR, as it is referred by variable, like the below obj.

    <object comment="This object represents a file_object with a path that matches the regular expression '/tmp/support/txt($|/.*)'." flag="does not exist" id="oval:org.mitre.oval.test:obj:142" version="1"/>


This is bug with interpreter, for not functioning as per the specification?


*** This flag value must be used if an entity is not found with a name that matches the value of the item_field property.
============================================================================================

    Obviously item_filed property appears, only when the object that object_component refers has item, let it be in complete collection or partial match, for ex : item_filed "result".

         Does not exist object case :


    <object comment="WMI querry that gets all the running services." flag="complete" id="oval:org.secpod.devel.oval:obj:20" version="0">
        <reference item_ref="1"/>
    </object>

    <wmi57_item id="1" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <namespace>root\cimv2</namespace>
      <wql>Select Name from Win32_Service where State="NOT Running"</wql>
      <result datatype="record" status="does not exist"/>
    </wmi57_item>

        So why to flag ERROR for a non existing entity ? This is as good as not found, but not an ERROR. ERROR is when, if the specified Namespace is invalid, or insufficient permissions to execute the query, and so on, w.r.t above example, and same in case of registry_test or file related tests well.

        Complete Status case:


    <object comment="WMI query that gets all the running services." flag="complete" id="oval:org.secpod.devel.oval:obj:20" version="0">
      <reference item_ref="1"/>
    </object>

    <wmi57_item id="1" xmlns="http://oval.mitre.org/XMLSchema/oval-system-characteristics-5#windows">
      <namespace>root\cimv2</namespace>
      <wql>Select Name from Win32_Service where State="Running"</wql>
      <result datatype="record">
        <oval-sc:field name="name">AeLookupSvc</oval-sc:field>
      </result>
      <result datatype="record">
        <oval-sc:field name="name">Alerter</oval-sc:field>
      </result>
        </wmi57_item>

*** This flag value must be used if a field is not found with a name that matches the value of the record_field property.
============================================================================================

    Here "record_field" is collected during the COMPLETE status of object, as in the above object with "Complete Status". i,e record_field "name" is collected only when the object is evaluated to successful i.e flagged to "COMPLETE", including item_filed status not set to "does not exist".

    The scenario of not found with "name that matches the value of record_field" is equivalent to wrong comparison or invalid comparison, but clearly not ERROR.


Bottom-line :
=========

B'coz the object_component treats the, "DNE" objects as error which it should not,  this is impacting the OVAL Functions as well, and evaluating to ERROR.

The "Resulting Flag" for ERROR instances in specification "5.3.5.3.1.4 Determining the Flag Value" of OVAL function says :

Resulting            Number of OVAL Components with the Specified Flag
     Flag          error      complete     incomplete   does not          not         not
                                                                              exist        collected   applicable

error               1+           0+               0+                  0+             0+          0+

does not         0             0+               0+                  1+              0            0
exist


 


But, if there is any oval_component on which the function operates (object_component in my case), not flagged to ERROR but Flagged to "Does Not Exist" should be evaluated to "DNE", as per the "does not exist" Resulting Flag in the above table says.


Let me know, what is your feedback on this and correct me if I interpreted it wrongly.




-- 
Thanks !!
Prabhu S A



On 02/05/2013 12:39 PM, Prabhu S Angadi wrote:

Hello,

Late into this conversation,  I was observing the community decision on this, 
and we were in discussion with my team over here, to draw some decision on this.

According to me Partial matches are not required. Obviously one has got nothing to
do with items collected flagged as "Does Not Exist", except for debugging purpose,
the usage of oval-sc messages would also help the same way here.
So I opt for not to consider the partial matches, and would like to make the
evaluation based on the 1st option  that David suggested.

i,e out of the three the 1st one.

**1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

I think that is how jOVAL is currently functioning.


But, it seems at one place where "partial match", or "DNE" object items have caught up with evaluation of the object and it is @ OVAL Object_component.
The ERROR Flag Enumeration in specification "5.3.5.3.2.2.1 Determining the Flag Value" says,

FlagEnumeration                                      Description
Value

error            This flag value must be used when the value does not conform to the specified
                    datatype as defined in the oval:DatatypeEnumeration.

                    This flag value must be used if the OVAL Object does not return any OVAL Items.

                    This flag value must be used if an entity is not found with a name that matches
                    the value of the item_field property.

                    This flag value must be used if a field is not found with a name that matches the
                    value of the record_field property.


Except the first scenario under description, the rest of the 3 scenarios seem to be dependent on the partial matches or "DNE" object items.
I have elaborated each one of these in specific, in the mail continued next.

(Sorry for breaking the mail in two parts, I had to do this, to clearly separate my view on Partial Matching and issue that I'm expecting.)


-- 
Thanks !!
Prabhu S A




On 01/29/2013 07:49 PM, Ulmer, John R. wrote:

One minor addition to conversation…  This may have been touched previously.  If so, sorry for the noise.

 

Clearly defining what constitutes a ‘partial match’ is slightly complicated by the items whose list of expected detail elements is dependent on the content.  I can only think of one case off the top of my head, the textfilecontent item where the content author may use paren’s, or not.  Or, a pattern contains paren’s and is so flexible that the string matches, but, no subexpression is caught:“(.*)”.

 

You may or may not get a ‘subexpression’ string from the pattern. 

If there are no parens in the pattern, does that mean the subexpression status = “Does Not Matter?”

 

Or , if there are paren’s in the pattern, but the pattern allows an empty string to match, does that mean the subexpression status = “Exists….. but, is empty?”

 

How do these situations impact the status of the item?

 

‘just muddying the waters…

 

From: [hidden email] [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 8:56 PM
To: [hidden email]
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

Thanks for the clarification!

I suppose that the extent to which they influence the evaluation depends on a several things, notably:
1) Whether the interpreter takes the collected_objects object flag attribute to heart (i.e., it sees "does not exist" and then doesn't evaluate the items)
2) Whether the interpreter takes the item status into consideration (i.e., a status of "does not exist" causes the item to be ignored)
3) Whether the interpreter takes the item entity status into consideration (same obvious interpretation)

Ovaldi ignores these indicators to its own, unique peril.  I think that's a defect in ovaldi, not the OVAL language itself.

Regards,
--David

On 1/28/2013 7:48 PM, Haynes, Dan wrote:

Hi David,

 

Sorry about that.  Context is everything ;). 

 

Partial matching is meant to be optional as described in the specification (Section 5.2.4.3):

 

“A partial match is when an OVAL Item, containing some information, is reported in the OVAL System Characteristics rather than simply not reporting the OVAL Item. Partial matches are useful for debugging purposes when an OVAL Item does not exist on the system or is not collected due to limitations in the OVAL Capable Product. Please note that the use of partial matches is optional.”   

 

What Andy was getting at there was if partial matches were truly optional they would not affect the evaluation of a test.  However, because they do affect evaluation, it is not possible for them to be optional because if one implementation uses partial matches and another doesn’t you will get different results.  Andy describes it in more detail in the beginning of his write-up that I provided a link to earlier.

 

Thanks,

Danny

 

From: David Solin [[hidden email]] On Behalf Of David Solin
Sent: Monday, January 28, 2013 7:58 PM
To: oval-developer-list OVAL Developer List/Closed Public Discussion
Cc: Haynes, Dan
Subject: Re: [OVAL-DEVELOPER-LIST] Partial Matching in OVAL

 

Hi Danny,

The snippet below suggests that partial matching (which is, the inclusion of items that are not actually matches for an object, such as non-existent items, or parent directories what do not contain the child file an object specifies) is currently mandated.  But where in the specification is this mandate described?  (I'd be looking for the word "MUST").

If I recall correctly it is only hinted at in various places in the schema documentation, and it exists -- as a flawed implementation -- only in ovaldi, and even the schematron chokes on that.

Regards,
--David Solin

On 1/28/2013 6:02 PM, Haynes, Dan wrote:

If partial matching is made truly optional

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

 

--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].



To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].



 

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

Michael Chisholm
In reply to this post by joval
On 2/7/2013 10:33 AM, David Solin wrote:

> On 2/7/2013 8:55 AM, Haynes, Dan wrote:
>>
>> Does this make sense?
>>
>
> Yes indeed!  In short, the existence_check truth table for "all_exist"
> exposes the problem, which is that the result will differ based on
> whether non-existent items were associated with the object.
>
> Now the question is, what does "all_exist" really mean?  For it to
> remain meaningful, we would need absolute rules on when to create
> non-existent items.
>

Fyi, I addressed check_existence="all_exist" in the original email of
this thread: what it is currently defined to mean, what it was probably
intended to mean, why it's problematic, and a proposal for what to do
about it.  I had hoped to provoke thought about this seven months ago :)

Andy

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Partial Matching in OVAL

joval
Indeed you did, and you summarized it quite well.  Here's the link:
http://making-security-measurable.1364806.n2.nabble.com/Partial-Matching-in-OVAL-td7578506.html


On 2/7/2013 3:30 PM, Michael Chisholm wrote:
On 2/7/2013 10:33 AM, David Solin wrote:
On 2/7/2013 8:55 AM, Haynes, Dan wrote:

Does this make sense?


Yes indeed!  In short, the existence_check truth table for "all_exist"
exposes the problem, which is that the result will differ based on
whether non-existent items were associated with the object.

Now the question is, what does "all_exist" really mean?  For it to
remain meaningful, we would need absolute rules on when to create
non-existent items.


Fyi, I addressed check_existence="all_exist" in the original email of this thread: what it is currently defined to mean, what it was probably intended to mean, why it's problematic, and a proposal for what to do about it.  I had hoped to provoke thought about this seven months ago :)

Andy

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].


--

jOVAL.org: SCAP Simplified.
Learn More | Features | Download

To unsubscribe, send an email message to [hidden email] with SIGNOFF OVAL-DEVELOPER-LIST in the BODY of the message. If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download