5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
Hi Everyone,

In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common, probably so that it could be used with variables.  This type defines the note element.  But, the notes element is still defined in oval-def:DefinitionType.

This introduces a namespace mismatch between <notes> and its child <note> — which causes validation errors when parsing a 5.10 OVAL document with a 5.11 parser.  We need to fix this in OVAL 5.11.1, as it’s a serious backwards-compatibility issue.

Best regards,
—David Solin
[hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Danny Haynes
Administrator
Yes, this change was made to support notes in variables.  I agree it is a serious backwards compatibility issue and it can be addressed with the 5.11.1 release.  Thanks for catching this and reporting it!

-Danny

>-----Original Message-----
>From: David Solin [mailto:[hidden email]]
>Sent: Monday, March 02, 2015 12:06 PM
>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect
>the fix for issue #6 was the culprit)
>
>Hi Everyone,
>
>In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
>probably so that it could be used with variables.  This type defines the note
>element.  But, the notes element is still defined in oval-def:DefinitionType.
>
>This introduces a namespace mismatch between <notes> and its child <note> —
>which causes validation errors when parsing a 5.10 OVAL document with a 5.11
>parser.  We need to fix this in OVAL 5.11.1, as it’s a serious backwards-
>compatibility issue.
>
>Best regards,
>—David Solin
>[hidden email]
>
>To unsubscribe, send an email message to [hidden email] with
>SIGNOFF OVAL-DEVELOPER-LIST
>in the BODY of the message.  If you have difficulties, write to OVAL-DEVELOPER-
>[hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Danny Haynes
Administrator
I forgot to include it, but, I added a tracker for this issue.

https://github.com/OVALProject/Language/issues/237

Thanks,

Danny


>-----Original Message-----
>From: Haynes, Dan [mailto:[hidden email]]
>Sent: Monday, March 02, 2015 9:06 PM
>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>(Suspect the fix for issue #6 was the culprit)
>
>Yes, this change was made to support notes in variables.  I agree it is a serious
>backwards compatibility issue and it can be addressed with the 5.11.1 release.
>Thanks for catching this and reporting it!
>
>-Danny
>
>>-----Original Message-----
>>From: David Solin [mailto:[hidden email]]
>>Sent: Monday, March 02, 2015 12:06 PM
>>To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect
>>the fix for issue #6 was the culprit)
>>
>>Hi Everyone,
>>
>>In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
>>probably so that it could be used with variables.  This type defines the note
>>element.  But, the notes element is still defined in oval-def:DefinitionType.
>>
>>This introduces a namespace mismatch between <notes> and its child <note> —
>>which causes validation errors when parsing a 5.10 OVAL document with a 5.11
>>parser.  We need to fix this in OVAL 5.11.1, as it’s a serious backwards-
>>compatibility issue.
>>
>>Best regards,
>>—David Solin
>>[hidden email]
>>
>>To unsubscribe, send an email message to [hidden email] with
>>SIGNOFF OVAL-DEVELOPER-LIST
>>in the BODY of the message.  If you have difficulties, write to OVAL-DEVELOPER-
>>[hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----
> From: "Dan Haynes" <[hidden email]>
> To: [hidden email]
> Sent: Tuesday, March 3, 2015 3:51:56 AM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> I forgot to include it, but, I added a tracker for this issue.
>
> https://github.com/OVALProject/Language/issues/237

I attached a pragmatic fix of the issue that keeps compatibility with
both OVAL 5.11 and 5.10.1.

Unfortunately it now allows empty <notes/> elements and I couldn't
figure out how to get rid of that. One solution is <xsd:assert/> but
that's XSD 1.1 only. Or it could be checked in schematron.

See:
https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f

--
Martin Preisler
Security Technologies | Red Hat, Inc.

> >-----Original Message-----
> >From: Haynes, Dan [mailto:[hidden email]]
> >Sent: Monday, March 02, 2015 9:06 PM
> >To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >(Suspect the fix for issue #6 was the culprit)
> >
> >Yes, this change was made to support notes in variables.  I agree it is a
> >serious
> >backwards compatibility issue and it can be addressed with the 5.11.1
> >release.
> >Thanks for catching this and reporting it!
> >
> >-Danny
> >
> >>-----Original Message-----
> >>From: David Solin [mailto:[hidden email]]
> >>Sent: Monday, March 02, 2015 12:06 PM
> >>To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>(Suspect
> >>the fix for issue #6 was the culprit)
> >>
> >>Hi Everyone,
> >>
> >>In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
> >>probably so that it could be used with variables.  This type defines the
> >>note
> >>element.  But, the notes element is still defined in
> >>oval-def:DefinitionType.
> >>
> >>This introduces a namespace mismatch between <notes> and its child <note> —
> >>which causes validation errors when parsing a 5.10 OVAL document with a
> >>5.11
> >>parser.  We need to fix this in OVAL 5.11.1, as it’s a serious backwards-
> >>compatibility issue.
> >>
> >>Best regards,
> >>—David Solin
> >>[hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
Hi Martin,

I spent most of Monday trying to come up with a similar solution, but I ended up giving up on it.  The reason was that every solution violates strict schema validation rules: cos-element-consistent, and likely also cos-nonambig.  Your solution unfortunately violates both.

I fear the only workable solution that gives you a valid schema is to just reintroduce oval-def:NotesType, and use it everywhere except in the oval-variables-schema.

Best regards,
—David Solin
[hidden email]


> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "Dan Haynes" <[hidden email]>
>> To: [hidden email]
>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>> I forgot to include it, but, I added a tracker for this issue.
>>
>> https://github.com/OVALProject/Language/issues/237
>
> I attached a pragmatic fix of the issue that keeps compatibility with
> both OVAL 5.11 and 5.10.1.
>
> Unfortunately it now allows empty <notes/> elements and I couldn't
> figure out how to get rid of that. One solution is <xsd:assert/> but
> that's XSD 1.1 only. Or it could be checked in schematron.
>
> See:
> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>>> -----Original Message-----
>>> From: Haynes, Dan [mailto:[hidden email]]
>>> Sent: Monday, March 02, 2015 9:06 PM
>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>> (Suspect the fix for issue #6 was the culprit)
>>>
>>> Yes, this change was made to support notes in variables.  I agree it is a
>>> serious
>>> backwards compatibility issue and it can be addressed with the 5.11.1
>>> release.
>>> Thanks for catching this and reporting it!
>>>
>>> -Danny
>>>
>>>> -----Original Message-----
>>>> From: David Solin [mailto:[hidden email]]
>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect
>>>> the fix for issue #6 was the culprit)
>>>>
>>>> Hi Everyone,
>>>>
>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
>>>> probably so that it could be used with variables.  This type defines the
>>>> note
>>>> element.  But, the notes element is still defined in
>>>> oval-def:DefinitionType.
>>>>
>>>> This introduces a namespace mismatch between <notes> and its child <note> —
>>>> which causes validation errors when parsing a 5.10 OVAL document with a
>>>> 5.11
>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious backwards-
>>>> compatibility issue.
>>>>
>>>> Best regards,
>>>> —David Solin
>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, March 19, 2015 3:37:19 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> Hi Martin,
>
> I spent most of Monday trying to come up with a similar solution, but I ended
> up giving up on it.  The reason was that every solution violates strict
> schema validation rules: cos-element-consistent, and likely also
> cos-nonambig.  Your solution unfortunately violates both.

Yes, solving these sort of issues in XSD is not trivial. I tried a few obvious
fixes like using element/@ref but those were causing the errors you mention.
The <xsd:extension/> solution works fine with libxml2.

How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
the errors?

> I fear the only workable solution that gives you a valid schema is to just
> reintroduce oval-def:NotesType, and use it everywhere except in the
> oval-variables-schema.

Doesn't this break compatibility with 5.11? In 5.11 content can have
<notes><oval:note>abc</oval:note></notes>.

If I understand it correctly, your solution marks this as invalid content.

The final solution should keep compat with both 5.10.1 and 5.11.

One other solution that I almost ended up using was to just use <xsd:any/>
with appropriate namespace and then validate that it's either oval:note or
oval-def:note in schematron.

--
Martin Preisler
Security Technologies | Red Hat, Inc.

> Best regards,
> —David Solin
> [hidden email]
>
>
> > On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]> wrote:
> >
> > ----- Original Message -----
> >> From: "Dan Haynes" <[hidden email]>
> >> To: [hidden email]
> >> Sent: Tuesday, March 3, 2015 3:51:56 AM
> >> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >> (Suspect the fix for issue #6 was the
> >> culprit)
> >>
> >> I forgot to include it, but, I added a tracker for this issue.
> >>
> >> https://github.com/OVALProject/Language/issues/237
> >
> > I attached a pragmatic fix of the issue that keeps compatibility with
> > both OVAL 5.11 and 5.10.1.
> >
> > Unfortunately it now allows empty <notes/> elements and I couldn't
> > figure out how to get rid of that. One solution is <xsd:assert/> but
> > that's XSD 1.1 only. Or it could be checked in schematron.
> >
> > See:
> > https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
> >
> > --
> > Martin Preisler
> > Security Technologies | Red Hat, Inc.
> >
> >>> -----Original Message-----
> >>> From: Haynes, Dan [mailto:[hidden email]]
> >>> Sent: Monday, March 02, 2015 9:06 PM
> >>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>> (Suspect the fix for issue #6 was the culprit)
> >>>
> >>> Yes, this change was made to support notes in variables.  I agree it is a
> >>> serious
> >>> backwards compatibility issue and it can be addressed with the 5.11.1
> >>> release.
> >>> Thanks for catching this and reporting it!
> >>>
> >>> -Danny
> >>>
> >>>> -----Original Message-----
> >>>> From: David Solin [mailto:[hidden email]]
> >>>> Sent: Monday, March 02, 2015 12:06 PM
> >>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>> (Suspect
> >>>> the fix for issue #6 was the culprit)
> >>>>
> >>>> Hi Everyone,
> >>>>
> >>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
> >>>> probably so that it could be used with variables.  This type defines the
> >>>> note
> >>>> element.  But, the notes element is still defined in
> >>>> oval-def:DefinitionType.
> >>>>
> >>>> This introduces a namespace mismatch between <notes> and its child
> >>>> <note> —
> >>>> which causes validation errors when parsing a 5.10 OVAL document with a
> >>>> 5.11
> >>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
> >>>> backwards-
> >>>> compatibility issue.
> >>>>
> >>>> Best regards,
> >>>> —David Solin
> >>>> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Thursday, March 19, 2015 3:37:19 PM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>> Hi Martin,
>>
>> I spent most of Monday trying to come up with a similar solution, but I ended
>> up giving up on it.  The reason was that every solution violates strict
>> schema validation rules: cos-element-consistent, and likely also
>> cos-nonambig.  Your solution unfortunately violates both.
>
> Yes, solving these sort of issues in XSD is not trivial. I tried a few obvious
> fixes like using element/@ref but those were causing the errors you mention.
> The <xsd:extension/> solution works fine with libxml2.
>
> How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
> the errors?
>

I learned of the errors from the JAXB compiler.  There are errors like the following for every object type (i.e., TONS of errors):

[ERROR] cos-element-consistent: Error for type '#AnonType_interface_object'. Multiple elements with name 'notes', with different types, appear in the model group.
  line 3299 of file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd

[ERROR] cos-nonambig: "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements from their substitution group) violate "Unique Particle Attribution". During validation against this schema, ambiguity would be created for those two particles.
  line 3299 of file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd


Reading online, XMLSpy does validate, if you have that available.


>> I fear the only workable solution that gives you a valid schema is to just
>> reintroduce oval-def:NotesType, and use it everywhere except in the
>> oval-variables-schema.
>
> Doesn't this break compatibility with 5.11? In 5.11 content can have
> <notes><oval:note>abc</oval:note></notes>.
>


Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I figure, no harm...


> If I understand it correctly, your solution marks this as invalid content.
>
> The final solution should keep compat with both 5.10.1 and 5.11.
>
> One other solution that I almost ended up using was to just use <xsd:any/>
> with appropriate namespace and then validate that it's either oval:note or
> oval-def:note in schematron.


I bet that might work.  But it’s horrible, isn’t it?!


>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>> Best regards,
>> —David Solin
>> [hidden email]
>>
>>
>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]> wrote:
>>>
>>> ----- Original Message -----
>>>> From: "Dan Haynes" <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect the fix for issue #6 was the
>>>> culprit)
>>>>
>>>> I forgot to include it, but, I added a tracker for this issue.
>>>>
>>>> https://github.com/OVALProject/Language/issues/237
>>>
>>> I attached a pragmatic fix of the issue that keeps compatibility with
>>> both OVAL 5.11 and 5.10.1.
>>>
>>> Unfortunately it now allows empty <notes/> elements and I couldn't
>>> figure out how to get rid of that. One solution is <xsd:assert/> but
>>> that's XSD 1.1 only. Or it could be checked in schematron.
>>>
>>> See:
>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>>>
>>> --
>>> Martin Preisler
>>> Security Technologies | Red Hat, Inc.
>>>
>>>>> -----Original Message-----
>>>>> From: Haynes, Dan [mailto:[hidden email]]
>>>>> Sent: Monday, March 02, 2015 9:06 PM
>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>> (Suspect the fix for issue #6 was the culprit)
>>>>>
>>>>> Yes, this change was made to support notes in variables.  I agree it is a
>>>>> serious
>>>>> backwards compatibility issue and it can be addressed with the 5.11.1
>>>>> release.
>>>>> Thanks for catching this and reporting it!
>>>>>
>>>>> -Danny
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: David Solin [mailto:[hidden email]]
>>>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>> (Suspect
>>>>>> the fix for issue #6 was the culprit)
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
>>>>>> probably so that it could be used with variables.  This type defines the
>>>>>> note
>>>>>> element.  But, the notes element is still defined in
>>>>>> oval-def:DefinitionType.
>>>>>>
>>>>>> This introduces a namespace mismatch between <notes> and its child
>>>>>> <note> —
>>>>>> which causes validation errors when parsing a 5.10 OVAL document with a
>>>>>> 5.11
>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
>>>>>> backwards-
>>>>>> compatibility issue.
>>>>>>
>>>>>> Best regards,
>>>>>> —David Solin
>>>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

David Solin
[hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, March 19, 2015 4:24:51 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> > On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]> wrote:
> >
> > ----- Original Message -----
> >> From: "David Solin" <[hidden email]>
> >> To: [hidden email]
> >> Sent: Thursday, March 19, 2015 3:37:19 PM
> >> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >> (Suspect the fix for issue #6 was the
> >> culprit)
> >>
> >> Hi Martin,
> >>
> >> I spent most of Monday trying to come up with a similar solution, but I
> >> ended
> >> up giving up on it.  The reason was that every solution violates strict
> >> schema validation rules: cos-element-consistent, and likely also
> >> cos-nonambig.  Your solution unfortunately violates both.
> >
> > Yes, solving these sort of issues in XSD is not trivial. I tried a few
> > obvious
> > fixes like using element/@ref but those were causing the errors you
> > mention.
> > The <xsd:extension/> solution works fine with libxml2.
> >
> > How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
> > the errors?
> >
>
> I learned of the errors from the JAXB compiler.  There are errors like the
> following for every object type (i.e., TONS of errors):
>
> [ERROR] cos-element-consistent: Error for type '#AnonType_interface_object'.
> Multiple elements with name 'notes', with different types, appear in the
> model group.
>   line 3299 of
>   file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>
> [ERROR] cos-nonambig:
> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements from
> their substitution group) violate "Unique Particle Attribution". During
> validation against this schema, ambiguity would be created for those two
> particles.
>   line 3299 of
>   file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd

Hmm, libxml2 is fine with it but I agree that this is a blocker.
We need a better fix for this.

> Reading online, XMLSpy does validate, if you have that available.
>
>
> >> I fear the only workable solution that gives you a valid schema is to just
> >> reintroduce oval-def:NotesType, and use it everywhere except in the
> >> oval-variables-schema.
> >
> > Doesn't this break compatibility with 5.11? In 5.11 content can have
> > <notes><oval:note>abc</oval:note></notes>.
> >
>
>
> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I figure, no
> harm...

Actually...

https://github.com/OpenSCAP/scap-security-guide/pull/478

The good news is that the notes elements are commented until this is fixed.

> > If I understand it correctly, your solution marks this as invalid content.
> >
> > The final solution should keep compat with both 5.10.1 and 5.11.
> >
> > One other solution that I almost ended up using was to just use <xsd:any/>
> > with appropriate namespace and then validate that it's either oval:note or
> > oval-def:note in schematron.
>
>
> I bet that might work.  But it’s horrible, isn’t it?!

Yes! But at the same time it allows people to write OVAL 5.11 right now and be
safe that it will validate against 5.11.1. The ends may justify the means
here :-)

I think that the <xsd:any/> + schematron solution is our best bet right now.
Any suggestions?

--
Martin Preisler
Security Technologies | Red Hat, Inc.

> > --
> > Martin Preisler
> > Security Technologies | Red Hat, Inc.
> >
> >> Best regards,
> >> —David Solin
> >> [hidden email]
> >>
> >>
> >>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]> wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "Dan Haynes" <[hidden email]>
> >>>> To: [hidden email]
> >>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
> >>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>> (Suspect the fix for issue #6 was the
> >>>> culprit)
> >>>>
> >>>> I forgot to include it, but, I added a tracker for this issue.
> >>>>
> >>>> https://github.com/OVALProject/Language/issues/237
> >>>
> >>> I attached a pragmatic fix of the issue that keeps compatibility with
> >>> both OVAL 5.11 and 5.10.1.
> >>>
> >>> Unfortunately it now allows empty <notes/> elements and I couldn't
> >>> figure out how to get rid of that. One solution is <xsd:assert/> but
> >>> that's XSD 1.1 only. Or it could be checked in schematron.
> >>>
> >>> See:
> >>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
> >>>
> >>> --
> >>> Martin Preisler
> >>> Security Technologies | Red Hat, Inc.
> >>>
> >>>>> -----Original Message-----
> >>>>> From: Haynes, Dan [mailto:[hidden email]]
> >>>>> Sent: Monday, March 02, 2015 9:06 PM
> >>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>>> (Suspect the fix for issue #6 was the culprit)
> >>>>>
> >>>>> Yes, this change was made to support notes in variables.  I agree it is
> >>>>> a
> >>>>> serious
> >>>>> backwards compatibility issue and it can be addressed with the 5.11.1
> >>>>> release.
> >>>>> Thanks for catching this and reporting it!
> >>>>>
> >>>>> -Danny
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: David Solin [mailto:[hidden email]]
> >>>>>> Sent: Monday, March 02, 2015 12:06 PM
> >>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>>>> (Suspect
> >>>>>> the fix for issue #6 was the culprit)
> >>>>>>
> >>>>>> Hi Everyone,
> >>>>>>
> >>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
> >>>>>> probably so that it could be used with variables.  This type defines
> >>>>>> the
> >>>>>> note
> >>>>>> element.  But, the notes element is still defined in
> >>>>>> oval-def:DefinitionType.
> >>>>>>
> >>>>>> This introduces a namespace mismatch between <notes> and its child
> >>>>>> <note> —
> >>>>>> which causes validation errors when parsing a 5.10 OVAL document with
> >>>>>> a
> >>>>>> 5.11
> >>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
> >>>>>> backwards-
> >>>>>> compatibility issue.
> >>>>>>
> >>>>>> Best regards,
> >>>>>> —David Solin
> >>>>>> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
I think the problem might be solvable using a substitution group.

That is how OVAL objects, tests and states all work, and there are examples with the same element name but different types, e.g., unix-def:file_test and win-def:file_test.

WDYT?

—David Solin
[hidden email]



> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]> wrote:
>
> I think that the <xsd:any/> + schematron solution is our best bet right now.
> Any suggestions?

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
In reply to this post by Martin Preisler
Hi Martin, etc.,

Here, I’ve gotten a substitution group working:
https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf

(The diff is taken based on a previous set of changes, where I had abandoned this approach in favor of just re-introducing the pre-5.11 oval-def:NotesType).

The only imperfect side-effect is that an <oval:notes> is permitted to have no <oval:note> children — otherwise the substitution group added for legacy content support would require at least one, which we don’t want.

Let me know what you all think of this approach!

Best,
—David Solin
[hidden email]



> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Thursday, March 19, 2015 4:24:51 PM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]> wrote:
>>>
>>> ----- Original Message -----
>>>> From: "David Solin" <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect the fix for issue #6 was the
>>>> culprit)
>>>>
>>>> Hi Martin,
>>>>
>>>> I spent most of Monday trying to come up with a similar solution, but I
>>>> ended
>>>> up giving up on it.  The reason was that every solution violates strict
>>>> schema validation rules: cos-element-consistent, and likely also
>>>> cos-nonambig.  Your solution unfortunately violates both.
>>>
>>> Yes, solving these sort of issues in XSD is not trivial. I tried a few
>>> obvious
>>> fixes like using element/@ref but those were causing the errors you
>>> mention.
>>> The <xsd:extension/> solution works fine with libxml2.
>>>
>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
>>> the errors?
>>>
>>
>> I learned of the errors from the JAXB compiler.  There are errors like the
>> following for every object type (i.e., TONS of errors):
>>
>> [ERROR] cos-element-consistent: Error for type '#AnonType_interface_object'.
>> Multiple elements with name 'notes', with different types, appear in the
>> model group.
>>  line 3299 of
>>  file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>
>> [ERROR] cos-nonambig:
>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements from
>> their substitution group) violate "Unique Particle Attribution". During
>> validation against this schema, ambiguity would be created for those two
>> particles.
>>  line 3299 of
>>  file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>
> Hmm, libxml2 is fine with it but I agree that this is a blocker.
> We need a better fix for this.
>
>> Reading online, XMLSpy does validate, if you have that available.
>>
>>
>>>> I fear the only workable solution that gives you a valid schema is to just
>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
>>>> oval-variables-schema.
>>>
>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
>>> <notes><oval:note>abc</oval:note></notes>.
>>>
>>
>>
>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I figure, no
>> harm...
>
> Actually...
>
> https://github.com/OpenSCAP/scap-security-guide/pull/478
>
> The good news is that the notes elements are commented until this is fixed.
>
>>> If I understand it correctly, your solution marks this as invalid content.
>>>
>>> The final solution should keep compat with both 5.10.1 and 5.11.
>>>
>>> One other solution that I almost ended up using was to just use <xsd:any/>
>>> with appropriate namespace and then validate that it's either oval:note or
>>> oval-def:note in schematron.
>>
>>
>> I bet that might work.  But it’s horrible, isn’t it?!
>
> Yes! But at the same time it allows people to write OVAL 5.11 right now and be
> safe that it will validate against 5.11.1. The ends may justify the means
> here :-)
>
> I think that the <xsd:any/> + schematron solution is our best bet right now.
> Any suggestions?
>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>>> --
>>> Martin Preisler
>>> Security Technologies | Red Hat, Inc.
>>>
>>>> Best regards,
>>>> —David Solin
>>>> [hidden email]
>>>>
>>>>
>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]> wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Dan Haynes" <[hidden email]>
>>>>>> To: [hidden email]
>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>> (Suspect the fix for issue #6 was the
>>>>>> culprit)
>>>>>>
>>>>>> I forgot to include it, but, I added a tracker for this issue.
>>>>>>
>>>>>> https://github.com/OVALProject/Language/issues/237
>>>>>
>>>>> I attached a pragmatic fix of the issue that keeps compatibility with
>>>>> both OVAL 5.11 and 5.10.1.
>>>>>
>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
>>>>> figure out how to get rid of that. One solution is <xsd:assert/> but
>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
>>>>>
>>>>> See:
>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>>>>>
>>>>> --
>>>>> Martin Preisler
>>>>> Security Technologies | Red Hat, Inc.
>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>>> (Suspect the fix for issue #6 was the culprit)
>>>>>>>
>>>>>>> Yes, this change was made to support notes in variables.  I agree it is
>>>>>>> a
>>>>>>> serious
>>>>>>> backwards compatibility issue and it can be addressed with the 5.11.1
>>>>>>> release.
>>>>>>> Thanks for catching this and reporting it!
>>>>>>>
>>>>>>> -Danny
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: David Solin [mailto:[hidden email]]
>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>>>> (Suspect
>>>>>>>> the fix for issue #6 was the culprit)
>>>>>>>>
>>>>>>>> Hi Everyone,
>>>>>>>>
>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL common,
>>>>>>>> probably so that it could be used with variables.  This type defines
>>>>>>>> the
>>>>>>>> note
>>>>>>>> element.  But, the notes element is still defined in
>>>>>>>> oval-def:DefinitionType.
>>>>>>>>
>>>>>>>> This introduces a namespace mismatch between <notes> and its child
>>>>>>>> <note> —
>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document with
>>>>>>>> a
>>>>>>>> 5.11
>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
>>>>>>>> backwards-
>>>>>>>> compatibility issue.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> —David Solin
>>>>>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, March 19, 2015 9:40:44 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> Hi Martin, etc.,
>
> Here, I’ve gotten a substitution group working:
> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
>
> (The diff is taken based on a previous set of changes, where I had abandoned
> this approach in favor of just re-introducing the pre-5.11
> oval-def:NotesType).
>
> The only imperfect side-effect is that an <oval:notes> is permitted to have
> no <oval:note> children — otherwise the substitution group added for legacy
> content support would require at least one, which we don’t want.
>
> Let me know what you all think of this approach!

Pretty clever solution, kudos!

It doesn't work with libxml2 or I am doing something wrong. It does not validate
<notes><oval-common:note>abc</oval-common:note></notes>. This is the message:

File './test_probes_systemdunitproperty.xml' line 24: Element '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing child element(s). Expected is one of ( {http://oval.mitre.org/XMLSchema/oval-common-5}note, {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
OpenSCAP Error: Invalid OVAL Definition (5.11) content in ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]

What's confusing to me is that the error message suggests that it should
accept oval-common:note. If I add oval-def:note the error disappears and
everything works as expected.

Is the schema supposed to reject
<notes><oval-common:note>a</oval-common:note></notes>?

If I change the first hunk in oval-definitions-schema.xsd to:
<xsd:complexType>
 <xsd:complexContent>
   <xsd:extension base="oval:NotesType">
     <xsd:sequence>
       <xsd:element name="note" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
     </xsd:sequence>
   </xsd:extension>
 </xsd:complexContent>
</xsd:complexType>

Then everything works as expected. Since OVAL 5.11 validates oval-common:note inside notes
I think the changed schema should as well. Will wait for confirmation before I commit
this to the OpenSCAP repo. Maybe I am misunderstanding something...

--
Martin Preisler
Security Technologies | Red Hat, Inc.

>
> Best,
> —David Solin
> [hidden email]
>
>
>
> > On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]> wrote:
> >
> > ----- Original Message -----
> >> From: "David Solin" <[hidden email]>
> >> To: [hidden email]
> >> Sent: Thursday, March 19, 2015 4:24:51 PM
> >> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >> (Suspect the fix for issue #6 was the
> >> culprit)
> >>
> >>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
> >>> wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "David Solin" <[hidden email]>
> >>>> To: [hidden email]
> >>>> Sent: Thursday, March 19, 2015 3:37:19 PM
> >>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>> (Suspect the fix for issue #6 was the
> >>>> culprit)
> >>>>
> >>>> Hi Martin,
> >>>>
> >>>> I spent most of Monday trying to come up with a similar solution, but I
> >>>> ended
> >>>> up giving up on it.  The reason was that every solution violates strict
> >>>> schema validation rules: cos-element-consistent, and likely also
> >>>> cos-nonambig.  Your solution unfortunately violates both.
> >>>
> >>> Yes, solving these sort of issues in XSD is not trivial. I tried a few
> >>> obvious
> >>> fixes like using element/@ref but those were causing the errors you
> >>> mention.
> >>> The <xsd:extension/> solution works fine with libxml2.
> >>>
> >>> How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
> >>> the errors?
> >>>
> >>
> >> I learned of the errors from the JAXB compiler.  There are errors like the
> >> following for every object type (i.e., TONS of errors):
> >>
> >> [ERROR] cos-element-consistent: Error for type
> >> '#AnonType_interface_object'.
> >> Multiple elements with name 'notes', with different types, appear in the
> >> model group.
> >>  line 3299 of
> >>  file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >>
> >> [ERROR] cos-nonambig:
> >> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
> >> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements
> >> from
> >> their substitution group) violate "Unique Particle Attribution". During
> >> validation against this schema, ambiguity would be created for those two
> >> particles.
> >>  line 3299 of
> >>  file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >
> > Hmm, libxml2 is fine with it but I agree that this is a blocker.
> > We need a better fix for this.
> >
> >> Reading online, XMLSpy does validate, if you have that available.
> >>
> >>
> >>>> I fear the only workable solution that gives you a valid schema is to
> >>>> just
> >>>> reintroduce oval-def:NotesType, and use it everywhere except in the
> >>>> oval-variables-schema.
> >>>
> >>> Doesn't this break compatibility with 5.11? In 5.11 content can have
> >>> <notes><oval:note>abc</oval:note></notes>.
> >>>
> >>
> >>
> >> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I figure,
> >> no
> >> harm...
> >
> > Actually...
> >
> > https://github.com/OpenSCAP/scap-security-guide/pull/478
> >
> > The good news is that the notes elements are commented until this is fixed.
> >
> >>> If I understand it correctly, your solution marks this as invalid
> >>> content.
> >>>
> >>> The final solution should keep compat with both 5.10.1 and 5.11.
> >>>
> >>> One other solution that I almost ended up using was to just use
> >>> <xsd:any/>
> >>> with appropriate namespace and then validate that it's either oval:note
> >>> or
> >>> oval-def:note in schematron.
> >>
> >>
> >> I bet that might work.  But it’s horrible, isn’t it?!
> >
> > Yes! But at the same time it allows people to write OVAL 5.11 right now and
> > be
> > safe that it will validate against 5.11.1. The ends may justify the means
> > here :-)
> >
> > I think that the <xsd:any/> + schematron solution is our best bet right
> > now.
> > Any suggestions?
> >
> > --
> > Martin Preisler
> > Security Technologies | Red Hat, Inc.
> >
> >>> --
> >>> Martin Preisler
> >>> Security Technologies | Red Hat, Inc.
> >>>
> >>>> Best regards,
> >>>> —David Solin
> >>>> [hidden email]
> >>>>
> >>>>
> >>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Dan Haynes" <[hidden email]>
> >>>>>> To: [hidden email]
> >>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
> >>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>> issue
> >>>>>> (Suspect the fix for issue #6 was the
> >>>>>> culprit)
> >>>>>>
> >>>>>> I forgot to include it, but, I added a tracker for this issue.
> >>>>>>
> >>>>>> https://github.com/OVALProject/Language/issues/237
> >>>>>
> >>>>> I attached a pragmatic fix of the issue that keeps compatibility with
> >>>>> both OVAL 5.11 and 5.10.1.
> >>>>>
> >>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
> >>>>> figure out how to get rid of that. One solution is <xsd:assert/> but
> >>>>> that's XSD 1.1 only. Or it could be checked in schematron.
> >>>>>
> >>>>> See:
> >>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
> >>>>>
> >>>>> --
> >>>>> Martin Preisler
> >>>>> Security Technologies | Red Hat, Inc.
> >>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Haynes, Dan [mailto:[hidden email]]
> >>>>>>> Sent: Monday, March 02, 2015 9:06 PM
> >>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>> issue
> >>>>>>> (Suspect the fix for issue #6 was the culprit)
> >>>>>>>
> >>>>>>> Yes, this change was made to support notes in variables.  I agree it
> >>>>>>> is
> >>>>>>> a
> >>>>>>> serious
> >>>>>>> backwards compatibility issue and it can be addressed with the 5.11.1
> >>>>>>> release.
> >>>>>>> Thanks for catching this and reporting it!
> >>>>>>>
> >>>>>>> -Danny
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: David Solin [mailto:[hidden email]]
> >>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
> >>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
> >>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>>>>>> (Suspect
> >>>>>>>> the fix for issue #6 was the culprit)
> >>>>>>>>
> >>>>>>>> Hi Everyone,
> >>>>>>>>
> >>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
> >>>>>>>> common,
> >>>>>>>> probably so that it could be used with variables.  This type defines
> >>>>>>>> the
> >>>>>>>> note
> >>>>>>>> element.  But, the notes element is still defined in
> >>>>>>>> oval-def:DefinitionType.
> >>>>>>>>
> >>>>>>>> This introduces a namespace mismatch between <notes> and its child
> >>>>>>>> <note> —
> >>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
> >>>>>>>> with
> >>>>>>>> a
> >>>>>>>> 5.11
> >>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
> >>>>>>>> backwards-
> >>>>>>>> compatibility issue.
> >>>>>>>>
> >>>>>>>> Best regards,
> >>>>>>>> —David Solin
> >>>>>>>> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
Hi Martin,

Is it possible this is just a simple namespace prefix problem, i.e., there is no xmlns:oval-common="http://oval.mitre.org/XMLSchema/oval-common-5”?

In most OVAL documents, the xml namespace prefix for common is simply “oval” (e.g., <oval:note>)

Regards,
—David Solin
[hidden email]



> On Mar 24, 2015, at 1:06 PM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Thursday, March 19, 2015 9:40:44 PM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>> Hi Martin, etc.,
>>
>> Here, I’ve gotten a substitution group working:
>> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
>>
>> (The diff is taken based on a previous set of changes, where I had abandoned
>> this approach in favor of just re-introducing the pre-5.11
>> oval-def:NotesType).
>>
>> The only imperfect side-effect is that an <oval:notes> is permitted to have
>> no <oval:note> children — otherwise the substitution group added for legacy
>> content support would require at least one, which we don’t want.
>>
>> Let me know what you all think of this approach!
>
> Pretty clever solution, kudos!
>
> It doesn't work with libxml2 or I am doing something wrong. It does not validate
> <notes><oval-common:note>abc</oval-common:note></notes>. This is the message:
>
> File './test_probes_systemdunitproperty.xml' line 24: Element '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing child element(s). Expected is one of ( {http://oval.mitre.org/XMLSchema/oval-common-5}note, {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
> OpenSCAP Error: Invalid OVAL Definition (5.11) content in ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]
>
> What's confusing to me is that the error message suggests that it should
> accept oval-common:note. If I add oval-def:note the error disappears and
> everything works as expected.
>
> Is the schema supposed to reject
> <notes><oval-common:note>a</oval-common:note></notes>?
>
> If I change the first hunk in oval-definitions-schema.xsd to:
> <xsd:complexType>
> <xsd:complexContent>
>   <xsd:extension base="oval:NotesType">
>     <xsd:sequence>
>       <xsd:element name="note" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
>     </xsd:sequence>
>   </xsd:extension>
> </xsd:complexContent>
> </xsd:complexType>
>
> Then everything works as expected. Since OVAL 5.11 validates oval-common:note inside notes
> I think the changed schema should as well. Will wait for confirmation before I commit
> this to the OpenSCAP repo. Maybe I am misunderstanding something...
>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>>
>> Best,
>> —David Solin
>> [hidden email]
>>
>>
>>
>>> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]> wrote:
>>>
>>> ----- Original Message -----
>>>> From: "David Solin" <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Thursday, March 19, 2015 4:24:51 PM
>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect the fix for issue #6 was the
>>>> culprit)
>>>>
>>>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "David Solin" <[hidden email]>
>>>>>> To: [hidden email]
>>>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>> (Suspect the fix for issue #6 was the
>>>>>> culprit)
>>>>>>
>>>>>> Hi Martin,
>>>>>>
>>>>>> I spent most of Monday trying to come up with a similar solution, but I
>>>>>> ended
>>>>>> up giving up on it.  The reason was that every solution violates strict
>>>>>> schema validation rules: cos-element-consistent, and likely also
>>>>>> cos-nonambig.  Your solution unfortunately violates both.
>>>>>
>>>>> Yes, solving these sort of issues in XSD is not trivial. I tried a few
>>>>> obvious
>>>>> fixes like using element/@ref but those were causing the errors you
>>>>> mention.
>>>>> The <xsd:extension/> solution works fine with libxml2.
>>>>>
>>>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you paste
>>>>> the errors?
>>>>>
>>>>
>>>> I learned of the errors from the JAXB compiler.  There are errors like the
>>>> following for every object type (i.e., TONS of errors):
>>>>
>>>> [ERROR] cos-element-consistent: Error for type
>>>> '#AnonType_interface_object'.
>>>> Multiple elements with name 'notes', with different types, appear in the
>>>> model group.
>>>> line 3299 of
>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>>
>>>> [ERROR] cos-nonambig:
>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements
>>>> from
>>>> their substitution group) violate "Unique Particle Attribution". During
>>>> validation against this schema, ambiguity would be created for those two
>>>> particles.
>>>> line 3299 of
>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>
>>> Hmm, libxml2 is fine with it but I agree that this is a blocker.
>>> We need a better fix for this.
>>>
>>>> Reading online, XMLSpy does validate, if you have that available.
>>>>
>>>>
>>>>>> I fear the only workable solution that gives you a valid schema is to
>>>>>> just
>>>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
>>>>>> oval-variables-schema.
>>>>>
>>>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
>>>>> <notes><oval:note>abc</oval:note></notes>.
>>>>>
>>>>
>>>>
>>>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I figure,
>>>> no
>>>> harm...
>>>
>>> Actually...
>>>
>>> https://github.com/OpenSCAP/scap-security-guide/pull/478
>>>
>>> The good news is that the notes elements are commented until this is fixed.
>>>
>>>>> If I understand it correctly, your solution marks this as invalid
>>>>> content.
>>>>>
>>>>> The final solution should keep compat with both 5.10.1 and 5.11.
>>>>>
>>>>> One other solution that I almost ended up using was to just use
>>>>> <xsd:any/>
>>>>> with appropriate namespace and then validate that it's either oval:note
>>>>> or
>>>>> oval-def:note in schematron.
>>>>
>>>>
>>>> I bet that might work.  But it’s horrible, isn’t it?!
>>>
>>> Yes! But at the same time it allows people to write OVAL 5.11 right now and
>>> be
>>> safe that it will validate against 5.11.1. The ends may justify the means
>>> here :-)
>>>
>>> I think that the <xsd:any/> + schematron solution is our best bet right
>>> now.
>>> Any suggestions?
>>>
>>> --
>>> Martin Preisler
>>> Security Technologies | Red Hat, Inc.
>>>
>>>>> --
>>>>> Martin Preisler
>>>>> Security Technologies | Red Hat, Inc.
>>>>>
>>>>>> Best regards,
>>>>>> —David Solin
>>>>>> [hidden email]
>>>>>>
>>>>>>
>>>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Dan Haynes" <[hidden email]>
>>>>>>>> To: [hidden email]
>>>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>> issue
>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>> culprit)
>>>>>>>>
>>>>>>>> I forgot to include it, but, I added a tracker for this issue.
>>>>>>>>
>>>>>>>> https://github.com/OVALProject/Language/issues/237
>>>>>>>
>>>>>>> I attached a pragmatic fix of the issue that keeps compatibility with
>>>>>>> both OVAL 5.11 and 5.10.1.
>>>>>>>
>>>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
>>>>>>> figure out how to get rid of that. One solution is <xsd:assert/> but
>>>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
>>>>>>>
>>>>>>> See:
>>>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>>>>>>>
>>>>>>> --
>>>>>>> Martin Preisler
>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
>>>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>> issue
>>>>>>>>> (Suspect the fix for issue #6 was the culprit)
>>>>>>>>>
>>>>>>>>> Yes, this change was made to support notes in variables.  I agree it
>>>>>>>>> is
>>>>>>>>> a
>>>>>>>>> serious
>>>>>>>>> backwards compatibility issue and it can be addressed with the 5.11.1
>>>>>>>>> release.
>>>>>>>>> Thanks for catching this and reporting it!
>>>>>>>>>
>>>>>>>>> -Danny
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: David Solin [mailto:[hidden email]]
>>>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public Discussion
>>>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>>>>>> (Suspect
>>>>>>>>>> the fix for issue #6 was the culprit)
>>>>>>>>>>
>>>>>>>>>> Hi Everyone,
>>>>>>>>>>
>>>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
>>>>>>>>>> common,
>>>>>>>>>> probably so that it could be used with variables.  This type defines
>>>>>>>>>> the
>>>>>>>>>> note
>>>>>>>>>> element.  But, the notes element is still defined in
>>>>>>>>>> oval-def:DefinitionType.
>>>>>>>>>>
>>>>>>>>>> This introduces a namespace mismatch between <notes> and its child
>>>>>>>>>> <note> —
>>>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
>>>>>>>>>> with
>>>>>>>>>> a
>>>>>>>>>> 5.11
>>>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
>>>>>>>>>> backwards-
>>>>>>>>>> compatibility issue.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> —David Solin
>>>>>>>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Wednesday, March 25, 2015 11:52:14 AM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> Hi Martin,
>
> Is it possible this is just a simple namespace prefix problem, i.e., there is
> no xmlns:oval-common="http://oval.mitre.org/XMLSchema/oval-common-5”?
>
> In most OVAL documents, the xml namespace prefix for common is simply “oval”
> (e.g., <oval:note>)

Nope, that is not the problem. Yeah, the namespace prefix was just oval,
I changed the snippet to emphasize that it's
"http://oval.mitre.org/XMLSchema/oval-common-5". The namespace URIs match
exactly, the problem must be elsewhere.

--
Martin Preisler
Security Technologies | Red Hat, Inc.

> Regards,
> —David Solin
> [hidden email]
>
>
>
> > On Mar 24, 2015, at 1:06 PM, Martin Preisler <[hidden email]> wrote:
> >
> > ----- Original Message -----
> >> From: "David Solin" <[hidden email]>
> >> To: [hidden email]
> >> Sent: Thursday, March 19, 2015 9:40:44 PM
> >> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >> (Suspect the fix for issue #6 was the
> >> culprit)
> >>
> >> Hi Martin, etc.,
> >>
> >> Here, I’ve gotten a substitution group working:
> >> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
> >>
> >> (The diff is taken based on a previous set of changes, where I had
> >> abandoned
> >> this approach in favor of just re-introducing the pre-5.11
> >> oval-def:NotesType).
> >>
> >> The only imperfect side-effect is that an <oval:notes> is permitted to
> >> have
> >> no <oval:note> children — otherwise the substitution group added for
> >> legacy
> >> content support would require at least one, which we don’t want.
> >>
> >> Let me know what you all think of this approach!
> >
> > Pretty clever solution, kudos!
> >
> > It doesn't work with libxml2 or I am doing something wrong. It does not
> > validate
> > <notes><oval-common:note>abc</oval-common:note></notes>. This is the
> > message:
> >
> > File './test_probes_systemdunitproperty.xml' line 24: Element
> > '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing child
> > element(s). Expected is one of (
> > {http://oval.mitre.org/XMLSchema/oval-common-5}note,
> > {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
> > OpenSCAP Error: Invalid OVAL Definition (5.11) content in
> > ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]
> >
> > What's confusing to me is that the error message suggests that it should
> > accept oval-common:note. If I add oval-def:note the error disappears and
> > everything works as expected.
> >
> > Is the schema supposed to reject
> > <notes><oval-common:note>a</oval-common:note></notes>?
> >
> > If I change the first hunk in oval-definitions-schema.xsd to:
> > <xsd:complexType>
> > <xsd:complexContent>
> >   <xsd:extension base="oval:NotesType">
> >     <xsd:sequence>
> >       <xsd:element name="note" type="xsd:string" minOccurs="0"
> >       maxOccurs="unbounded"/>
> >     </xsd:sequence>
> >   </xsd:extension>
> > </xsd:complexContent>
> > </xsd:complexType>
> >
> > Then everything works as expected. Since OVAL 5.11 validates
> > oval-common:note inside notes
> > I think the changed schema should as well. Will wait for confirmation
> > before I commit
> > this to the OpenSCAP repo. Maybe I am misunderstanding something...
> >
> > --
> > Martin Preisler
> > Security Technologies | Red Hat, Inc.
> >
> >>
> >> Best,
> >> —David Solin
> >> [hidden email]
> >>
> >>
> >>
> >>> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]>
> >>> wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "David Solin" <[hidden email]>
> >>>> To: [hidden email]
> >>>> Sent: Thursday, March 19, 2015 4:24:51 PM
> >>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>> (Suspect the fix for issue #6 was the
> >>>> culprit)
> >>>>
> >>>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "David Solin" <[hidden email]>
> >>>>>> To: [hidden email]
> >>>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
> >>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>> issue
> >>>>>> (Suspect the fix for issue #6 was the
> >>>>>> culprit)
> >>>>>>
> >>>>>> Hi Martin,
> >>>>>>
> >>>>>> I spent most of Monday trying to come up with a similar solution, but
> >>>>>> I
> >>>>>> ended
> >>>>>> up giving up on it.  The reason was that every solution violates
> >>>>>> strict
> >>>>>> schema validation rules: cos-element-consistent, and likely also
> >>>>>> cos-nonambig.  Your solution unfortunately violates both.
> >>>>>
> >>>>> Yes, solving these sort of issues in XSD is not trivial. I tried a few
> >>>>> obvious
> >>>>> fixes like using element/@ref but those were causing the errors you
> >>>>> mention.
> >>>>> The <xsd:extension/> solution works fine with libxml2.
> >>>>>
> >>>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you
> >>>>> paste
> >>>>> the errors?
> >>>>>
> >>>>
> >>>> I learned of the errors from the JAXB compiler.  There are errors like
> >>>> the
> >>>> following for every object type (i.e., TONS of errors):
> >>>>
> >>>> [ERROR] cos-element-consistent: Error for type
> >>>> '#AnonType_interface_object'.
> >>>> Multiple elements with name 'notes', with different types, appear in the
> >>>> model group.
> >>>> line 3299 of
> >>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >>>>
> >>>> [ERROR] cos-nonambig:
> >>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
> >>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements
> >>>> from
> >>>> their substitution group) violate "Unique Particle Attribution". During
> >>>> validation against this schema, ambiguity would be created for those two
> >>>> particles.
> >>>> line 3299 of
> >>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >>>
> >>> Hmm, libxml2 is fine with it but I agree that this is a blocker.
> >>> We need a better fix for this.
> >>>
> >>>> Reading online, XMLSpy does validate, if you have that available.
> >>>>
> >>>>
> >>>>>> I fear the only workable solution that gives you a valid schema is to
> >>>>>> just
> >>>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
> >>>>>> oval-variables-schema.
> >>>>>
> >>>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
> >>>>> <notes><oval:note>abc</oval:note></notes>.
> >>>>>
> >>>>
> >>>>
> >>>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I
> >>>> figure,
> >>>> no
> >>>> harm...
> >>>
> >>> Actually...
> >>>
> >>> https://github.com/OpenSCAP/scap-security-guide/pull/478
> >>>
> >>> The good news is that the notes elements are commented until this is
> >>> fixed.
> >>>
> >>>>> If I understand it correctly, your solution marks this as invalid
> >>>>> content.
> >>>>>
> >>>>> The final solution should keep compat with both 5.10.1 and 5.11.
> >>>>>
> >>>>> One other solution that I almost ended up using was to just use
> >>>>> <xsd:any/>
> >>>>> with appropriate namespace and then validate that it's either oval:note
> >>>>> or
> >>>>> oval-def:note in schematron.
> >>>>
> >>>>
> >>>> I bet that might work.  But it’s horrible, isn’t it?!
> >>>
> >>> Yes! But at the same time it allows people to write OVAL 5.11 right now
> >>> and
> >>> be
> >>> safe that it will validate against 5.11.1. The ends may justify the means
> >>> here :-)
> >>>
> >>> I think that the <xsd:any/> + schematron solution is our best bet right
> >>> now.
> >>> Any suggestions?
> >>>
> >>> --
> >>> Martin Preisler
> >>> Security Technologies | Red Hat, Inc.
> >>>
> >>>>> --
> >>>>> Martin Preisler
> >>>>> Security Technologies | Red Hat, Inc.
> >>>>>
> >>>>>> Best regards,
> >>>>>> —David Solin
> >>>>>> [hidden email]
> >>>>>>
> >>>>>>
> >>>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "Dan Haynes" <[hidden email]>
> >>>>>>>> To: [hidden email]
> >>>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
> >>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>> issue
> >>>>>>>> (Suspect the fix for issue #6 was the
> >>>>>>>> culprit)
> >>>>>>>>
> >>>>>>>> I forgot to include it, but, I added a tracker for this issue.
> >>>>>>>>
> >>>>>>>> https://github.com/OVALProject/Language/issues/237
> >>>>>>>
> >>>>>>> I attached a pragmatic fix of the issue that keeps compatibility with
> >>>>>>> both OVAL 5.11 and 5.10.1.
> >>>>>>>
> >>>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
> >>>>>>> figure out how to get rid of that. One solution is <xsd:assert/> but
> >>>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
> >>>>>>>
> >>>>>>> See:
> >>>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
> >>>>>>>
> >>>>>>> --
> >>>>>>> Martin Preisler
> >>>>>>> Security Technologies | Red Hat, Inc.
> >>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
> >>>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
> >>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
> >>>>>>>>> Discussion
> >>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>>> issue
> >>>>>>>>> (Suspect the fix for issue #6 was the culprit)
> >>>>>>>>>
> >>>>>>>>> Yes, this change was made to support notes in variables.  I agree
> >>>>>>>>> it
> >>>>>>>>> is
> >>>>>>>>> a
> >>>>>>>>> serious
> >>>>>>>>> backwards compatibility issue and it can be addressed with the
> >>>>>>>>> 5.11.1
> >>>>>>>>> release.
> >>>>>>>>> Thanks for catching this and reporting it!
> >>>>>>>>>
> >>>>>>>>> -Danny
> >>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: David Solin [mailto:[hidden email]]
> >>>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
> >>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
> >>>>>>>>>> Discussion
> >>>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>>>> issue
> >>>>>>>>>> (Suspect
> >>>>>>>>>> the fix for issue #6 was the culprit)
> >>>>>>>>>>
> >>>>>>>>>> Hi Everyone,
> >>>>>>>>>>
> >>>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
> >>>>>>>>>> common,
> >>>>>>>>>> probably so that it could be used with variables.  This type
> >>>>>>>>>> defines
> >>>>>>>>>> the
> >>>>>>>>>> note
> >>>>>>>>>> element.  But, the notes element is still defined in
> >>>>>>>>>> oval-def:DefinitionType.
> >>>>>>>>>>
> >>>>>>>>>> This introduces a namespace mismatch between <notes> and its child
> >>>>>>>>>> <note> —
> >>>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
> >>>>>>>>>> with
> >>>>>>>>>> a
> >>>>>>>>>> 5.11
> >>>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
> >>>>>>>>>> backwards-
> >>>>>>>>>> compatibility issue.
> >>>>>>>>>>
> >>>>>>>>>> Best regards,
> >>>>>>>>>> —David Solin
> >>>>>>>>>> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
So, what is the namespace of the outer <notes> tag?  If it’s oval-def, then this makes perfect sense.

I created the oval-def:notes strictly for compatibility with pre-5.11 oval-def:notes.  That is, it must have at least one oval-def:note.  As a side-effect of being a substitution element, it can optionally have oval:note children as well — but it still must have at least one oval-def:note.  I think that’s why you’re getting that error.

I also deprecated oval-def:notes, so you should (if writing 5.11 content) use the new oval:notes, which may have 0 or more oval:notes, and no other children.

Best regards,
—David Solin
[hidden email]


> On Mar 25, 2015, at 5:57 AM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Wednesday, March 25, 2015 11:52:14 AM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>> Hi Martin,
>>
>> Is it possible this is just a simple namespace prefix problem, i.e., there is
>> no xmlns:oval-common="http://oval.mitre.org/XMLSchema/oval-common-5”?
>>
>> In most OVAL documents, the xml namespace prefix for common is simply “oval”
>> (e.g., <oval:note>)
>
> Nope, that is not the problem. Yeah, the namespace prefix was just oval,
> I changed the snippet to emphasize that it's
> "http://oval.mitre.org/XMLSchema/oval-common-5". The namespace URIs match
> exactly, the problem must be elsewhere.
>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>> Regards,
>> —David Solin
>> [hidden email]
>>
>>
>>
>>> On Mar 24, 2015, at 1:06 PM, Martin Preisler <[hidden email]> wrote:
>>>
>>> ----- Original Message -----
>>>> From: "David Solin" <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Thursday, March 19, 2015 9:40:44 PM
>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect the fix for issue #6 was the
>>>> culprit)
>>>>
>>>> Hi Martin, etc.,
>>>>
>>>> Here, I’ve gotten a substitution group working:
>>>> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
>>>>
>>>> (The diff is taken based on a previous set of changes, where I had
>>>> abandoned
>>>> this approach in favor of just re-introducing the pre-5.11
>>>> oval-def:NotesType).
>>>>
>>>> The only imperfect side-effect is that an <oval:notes> is permitted to
>>>> have
>>>> no <oval:note> children — otherwise the substitution group added for
>>>> legacy
>>>> content support would require at least one, which we don’t want.
>>>>
>>>> Let me know what you all think of this approach!
>>>
>>> Pretty clever solution, kudos!
>>>
>>> It doesn't work with libxml2 or I am doing something wrong. It does not
>>> validate
>>> <notes><oval-common:note>abc</oval-common:note></notes>. This is the
>>> message:
>>>
>>> File './test_probes_systemdunitproperty.xml' line 24: Element
>>> '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing child
>>> element(s). Expected is one of (
>>> {http://oval.mitre.org/XMLSchema/oval-common-5}note,
>>> {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
>>> OpenSCAP Error: Invalid OVAL Definition (5.11) content in
>>> ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]
>>>
>>> What's confusing to me is that the error message suggests that it should
>>> accept oval-common:note. If I add oval-def:note the error disappears and
>>> everything works as expected.
>>>
>>> Is the schema supposed to reject
>>> <notes><oval-common:note>a</oval-common:note></notes>?
>>>
>>> If I change the first hunk in oval-definitions-schema.xsd to:
>>> <xsd:complexType>
>>> <xsd:complexContent>
>>>  <xsd:extension base="oval:NotesType">
>>>    <xsd:sequence>
>>>      <xsd:element name="note" type="xsd:string" minOccurs="0"
>>>      maxOccurs="unbounded"/>
>>>    </xsd:sequence>
>>>  </xsd:extension>
>>> </xsd:complexContent>
>>> </xsd:complexType>
>>>
>>> Then everything works as expected. Since OVAL 5.11 validates
>>> oval-common:note inside notes
>>> I think the changed schema should as well. Will wait for confirmation
>>> before I commit
>>> this to the OpenSCAP repo. Maybe I am misunderstanding something...
>>>
>>> --
>>> Martin Preisler
>>> Security Technologies | Red Hat, Inc.
>>>
>>>>
>>>> Best,
>>>> —David Solin
>>>> [hidden email]
>>>>
>>>>
>>>>
>>>>> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]>
>>>>> wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "David Solin" <[hidden email]>
>>>>>> To: [hidden email]
>>>>>> Sent: Thursday, March 19, 2015 4:24:51 PM
>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>> (Suspect the fix for issue #6 was the
>>>>>> culprit)
>>>>>>
>>>>>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "David Solin" <[hidden email]>
>>>>>>>> To: [hidden email]
>>>>>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>> issue
>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>> culprit)
>>>>>>>>
>>>>>>>> Hi Martin,
>>>>>>>>
>>>>>>>> I spent most of Monday trying to come up with a similar solution, but
>>>>>>>> I
>>>>>>>> ended
>>>>>>>> up giving up on it.  The reason was that every solution violates
>>>>>>>> strict
>>>>>>>> schema validation rules: cos-element-consistent, and likely also
>>>>>>>> cos-nonambig.  Your solution unfortunately violates both.
>>>>>>>
>>>>>>> Yes, solving these sort of issues in XSD is not trivial. I tried a few
>>>>>>> obvious
>>>>>>> fixes like using element/@ref but those were causing the errors you
>>>>>>> mention.
>>>>>>> The <xsd:extension/> solution works fine with libxml2.
>>>>>>>
>>>>>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you
>>>>>>> paste
>>>>>>> the errors?
>>>>>>>
>>>>>>
>>>>>> I learned of the errors from the JAXB compiler.  There are errors like
>>>>>> the
>>>>>> following for every object type (i.e., TONS of errors):
>>>>>>
>>>>>> [ERROR] cos-element-consistent: Error for type
>>>>>> '#AnonType_interface_object'.
>>>>>> Multiple elements with name 'notes', with different types, appear in the
>>>>>> model group.
>>>>>> line 3299 of
>>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>>>>
>>>>>> [ERROR] cos-nonambig:
>>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
>>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or elements
>>>>>> from
>>>>>> their substitution group) violate "Unique Particle Attribution". During
>>>>>> validation against this schema, ambiguity would be created for those two
>>>>>> particles.
>>>>>> line 3299 of
>>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>>>
>>>>> Hmm, libxml2 is fine with it but I agree that this is a blocker.
>>>>> We need a better fix for this.
>>>>>
>>>>>> Reading online, XMLSpy does validate, if you have that available.
>>>>>>
>>>>>>
>>>>>>>> I fear the only workable solution that gives you a valid schema is to
>>>>>>>> just
>>>>>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
>>>>>>>> oval-variables-schema.
>>>>>>>
>>>>>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
>>>>>>> <notes><oval:note>abc</oval:note></notes>.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I
>>>>>> figure,
>>>>>> no
>>>>>> harm...
>>>>>
>>>>> Actually...
>>>>>
>>>>> https://github.com/OpenSCAP/scap-security-guide/pull/478
>>>>>
>>>>> The good news is that the notes elements are commented until this is
>>>>> fixed.
>>>>>
>>>>>>> If I understand it correctly, your solution marks this as invalid
>>>>>>> content.
>>>>>>>
>>>>>>> The final solution should keep compat with both 5.10.1 and 5.11.
>>>>>>>
>>>>>>> One other solution that I almost ended up using was to just use
>>>>>>> <xsd:any/>
>>>>>>> with appropriate namespace and then validate that it's either oval:note
>>>>>>> or
>>>>>>> oval-def:note in schematron.
>>>>>>
>>>>>>
>>>>>> I bet that might work.  But it’s horrible, isn’t it?!
>>>>>
>>>>> Yes! But at the same time it allows people to write OVAL 5.11 right now
>>>>> and
>>>>> be
>>>>> safe that it will validate against 5.11.1. The ends may justify the means
>>>>> here :-)
>>>>>
>>>>> I think that the <xsd:any/> + schematron solution is our best bet right
>>>>> now.
>>>>> Any suggestions?
>>>>>
>>>>> --
>>>>> Martin Preisler
>>>>> Security Technologies | Red Hat, Inc.
>>>>>
>>>>>>> --
>>>>>>> Martin Preisler
>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> —David Solin
>>>>>>>> [hidden email]
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "Dan Haynes" <[hidden email]>
>>>>>>>>>> To: [hidden email]
>>>>>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>> issue
>>>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>>>> culprit)
>>>>>>>>>>
>>>>>>>>>> I forgot to include it, but, I added a tracker for this issue.
>>>>>>>>>>
>>>>>>>>>> https://github.com/OVALProject/Language/issues/237
>>>>>>>>>
>>>>>>>>> I attached a pragmatic fix of the issue that keeps compatibility with
>>>>>>>>> both OVAL 5.11 and 5.10.1.
>>>>>>>>>
>>>>>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
>>>>>>>>> figure out how to get rid of that. One solution is <xsd:assert/> but
>>>>>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
>>>>>>>>>
>>>>>>>>> See:
>>>>>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Martin Preisler
>>>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
>>>>>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
>>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
>>>>>>>>>>> Discussion
>>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>>> issue
>>>>>>>>>>> (Suspect the fix for issue #6 was the culprit)
>>>>>>>>>>>
>>>>>>>>>>> Yes, this change was made to support notes in variables.  I agree
>>>>>>>>>>> it
>>>>>>>>>>> is
>>>>>>>>>>> a
>>>>>>>>>>> serious
>>>>>>>>>>> backwards compatibility issue and it can be addressed with the
>>>>>>>>>>> 5.11.1
>>>>>>>>>>> release.
>>>>>>>>>>> Thanks for catching this and reporting it!
>>>>>>>>>>>
>>>>>>>>>>> -Danny
>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: David Solin [mailto:[hidden email]]
>>>>>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
>>>>>>>>>>>> Discussion
>>>>>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>>>> issue
>>>>>>>>>>>> (Suspect
>>>>>>>>>>>> the fix for issue #6 was the culprit)
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
>>>>>>>>>>>> common,
>>>>>>>>>>>> probably so that it could be used with variables.  This type
>>>>>>>>>>>> defines
>>>>>>>>>>>> the
>>>>>>>>>>>> note
>>>>>>>>>>>> element.  But, the notes element is still defined in
>>>>>>>>>>>> oval-def:DefinitionType.
>>>>>>>>>>>>
>>>>>>>>>>>> This introduces a namespace mismatch between <notes> and its child
>>>>>>>>>>>> <note> —
>>>>>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
>>>>>>>>>>>> with
>>>>>>>>>>>> a
>>>>>>>>>>>> 5.11
>>>>>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
>>>>>>>>>>>> backwards-
>>>>>>>>>>>> compatibility issue.
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> —David Solin
>>>>>>>>>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

Martin Preisler
----- Original Message -----
> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Wednesday, March 25, 2015 1:52:12 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
> culprit)
>
> So, what is the namespace of the outer <notes> tag?  If it’s oval-def, then
> this makes perfect sense.

oval-def

> I created the oval-def:notes strictly for compatibility with pre-5.11
> oval-def:notes.  That is, it must have at least one oval-def:note.  As a
> side-effect of being a substitution element, it can optionally have
> oval:note children as well — but it still must have at least one
> oval-def:note.  I think that’s why you’re getting that error.

OVAL 5.11 allows <oval-def:notes><oval-common:note>a</oval-common:note></oval-def:notes>.
That's why I am testing it. Since it uses <xsd:element name=.. type=../>
the namespace will be the targetNamespace of the XSD, which is oval-def.
It doesn't matter that the type is oval-common:notesType, the namespace
of the notes still needs to be $targetNamespace.

OVAL 5.11 allows <oval-def:notes><oval-common:note>a</oval-common:note></oval-def:notes>
in the oval-def XSD and <oval-common:notes><oval-common:note>b</oval-common:note></oval-common:notes>
in the oval-common XSD.

--
Martin Preisler
Security Technologies | Red Hat, Inc.

> I also deprecated oval-def:notes, so you should (if writing 5.11 content) use
> the new oval:notes, which may have 0 or more oval:notes, and no other
> children.
>
> Best regards,
> —David Solin
> [hidden email]
>
>
> > On Mar 25, 2015, at 5:57 AM, Martin Preisler <[hidden email]> wrote:
> >
> > ----- Original Message -----
> >> From: "David Solin" <[hidden email]>
> >> To: [hidden email]
> >> Sent: Wednesday, March 25, 2015 11:52:14 AM
> >> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >> (Suspect the fix for issue #6 was the
> >> culprit)
> >>
> >> Hi Martin,
> >>
> >> Is it possible this is just a simple namespace prefix problem, i.e., there
> >> is
> >> no xmlns:oval-common="http://oval.mitre.org/XMLSchema/oval-common-5”?
> >>
> >> In most OVAL documents, the xml namespace prefix for common is simply
> >> “oval”
> >> (e.g., <oval:note>)
> >
> > Nope, that is not the problem. Yeah, the namespace prefix was just oval,
> > I changed the snippet to emphasize that it's
> > "http://oval.mitre.org/XMLSchema/oval-common-5". The namespace URIs match
> > exactly, the problem must be elsewhere.
> >
> > --
> > Martin Preisler
> > Security Technologies | Red Hat, Inc.
> >
> >> Regards,
> >> —David Solin
> >> [hidden email]
> >>
> >>
> >>
> >>> On Mar 24, 2015, at 1:06 PM, Martin Preisler <[hidden email]> wrote:
> >>>
> >>> ----- Original Message -----
> >>>> From: "David Solin" <[hidden email]>
> >>>> To: [hidden email]
> >>>> Sent: Thursday, March 19, 2015 9:40:44 PM
> >>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
> >>>> (Suspect the fix for issue #6 was the
> >>>> culprit)
> >>>>
> >>>> Hi Martin, etc.,
> >>>>
> >>>> Here, I’ve gotten a substitution group working:
> >>>> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
> >>>>
> >>>> (The diff is taken based on a previous set of changes, where I had
> >>>> abandoned
> >>>> this approach in favor of just re-introducing the pre-5.11
> >>>> oval-def:NotesType).
> >>>>
> >>>> The only imperfect side-effect is that an <oval:notes> is permitted to
> >>>> have
> >>>> no <oval:note> children — otherwise the substitution group added for
> >>>> legacy
> >>>> content support would require at least one, which we don’t want.
> >>>>
> >>>> Let me know what you all think of this approach!
> >>>
> >>> Pretty clever solution, kudos!
> >>>
> >>> It doesn't work with libxml2 or I am doing something wrong. It does not
> >>> validate
> >>> <notes><oval-common:note>abc</oval-common:note></notes>. This is the
> >>> message:
> >>>
> >>> File './test_probes_systemdunitproperty.xml' line 24: Element
> >>> '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing
> >>> child
> >>> element(s). Expected is one of (
> >>> {http://oval.mitre.org/XMLSchema/oval-common-5}note,
> >>> {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
> >>> OpenSCAP Error: Invalid OVAL Definition (5.11) content in
> >>> ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]
> >>>
> >>> What's confusing to me is that the error message suggests that it should
> >>> accept oval-common:note. If I add oval-def:note the error disappears and
> >>> everything works as expected.
> >>>
> >>> Is the schema supposed to reject
> >>> <notes><oval-common:note>a</oval-common:note></notes>?
> >>>
> >>> If I change the first hunk in oval-definitions-schema.xsd to:
> >>> <xsd:complexType>
> >>> <xsd:complexContent>
> >>>  <xsd:extension base="oval:NotesType">
> >>>    <xsd:sequence>
> >>>      <xsd:element name="note" type="xsd:string" minOccurs="0"
> >>>      maxOccurs="unbounded"/>
> >>>    </xsd:sequence>
> >>>  </xsd:extension>
> >>> </xsd:complexContent>
> >>> </xsd:complexType>
> >>>
> >>> Then everything works as expected. Since OVAL 5.11 validates
> >>> oval-common:note inside notes
> >>> I think the changed schema should as well. Will wait for confirmation
> >>> before I commit
> >>> this to the OpenSCAP repo. Maybe I am misunderstanding something...
> >>>
> >>> --
> >>> Martin Preisler
> >>> Security Technologies | Red Hat, Inc.
> >>>
> >>>>
> >>>> Best,
> >>>> —David Solin
> >>>> [hidden email]
> >>>>
> >>>>
> >>>>
> >>>>> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "David Solin" <[hidden email]>
> >>>>>> To: [hidden email]
> >>>>>> Sent: Thursday, March 19, 2015 4:24:51 PM
> >>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>> issue
> >>>>>> (Suspect the fix for issue #6 was the
> >>>>>> culprit)
> >>>>>>
> >>>>>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "David Solin" <[hidden email]>
> >>>>>>>> To: [hidden email]
> >>>>>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
> >>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>> issue
> >>>>>>>> (Suspect the fix for issue #6 was the
> >>>>>>>> culprit)
> >>>>>>>>
> >>>>>>>> Hi Martin,
> >>>>>>>>
> >>>>>>>> I spent most of Monday trying to come up with a similar solution,
> >>>>>>>> but
> >>>>>>>> I
> >>>>>>>> ended
> >>>>>>>> up giving up on it.  The reason was that every solution violates
> >>>>>>>> strict
> >>>>>>>> schema validation rules: cos-element-consistent, and likely also
> >>>>>>>> cos-nonambig.  Your solution unfortunately violates both.
> >>>>>>>
> >>>>>>> Yes, solving these sort of issues in XSD is not trivial. I tried a
> >>>>>>> few
> >>>>>>> obvious
> >>>>>>> fixes like using element/@ref but those were causing the errors you
> >>>>>>> mention.
> >>>>>>> The <xsd:extension/> solution works fine with libxml2.
> >>>>>>>
> >>>>>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you
> >>>>>>> paste
> >>>>>>> the errors?
> >>>>>>>
> >>>>>>
> >>>>>> I learned of the errors from the JAXB compiler.  There are errors like
> >>>>>> the
> >>>>>> following for every object type (i.e., TONS of errors):
> >>>>>>
> >>>>>> [ERROR] cos-element-consistent: Error for type
> >>>>>> '#AnonType_interface_object'.
> >>>>>> Multiple elements with name 'notes', with different types, appear in
> >>>>>> the
> >>>>>> model group.
> >>>>>> line 3299 of
> >>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >>>>>>
> >>>>>> [ERROR] cos-nonambig:
> >>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
> >>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or
> >>>>>> elements
> >>>>>> from
> >>>>>> their substitution group) violate "Unique Particle Attribution".
> >>>>>> During
> >>>>>> validation against this schema, ambiguity would be created for those
> >>>>>> two
> >>>>>> particles.
> >>>>>> line 3299 of
> >>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
> >>>>>
> >>>>> Hmm, libxml2 is fine with it but I agree that this is a blocker.
> >>>>> We need a better fix for this.
> >>>>>
> >>>>>> Reading online, XMLSpy does validate, if you have that available.
> >>>>>>
> >>>>>>
> >>>>>>>> I fear the only workable solution that gives you a valid schema is
> >>>>>>>> to
> >>>>>>>> just
> >>>>>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
> >>>>>>>> oval-variables-schema.
> >>>>>>>
> >>>>>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
> >>>>>>> <notes><oval:note>abc</oval:note></notes>.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I
> >>>>>> figure,
> >>>>>> no
> >>>>>> harm...
> >>>>>
> >>>>> Actually...
> >>>>>
> >>>>> https://github.com/OpenSCAP/scap-security-guide/pull/478
> >>>>>
> >>>>> The good news is that the notes elements are commented until this is
> >>>>> fixed.
> >>>>>
> >>>>>>> If I understand it correctly, your solution marks this as invalid
> >>>>>>> content.
> >>>>>>>
> >>>>>>> The final solution should keep compat with both 5.10.1 and 5.11.
> >>>>>>>
> >>>>>>> One other solution that I almost ended up using was to just use
> >>>>>>> <xsd:any/>
> >>>>>>> with appropriate namespace and then validate that it's either
> >>>>>>> oval:note
> >>>>>>> or
> >>>>>>> oval-def:note in schematron.
> >>>>>>
> >>>>>>
> >>>>>> I bet that might work.  But it’s horrible, isn’t it?!
> >>>>>
> >>>>> Yes! But at the same time it allows people to write OVAL 5.11 right now
> >>>>> and
> >>>>> be
> >>>>> safe that it will validate against 5.11.1. The ends may justify the
> >>>>> means
> >>>>> here :-)
> >>>>>
> >>>>> I think that the <xsd:any/> + schematron solution is our best bet right
> >>>>> now.
> >>>>> Any suggestions?
> >>>>>
> >>>>> --
> >>>>> Martin Preisler
> >>>>> Security Technologies | Red Hat, Inc.
> >>>>>
> >>>>>>> --
> >>>>>>> Martin Preisler
> >>>>>>> Security Technologies | Red Hat, Inc.
> >>>>>>>
> >>>>>>>> Best regards,
> >>>>>>>> —David Solin
> >>>>>>>> [hidden email]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> ----- Original Message -----
> >>>>>>>>>> From: "Dan Haynes" <[hidden email]>
> >>>>>>>>>> To: [hidden email]
> >>>>>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
> >>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>>>> issue
> >>>>>>>>>> (Suspect the fix for issue #6 was the
> >>>>>>>>>> culprit)
> >>>>>>>>>>
> >>>>>>>>>> I forgot to include it, but, I added a tracker for this issue.
> >>>>>>>>>>
> >>>>>>>>>> https://github.com/OVALProject/Language/issues/237
> >>>>>>>>>
> >>>>>>>>> I attached a pragmatic fix of the issue that keeps compatibility
> >>>>>>>>> with
> >>>>>>>>> both OVAL 5.11 and 5.10.1.
> >>>>>>>>>
> >>>>>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
> >>>>>>>>> figure out how to get rid of that. One solution is <xsd:assert/>
> >>>>>>>>> but
> >>>>>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
> >>>>>>>>>
> >>>>>>>>> See:
> >>>>>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Martin Preisler
> >>>>>>>>> Security Technologies | Red Hat, Inc.
> >>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
> >>>>>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
> >>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
> >>>>>>>>>>> Discussion
> >>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>>>>> issue
> >>>>>>>>>>> (Suspect the fix for issue #6 was the culprit)
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, this change was made to support notes in variables.  I agree
> >>>>>>>>>>> it
> >>>>>>>>>>> is
> >>>>>>>>>>> a
> >>>>>>>>>>> serious
> >>>>>>>>>>> backwards compatibility issue and it can be addressed with the
> >>>>>>>>>>> 5.11.1
> >>>>>>>>>>> release.
> >>>>>>>>>>> Thanks for catching this and reporting it!
> >>>>>>>>>>>
> >>>>>>>>>>> -Danny
> >>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: David Solin [mailto:[hidden email]]
> >>>>>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
> >>>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
> >>>>>>>>>>>> Discussion
> >>>>>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
> >>>>>>>>>>>> issue
> >>>>>>>>>>>> (Suspect
> >>>>>>>>>>>> the fix for issue #6 was the culprit)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Everyone,
> >>>>>>>>>>>>
> >>>>>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
> >>>>>>>>>>>> common,
> >>>>>>>>>>>> probably so that it could be used with variables.  This type
> >>>>>>>>>>>> defines
> >>>>>>>>>>>> the
> >>>>>>>>>>>> note
> >>>>>>>>>>>> element.  But, the notes element is still defined in
> >>>>>>>>>>>> oval-def:DefinitionType.
> >>>>>>>>>>>>
> >>>>>>>>>>>> This introduces a namespace mismatch between <notes> and its
> >>>>>>>>>>>> child
> >>>>>>>>>>>> <note> —
> >>>>>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
> >>>>>>>>>>>> with
> >>>>>>>>>>>> a
> >>>>>>>>>>>> 5.11
> >>>>>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
> >>>>>>>>>>>> backwards-
> >>>>>>>>>>>> compatibility issue.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best regards,
> >>>>>>>>>>>> —David Solin
> >>>>>>>>>>>> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the culprit)

joval
Oh, ok, I understand.  In that case, yes, minOccurs=“0” would indeed be necessary!  Let’s make that change.

David Solin
[hidden email]


> On Mar 25, 2015, at 8:04 AM, Martin Preisler <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Wednesday, March 25, 2015 1:52:12 PM
>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue (Suspect the fix for issue #6 was the
>> culprit)
>>
>> So, what is the namespace of the outer <notes> tag?  If it’s oval-def, then
>> this makes perfect sense.
>
> oval-def
>
>> I created the oval-def:notes strictly for compatibility with pre-5.11
>> oval-def:notes.  That is, it must have at least one oval-def:note.  As a
>> side-effect of being a substitution element, it can optionally have
>> oval:note children as well — but it still must have at least one
>> oval-def:note.  I think that’s why you’re getting that error.
>
> OVAL 5.11 allows <oval-def:notes><oval-common:note>a</oval-common:note></oval-def:notes>.
> That's why I am testing it. Since it uses <xsd:element name=.. type=../>
> the namespace will be the targetNamespace of the XSD, which is oval-def.
> It doesn't matter that the type is oval-common:notesType, the namespace
> of the notes still needs to be $targetNamespace.
>
> OVAL 5.11 allows <oval-def:notes><oval-common:note>a</oval-common:note></oval-def:notes>
> in the oval-def XSD and <oval-common:notes><oval-common:note>b</oval-common:note></oval-common:notes>
> in the oval-common XSD.
>
> --
> Martin Preisler
> Security Technologies | Red Hat, Inc.
>
>> I also deprecated oval-def:notes, so you should (if writing 5.11 content) use
>> the new oval:notes, which may have 0 or more oval:notes, and no other
>> children.
>>
>> Best regards,
>> —David Solin
>> [hidden email]
>>
>>
>>> On Mar 25, 2015, at 5:57 AM, Martin Preisler <[hidden email]> wrote:
>>>
>>> ----- Original Message -----
>>>> From: "David Solin" <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Wednesday, March 25, 2015 11:52:14 AM
>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>> (Suspect the fix for issue #6 was the
>>>> culprit)
>>>>
>>>> Hi Martin,
>>>>
>>>> Is it possible this is just a simple namespace prefix problem, i.e., there
>>>> is
>>>> no xmlns:oval-common="http://oval.mitre.org/XMLSchema/oval-common-5”?
>>>>
>>>> In most OVAL documents, the xml namespace prefix for common is simply
>>>> “oval”
>>>> (e.g., <oval:note>)
>>>
>>> Nope, that is not the problem. Yeah, the namespace prefix was just oval,
>>> I changed the snippet to emphasize that it's
>>> "http://oval.mitre.org/XMLSchema/oval-common-5". The namespace URIs match
>>> exactly, the problem must be elsewhere.
>>>
>>> --
>>> Martin Preisler
>>> Security Technologies | Red Hat, Inc.
>>>
>>>> Regards,
>>>> —David Solin
>>>> [hidden email]
>>>>
>>>>
>>>>
>>>>> On Mar 24, 2015, at 1:06 PM, Martin Preisler <[hidden email]> wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "David Solin" <[hidden email]>
>>>>>> To: [hidden email]
>>>>>> Sent: Thursday, March 19, 2015 9:40:44 PM
>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility issue
>>>>>> (Suspect the fix for issue #6 was the
>>>>>> culprit)
>>>>>>
>>>>>> Hi Martin, etc.,
>>>>>>
>>>>>> Here, I’ve gotten a substitution group working:
>>>>>> https://github.com/joval/jOVAL/commit/d8c2d6a177776e2a928d3e1aeb60f570c58716bf
>>>>>>
>>>>>> (The diff is taken based on a previous set of changes, where I had
>>>>>> abandoned
>>>>>> this approach in favor of just re-introducing the pre-5.11
>>>>>> oval-def:NotesType).
>>>>>>
>>>>>> The only imperfect side-effect is that an <oval:notes> is permitted to
>>>>>> have
>>>>>> no <oval:note> children — otherwise the substitution group added for
>>>>>> legacy
>>>>>> content support would require at least one, which we don’t want.
>>>>>>
>>>>>> Let me know what you all think of this approach!
>>>>>
>>>>> Pretty clever solution, kudos!
>>>>>
>>>>> It doesn't work with libxml2 or I am doing something wrong. It does not
>>>>> validate
>>>>> <notes><oval-common:note>abc</oval-common:note></notes>. This is the
>>>>> message:
>>>>>
>>>>> File './test_probes_systemdunitproperty.xml' line 24: Element
>>>>> '{http://oval.mitre.org/XMLSchema/oval-definitions-5}notes': Missing
>>>>> child
>>>>> element(s). Expected is one of (
>>>>> {http://oval.mitre.org/XMLSchema/oval-common-5}note,
>>>>> {http://oval.mitre.org/XMLSchema/oval-definitions-5}note ).
>>>>> OpenSCAP Error: Invalid OVAL Definition (5.11) content in
>>>>> ./test_probes_systemdunitproperty.xml. [oscap_source.c:205]
>>>>>
>>>>> What's confusing to me is that the error message suggests that it should
>>>>> accept oval-common:note. If I add oval-def:note the error disappears and
>>>>> everything works as expected.
>>>>>
>>>>> Is the schema supposed to reject
>>>>> <notes><oval-common:note>a</oval-common:note></notes>?
>>>>>
>>>>> If I change the first hunk in oval-definitions-schema.xsd to:
>>>>> <xsd:complexType>
>>>>> <xsd:complexContent>
>>>>> <xsd:extension base="oval:NotesType">
>>>>>   <xsd:sequence>
>>>>>     <xsd:element name="note" type="xsd:string" minOccurs="0"
>>>>>     maxOccurs="unbounded"/>
>>>>>   </xsd:sequence>
>>>>> </xsd:extension>
>>>>> </xsd:complexContent>
>>>>> </xsd:complexType>
>>>>>
>>>>> Then everything works as expected. Since OVAL 5.11 validates
>>>>> oval-common:note inside notes
>>>>> I think the changed schema should as well. Will wait for confirmation
>>>>> before I commit
>>>>> this to the OpenSCAP repo. Maybe I am misunderstanding something...
>>>>>
>>>>> --
>>>>> Martin Preisler
>>>>> Security Technologies | Red Hat, Inc.
>>>>>
>>>>>>
>>>>>> Best,
>>>>>> —David Solin
>>>>>> [hidden email]
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Mar 19, 2015, at 10:39 AM, Martin Preisler <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "David Solin" <[hidden email]>
>>>>>>>> To: [hidden email]
>>>>>>>> Sent: Thursday, March 19, 2015 4:24:51 PM
>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>> issue
>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>> culprit)
>>>>>>>>
>>>>>>>>> On Mar 19, 2015, at 10:17 AM, Martin Preisler <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "David Solin" <[hidden email]>
>>>>>>>>>> To: [hidden email]
>>>>>>>>>> Sent: Thursday, March 19, 2015 3:37:19 PM
>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>> issue
>>>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>>>> culprit)
>>>>>>>>>>
>>>>>>>>>> Hi Martin,
>>>>>>>>>>
>>>>>>>>>> I spent most of Monday trying to come up with a similar solution,
>>>>>>>>>> but
>>>>>>>>>> I
>>>>>>>>>> ended
>>>>>>>>>> up giving up on it.  The reason was that every solution violates
>>>>>>>>>> strict
>>>>>>>>>> schema validation rules: cos-element-consistent, and likely also
>>>>>>>>>> cos-nonambig.  Your solution unfortunately violates both.
>>>>>>>>>
>>>>>>>>> Yes, solving these sort of issues in XSD is not trivial. I tried a
>>>>>>>>> few
>>>>>>>>> obvious
>>>>>>>>> fixes like using element/@ref but those were causing the errors you
>>>>>>>>> mention.
>>>>>>>>> The <xsd:extension/> solution works fine with libxml2.
>>>>>>>>>
>>>>>>>>> How do you test XSD validity? Are you using Altova XMLSpy? Can you
>>>>>>>>> paste
>>>>>>>>> the errors?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I learned of the errors from the JAXB compiler.  There are errors like
>>>>>>>> the
>>>>>>>> following for every object type (i.e., TONS of errors):
>>>>>>>>
>>>>>>>> [ERROR] cos-element-consistent: Error for type
>>>>>>>> '#AnonType_interface_object'.
>>>>>>>> Multiple elements with name 'notes', with different types, appear in
>>>>>>>> the
>>>>>>>> model group.
>>>>>>>> line 3299 of
>>>>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>>>>>>
>>>>>>>> [ERROR] cos-nonambig:
>>>>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes and
>>>>>>>> "http://oval.mitre.org/XMLSchema/oval-definitions-5":notes (or
>>>>>>>> elements
>>>>>>>> from
>>>>>>>> their substitution group) violate "Unique Particle Attribution".
>>>>>>>> During
>>>>>>>> validation against this schema, ambiguity would be created for those
>>>>>>>> two
>>>>>>>> particles.
>>>>>>>> line 3299 of
>>>>>>>> file:/Users/solind/Development/jOVAL/scap/schemas/oval-5.11/windows-definitions-schema.xsd
>>>>>>>
>>>>>>> Hmm, libxml2 is fine with it but I agree that this is a blocker.
>>>>>>> We need a better fix for this.
>>>>>>>
>>>>>>>> Reading online, XMLSpy does validate, if you have that available.
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I fear the only workable solution that gives you a valid schema is
>>>>>>>>>> to
>>>>>>>>>> just
>>>>>>>>>> reintroduce oval-def:NotesType, and use it everywhere except in the
>>>>>>>>>> oval-variables-schema.
>>>>>>>>>
>>>>>>>>> Doesn't this break compatibility with 5.11? In 5.11 content can have
>>>>>>>>> <notes><oval:note>abc</oval:note></notes>.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes!  But there is no 5.11 content, because 5.11 is broken!  So, I
>>>>>>>> figure,
>>>>>>>> no
>>>>>>>> harm...
>>>>>>>
>>>>>>> Actually...
>>>>>>>
>>>>>>> https://github.com/OpenSCAP/scap-security-guide/pull/478
>>>>>>>
>>>>>>> The good news is that the notes elements are commented until this is
>>>>>>> fixed.
>>>>>>>
>>>>>>>>> If I understand it correctly, your solution marks this as invalid
>>>>>>>>> content.
>>>>>>>>>
>>>>>>>>> The final solution should keep compat with both 5.10.1 and 5.11.
>>>>>>>>>
>>>>>>>>> One other solution that I almost ended up using was to just use
>>>>>>>>> <xsd:any/>
>>>>>>>>> with appropriate namespace and then validate that it's either
>>>>>>>>> oval:note
>>>>>>>>> or
>>>>>>>>> oval-def:note in schematron.
>>>>>>>>
>>>>>>>>
>>>>>>>> I bet that might work.  But it’s horrible, isn’t it?!
>>>>>>>
>>>>>>> Yes! But at the same time it allows people to write OVAL 5.11 right now
>>>>>>> and
>>>>>>> be
>>>>>>> safe that it will validate against 5.11.1. The ends may justify the
>>>>>>> means
>>>>>>> here :-)
>>>>>>>
>>>>>>> I think that the <xsd:any/> + schematron solution is our best bet right
>>>>>>> now.
>>>>>>> Any suggestions?
>>>>>>>
>>>>>>> --
>>>>>>> Martin Preisler
>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>
>>>>>>>>> --
>>>>>>>>> Martin Preisler
>>>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> —David Solin
>>>>>>>>>> [hidden email]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Mar 19, 2015, at 8:41 AM, Martin Preisler <[hidden email]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>> From: "Dan Haynes" <[hidden email]>
>>>>>>>>>>>> To: [hidden email]
>>>>>>>>>>>> Sent: Tuesday, March 3, 2015 3:51:56 AM
>>>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>>>> issue
>>>>>>>>>>>> (Suspect the fix for issue #6 was the
>>>>>>>>>>>> culprit)
>>>>>>>>>>>>
>>>>>>>>>>>> I forgot to include it, but, I added a tracker for this issue.
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/OVALProject/Language/issues/237
>>>>>>>>>>>
>>>>>>>>>>> I attached a pragmatic fix of the issue that keeps compatibility
>>>>>>>>>>> with
>>>>>>>>>>> both OVAL 5.11 and 5.10.1.
>>>>>>>>>>>
>>>>>>>>>>> Unfortunately it now allows empty <notes/> elements and I couldn't
>>>>>>>>>>> figure out how to get rid of that. One solution is <xsd:assert/>
>>>>>>>>>>> but
>>>>>>>>>>> that's XSD 1.1 only. Or it could be checked in schematron.
>>>>>>>>>>>
>>>>>>>>>>> See:
>>>>>>>>>>> https://github.com/OpenSCAP/openscap/commit/8e1f69c0bacb974f604f56a12da5386e1bfa8c8f
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Martin Preisler
>>>>>>>>>>> Security Technologies | Red Hat, Inc.
>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Haynes, Dan [mailto:[hidden email]]
>>>>>>>>>>>>> Sent: Monday, March 02, 2015 9:06 PM
>>>>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
>>>>>>>>>>>>> Discussion
>>>>>>>>>>>>> Subject: Re: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>>>>> issue
>>>>>>>>>>>>> (Suspect the fix for issue #6 was the culprit)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, this change was made to support notes in variables.  I agree
>>>>>>>>>>>>> it
>>>>>>>>>>>>> is
>>>>>>>>>>>>> a
>>>>>>>>>>>>> serious
>>>>>>>>>>>>> backwards compatibility issue and it can be addressed with the
>>>>>>>>>>>>> 5.11.1
>>>>>>>>>>>>> release.
>>>>>>>>>>>>> Thanks for catching this and reporting it!
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Danny
>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: David Solin [mailto:[hidden email]]
>>>>>>>>>>>>>> Sent: Monday, March 02, 2015 12:06 PM
>>>>>>>>>>>>>> To: oval-developer-list OVAL Developer List/Closed Public
>>>>>>>>>>>>>> Discussion
>>>>>>>>>>>>>> Subject: [OVAL-DEVELOPER-LIST] 5.11 backwards-incompatibility
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>> (Suspect
>>>>>>>>>>>>>> the fix for issue #6 was the culprit)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In OVAL 5.11, oval-def:NotesType definition was moved to OVAL
>>>>>>>>>>>>>> common,
>>>>>>>>>>>>>> probably so that it could be used with variables.  This type
>>>>>>>>>>>>>> defines
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> note
>>>>>>>>>>>>>> element.  But, the notes element is still defined in
>>>>>>>>>>>>>> oval-def:DefinitionType.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This introduces a namespace mismatch between <notes> and its
>>>>>>>>>>>>>> child
>>>>>>>>>>>>>> <note> —
>>>>>>>>>>>>>> which causes validation errors when parsing a 5.10 OVAL document
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> 5.11
>>>>>>>>>>>>>> parser.  We need to fix this in OVAL 5.11.1, as it’s a serious
>>>>>>>>>>>>>> backwards-
>>>>>>>>>>>>>> compatibility issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>> —David Solin
>>>>>>>>>>>>>> [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download