Error in the schema documentation for GlobToRegexFunctionType

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Error in the schema documentation for GlobToRegexFunctionType

joval
Here is the documentation, for reference:
http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType

The examples in the documentation for the glob_noescape attribute seem wrong to me.

For glob_noescape=true, the glob pattern '\*' should become the regex '\*', not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob pattern '\?' should become the regex '\?', not the regex ‘\.'

Am I right?

Thanks,
--David Solin
[hidden email]
To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

joval
I’m sorry, there was a little cognitive dissonance on my part with the noescape attribute.  Noescape=true means that the \ character is treated as a literal, not as an escape.

Hence, for noescape=false (i.e., \ is an escape), glob \* implies pattern ^\*$ (i.e., the * is escaped).  Similarly, for noescape=false, glob \? implies pattern ^\?$ (i.e., the ? is escaped).

But, the schema documentation says (for noescape=false):
\* -> ^\\.*$
\? -> ^\\.$

(This is precisely the noescape=true case!)

The schema documentation is likewise incorrect for the noescape=true case:
\* -> ^\.*$  (it should be ^\\.*$)
\? -> ^\.$ (it should be ^\\.$)

So, it’s not just a simple case of flipping the meaning of true and false in the examples.  We should correct this with 5.11.1.

Regards,
—David Solin


> On Mar 1, 2015, at 1:35 PM, David Solin <[hidden email]> wrote:
>
> Here is the documentation, for reference:
> http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
>
> The examples in the documentation for the glob_noescape attribute seem wrong to me.
>
> For glob_noescape=true, the glob pattern '\*' should become the regex '\*', not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob pattern '\?' should become the regex '\?', not the regex ‘\.'
>
> Am I right?
>
> Thanks,
> --David Solin
> [hidden email]

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

Jan Lieskovsky
In reply to this post by joval
Hello David,

----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Sunday, March 1, 2015 8:35:23 PM
> Subject: [OVAL-DEVELOPER-LIST] Error in the schema documentation for GlobToRegexFunctionType
>
> Here is the documentation, for reference:
> http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
>
> The examples in the documentation for the glob_noescape attribute seem wrong
> to me.

Thanks for the heads-up. Will have a further look into this & comment after review.

Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

P.S.: Sorry for late reply, other duties prevented me from sooner reaction :(.

>
> For glob_noescape=true, the glob pattern '\*' should become the regex '\*',
> not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob
> pattern '\?' should become the regex '\?', not the regex ‘\.'
>
> Am I right?
>
> Thanks,
> --David Solin
> [hidden email]
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to
> [hidden email].
>

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

joval
Thanks, Jan.  Be sure to find my follow-up post, which contains a corrected description of the documentation errors.

David Solin
[hidden email]



> On Mar 5, 2015, at 11:49 AM, Jan Lieskovsky <[hidden email]> wrote:
>
> Hello David,
>
> ----- Original Message -----
>> From: "David Solin" <[hidden email]>
>> To: [hidden email]
>> Sent: Sunday, March 1, 2015 8:35:23 PM
>> Subject: [OVAL-DEVELOPER-LIST] Error in the schema documentation for GlobToRegexFunctionType
>>
>> Here is the documentation, for reference:
>> http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
>>
>> The examples in the documentation for the glob_noescape attribute seem wrong
>> to me.
>
> Thanks for the heads-up. Will have a further look into this & comment after review.
>
> Regards, Jan.
> --
> Jan iankko Lieskovsky / Red Hat Security Technologies Team
>
> P.S.: Sorry for late reply, other duties prevented me from sooner reaction :(.
>
>>
>> For glob_noescape=true, the glob pattern '\*' should become the regex '\*',
>> not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob
>> pattern '\?' should become the regex '\?', not the regex ‘\.'
>>
>> Am I right?
>>
>> Thanks,
>> --David Solin
>> [hidden email]
>> To unsubscribe, send an email message to [hidden email] with
>> SIGNOFF OVAL-DEVELOPER-LIST
>> in the BODY of the message.  If you have difficulties, write to
>> [hidden email].
>>
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

jOVAL.org: OVAL implemented in Java.
Scan any machine from any machine. For free!
Learn More | Features | Download

Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

Jan Lieskovsky
In reply to this post by joval
Hello David, OVAL developers,

  looked further into this (instead of commenting just about
one concrete glob example wanted to come with C & Perl implementation
demonstrating the behaviour for arbitrary glob expression upon request - see
the attachment). See the replies inline.

----- Original Message -----

> From: "David Solin" <[hidden email]>
> To: [hidden email]
> Sent: Monday, March 2, 2015 3:53:25 AM
> Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for GlobToRegexFunctionType
>
> I’m sorry, there was a little cognitive dissonance on my part with the
> noescape attribute.  Noescape=true means that the \ character is treated as
> a literal, not as an escape.
>
> Hence, for noescape=false (i.e., \ is an escape), glob \* implies pattern
> ^\*$ (i.e., the * is escaped).  Similarly, for noescape=false, glob \?
> implies pattern ^\?$ (i.e., the ? is escaped).

This is partially true. Completely true in the sense that the corresponding
Perl regular expressions provided as examples in OVAL 5.11 version:
  http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType

are wrong. But IMHO it's wrong in the sense that glob '\*' implies pattern
^\*$ with noescape=false, and glob '\?' implies pattern ^\?$ with noescape=false.
IMHO wrong in the sense corresponding glob '*' and '?' characters aren't expanded
to their Perl regex counterparts. See the perl_sample.pl output (the table) below
for further clarification.

>
> But, the schema documentation says (for noescape=false):
> \* -> ^\\.*$
> \? -> ^\\.$
>
> (This is precisely the noescape=true case!)

Agree with this. These should be flipped.

>
> The schema documentation is likewise incorrect for the noescape=true case:
> \* -> ^\.*$  (it should be ^\\.*$)
> \? -> ^\.$ (it should be ^\\.$)
>
> So, it’s not just a simple case of flipping the meaning of true and false in
> the examples.  We should correct this with 5.11.1.

As mentioned earlier created C glob(3) source code example & corresponding
Perl script to demonstrate the expected glob_to_regex() behaviour.

Also added '\[hello\]' as another glob expression to document the behavior

Running 'perl_sample.pl' for the following three '\*', '\?', '\[hello\]'
glob expressions returns the following table:

# Results for C '\*' glob:
----------------------------------------
  noescape | Perl regular expression
----------------------------------------
     FALSE |      ^\.*$
      TRUE |     ^\\.*$
----------------------------------------


# Results for C '\?' glob:
----------------------------------------
  noescape | Perl regular expression
----------------------------------------
     FALSE |       ^\.$
      TRUE |      ^\\.$
----------------------------------------


# Results for C '\[hello\]' glob:
----------------------------------------
  noescape | Perl regular expression
----------------------------------------
     FALSE | ^\[hello\]$
      TRUE | ^\\[hello\\]$
----------------------------------------

and also as can be verified by running './glob_sample' and './perl_sample.pl'
from the attachment, the C and Perl implementations return same results for
given files (meaning the created Perl regular expressions are equivalent
to C globs).

>
> Regards,
> —David Solin
>
>
> > On Mar 1, 2015, at 1:35 PM, David Solin <[hidden email]> wrote:
> >
> > Here is the documentation, for reference:
> > http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
> >
> > The examples in the documentation for the glob_noescape attribute seem
> > wrong to me.
> >
> > For glob_noescape=true, the glob pattern '\*' should become the regex '\*',
> > not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob
> > pattern '\?' should become the regex '\?', not the regex ‘\.'
> >
> > Am I right?
> >
> > Thanks,
> > --David Solin
> > [hidden email]
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to
> [hidden email].
>

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

Jan Lieskovsky

Forgot to attach the intended attachment in the
previous reply, so it's attached below(sorry
for doubled post :().

After review it could be hopefully included into
the OVAL Sandbox Git repository as a convenience method
how to demonstrate intended glob_to_regex() behaviour
also on glob expressions not covered within OVAL 5.11+
language documentation.

Thank you && Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

----- Original Message -----

> From: "Jan Lieskovsky" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, March 12, 2015 4:16:24 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for GlobToRegexFunctionType
>
> Hello David, OVAL developers,
>
>   looked further into this (instead of commenting just about
> one concrete glob example wanted to come with C & Perl implementation
> demonstrating the behaviour for arbitrary glob expression upon request - see
> the attachment). See the replies inline.
>
> ----- Original Message -----
> > From: "David Solin" <[hidden email]>
> > To: [hidden email]
> > Sent: Monday, March 2, 2015 3:53:25 AM
> > Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for
> > GlobToRegexFunctionType
> >
> > I’m sorry, there was a little cognitive dissonance on my part with the
> > noescape attribute.  Noescape=true means that the \ character is treated as
> > a literal, not as an escape.
> >
> > Hence, for noescape=false (i.e., \ is an escape), glob \* implies pattern
> > ^\*$ (i.e., the * is escaped).  Similarly, for noescape=false, glob \?
> > implies pattern ^\?$ (i.e., the ? is escaped).
>
> This is partially true. Completely true in the sense that the corresponding
> Perl regular expressions provided as examples in OVAL 5.11 version:
>   http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
>
> are wrong. But IMHO it's wrong in the sense that glob '\*' implies pattern
> ^\*$ with noescape=false, and glob '\?' implies pattern ^\?$ with
> noescape=false.
> IMHO wrong in the sense corresponding glob '*' and '?' characters aren't
> expanded
> to their Perl regex counterparts. See the perl_sample.pl output (the table)
> below
> for further clarification.
>
> >
> > But, the schema documentation says (for noescape=false):
> > \* -> ^\\.*$
> > \? -> ^\\.$
> >
> > (This is precisely the noescape=true case!)
>
> Agree with this. These should be flipped.
>
> >
> > The schema documentation is likewise incorrect for the noescape=true case:
> > \* -> ^\.*$  (it should be ^\\.*$)
> > \? -> ^\.$ (it should be ^\\.$)
> >
> > So, it’s not just a simple case of flipping the meaning of true and false
> > in
> > the examples.  We should correct this with 5.11.1.
>
> As mentioned earlier created C glob(3) source code example & corresponding
> Perl script to demonstrate the expected glob_to_regex() behaviour.
>
> Also added '\[hello\]' as another glob expression to document the behavior
>
> Running 'perl_sample.pl' for the following three '\*', '\?', '\[hello\]'
> glob expressions returns the following table:
>
> # Results for C '\*' glob:
> ----------------------------------------
>   noescape | Perl regular expression
> ----------------------------------------
>      FALSE |      ^\.*$
>       TRUE |     ^\\.*$
> ----------------------------------------
>
>
> # Results for C '\?' glob:
> ----------------------------------------
>   noescape | Perl regular expression
> ----------------------------------------
>      FALSE |       ^\.$
>       TRUE |      ^\\.$
> ----------------------------------------
>
>
> # Results for C '\[hello\]' glob:
> ----------------------------------------
>   noescape | Perl regular expression
> ----------------------------------------
>      FALSE | ^\[hello\]$
>       TRUE | ^\\[hello\\]$
> ----------------------------------------
>
> and also as can be verified by running './glob_sample' and './perl_sample.pl'
> from the attachment, the C and Perl implementations return same results for
> given files (meaning the created Perl regular expressions are equivalent
> to C globs).
>
> >
> > Regards,
> > —David Solin
> >
> >
> > > On Mar 1, 2015, at 1:35 PM, David Solin <[hidden email]> wrote:
> > >
> > > Here is the documentation, for reference:
> > > http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
> > >
> > > The examples in the documentation for the glob_noescape attribute seem
> > > wrong to me.
> > >
> > > For glob_noescape=true, the glob pattern '\*' should become the regex
> > > '\*',
> > > not the regex '\.*’  Similarly, for the glob_noescape=true case, the glob
> > > pattern '\?' should become the regex '\?', not the regex ‘\.'
> > >
> > > Am I right?
> > >
> > > Thanks,
> > > --David Solin
> > > [hidden email]
> >
> > To unsubscribe, send an email message to [hidden email] with
> > SIGNOFF OVAL-DEVELOPER-LIST
> > in the BODY of the message.  If you have difficulties, write to
> > [hidden email].
> >
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to
> [hidden email].
>
To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].

glob_to_regex.tar.gz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Error in the schema documentation for GlobToRegexFunctionType

Jan Lieskovsky
Yet, noticed the Mitre mail filtering removed the Perl script part
from original attachment.

To review the scripts if interested, the tarball (original attachment)
is available for download here:
  [1] https://fedorapeople.org/~jlieskov/glob_to_regex.tar.gz


Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team


----- Original Message -----

> From: "Jan Lieskovsky" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, March 12, 2015 4:24:25 PM
> Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for GlobToRegexFunctionType
>
>
> Forgot to attach the intended attachment in the
> previous reply, so it's attached below(sorry
> for doubled post :().
>
> After review it could be hopefully included into
> the OVAL Sandbox Git repository as a convenience method
> how to demonstrate intended glob_to_regex() behaviour
> also on glob expressions not covered within OVAL 5.11+
> language documentation.
>
> Thank you && Regards, Jan.
> --
> Jan iankko Lieskovsky / Red Hat Security Technologies Team
>
> ----- Original Message -----
> > From: "Jan Lieskovsky" <[hidden email]>
> > To: [hidden email]
> > Sent: Thursday, March 12, 2015 4:16:24 PM
> > Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for
> > GlobToRegexFunctionType
> >
> > Hello David, OVAL developers,
> >
> >   looked further into this (instead of commenting just about
> > one concrete glob example wanted to come with C & Perl implementation
> > demonstrating the behaviour for arbitrary glob expression upon request -
> > see
> > the attachment). See the replies inline.
> >
> > ----- Original Message -----
> > > From: "David Solin" <[hidden email]>
> > > To: [hidden email]
> > > Sent: Monday, March 2, 2015 3:53:25 AM
> > > Subject: Re: [OVAL-DEVELOPER-LIST] Error in the schema documentation for
> > > GlobToRegexFunctionType
> > >
> > > I’m sorry, there was a little cognitive dissonance on my part with the
> > > noescape attribute.  Noescape=true means that the \ character is treated
> > > as
> > > a literal, not as an escape.
> > >
> > > Hence, for noescape=false (i.e., \ is an escape), glob \* implies pattern
> > > ^\*$ (i.e., the * is escaped).  Similarly, for noescape=false, glob \?
> > > implies pattern ^\?$ (i.e., the ? is escaped).
> >
> > This is partially true. Completely true in the sense that the corresponding
> > Perl regular expressions provided as examples in OVAL 5.11 version:
> >   http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
> >
> > are wrong. But IMHO it's wrong in the sense that glob '\*' implies pattern
> > ^\*$ with noescape=false, and glob '\?' implies pattern ^\?$ with
> > noescape=false.
> > IMHO wrong in the sense corresponding glob '*' and '?' characters aren't
> > expanded
> > to their Perl regex counterparts. See the perl_sample.pl output (the table)
> > below
> > for further clarification.
> >
> > >
> > > But, the schema documentation says (for noescape=false):
> > > \* -> ^\\.*$
> > > \? -> ^\\.$
> > >
> > > (This is precisely the noescape=true case!)
> >
> > Agree with this. These should be flipped.
> >
> > >
> > > The schema documentation is likewise incorrect for the noescape=true
> > > case:
> > > \* -> ^\.*$  (it should be ^\\.*$)
> > > \? -> ^\.$ (it should be ^\\.$)
> > >
> > > So, it’s not just a simple case of flipping the meaning of true and false
> > > in
> > > the examples.  We should correct this with 5.11.1.
> >
> > As mentioned earlier created C glob(3) source code example & corresponding
> > Perl script to demonstrate the expected glob_to_regex() behaviour.
> >
> > Also added '\[hello\]' as another glob expression to document the behavior
> >
> > Running 'perl_sample.pl' for the following three '\*', '\?', '\[hello\]'
> > glob expressions returns the following table:
> >
> > # Results for C '\*' glob:
> > ----------------------------------------
> >   noescape | Perl regular expression
> > ----------------------------------------
> >      FALSE |      ^\.*$
> >       TRUE |     ^\\.*$
> > ----------------------------------------
> >
> >
> > # Results for C '\?' glob:
> > ----------------------------------------
> >   noescape | Perl regular expression
> > ----------------------------------------
> >      FALSE |       ^\.$
> >       TRUE |      ^\\.$
> > ----------------------------------------
> >
> >
> > # Results for C '\[hello\]' glob:
> > ----------------------------------------
> >   noescape | Perl regular expression
> > ----------------------------------------
> >      FALSE | ^\[hello\]$
> >       TRUE | ^\\[hello\\]$
> > ----------------------------------------
> >
> > and also as can be verified by running './glob_sample' and
> > './perl_sample.pl'
> > from the attachment, the C and Perl implementations return same results for
> > given files (meaning the created Perl regular expressions are equivalent
> > to C globs).
> >
> > >
> > > Regards,
> > > —David Solin
> > >
> > >
> > > > On Mar 1, 2015, at 1:35 PM, David Solin <[hidden email]> wrote:
> > > >
> > > > Here is the documentation, for reference:
> > > > http://oval.mitre.org/language/version5.11/ovaldefinition/documentation/oval-definitions-schema.html#GlobToRegexFunctionType
> > > >
> > > > The examples in the documentation for the glob_noescape attribute seem
> > > > wrong to me.
> > > >
> > > > For glob_noescape=true, the glob pattern '\*' should become the regex
> > > > '\*',
> > > > not the regex '\.*’  Similarly, for the glob_noescape=true case, the
> > > > glob
> > > > pattern '\?' should become the regex '\?', not the regex ‘\.'
> > > >
> > > > Am I right?
> > > >
> > > > Thanks,
> > > > --David Solin
> > > > [hidden email]
> > >
> > > To unsubscribe, send an email message to [hidden email] with
> > > SIGNOFF OVAL-DEVELOPER-LIST
> > > in the BODY of the message.  If you have difficulties, write to
> > > [hidden email].
> > >
> >
> > To unsubscribe, send an email message to [hidden email] with
> > SIGNOFF OVAL-DEVELOPER-LIST
> > in the BODY of the message.  If you have difficulties, write to
> > [hidden email].
> >
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to
> [hidden email].
>

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].