Re: Linux schema issues

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Linux schema issues

Matthew N. Wojcik
To the list: I'd sent Javier mail asking for his thoughts on the
discussion regarding RPM version comparisons, and I think it's worth
following up to the list now.  For one thing, I tried to summarize or
clarify some of the details of that RedHat_RPMVersionCompare table Jay
mentioned.  So here's the thread...

On Fri, 27 Jun 2003, Javier Fernandez-Sanguino wrote:

> Matthew N. Wojcik wrote:
> >
> > This may all be clear to you already, but the discussion on the list
> > has been a little scattered, so maybe it will help if I recap what
> > we've been talking about for Red Hat first.
>
> > It's a table that would look something like:
> >
> > RedHat_RPMVersionCompare -- Hold the results of specific RPM
> >                          -- version comparisons
> > ------------------------
> > RPMName                  -- Package name
> > RPMTestedVersion         -- Version string to test against
> > RPMInstalledVersion      -- 'earlier', 'equal', or 'later'
> >                          -- or maybe -1, 0, or 1

[Note that I changed the column names a bit from what Jay had
proposed, though I'm still not sure I'm totally happy with them yet.
And Mark has pointed out that we need to be aware of package epochs as
well.]

> >
> > The idea being that an OVAL-compatible tool would populate this table
> > by using librpm API calls for specific package versions.  So if we
> > imagine a hypothetical vulnerability in package foo that was fixed in
> > version foo-0.17-7, we'd have a subquery like:
>
> In the Debian case the calls could be made either by calling dpkg
> directly (dpkg --compare-versions) or by creating a library that
> implemented the same comparison code in dpkg. As a matter of fact (have
> not checked fully) apt provides a C++ library (libapt-pkg) which
> provides a DoCmpVersion.

See below for my reaction to this, after Javi's later comments about a
library for dpkg...

>
> >
> > (SELECT 'foo earlier than 0.17-7 installed' FROM RedHat_RPMVersionCompare
> >   WHERE RPMName = 'foo' AND
> >         RPMTestedVersion = '0.17-7' AND
> >         RPMInstalledVersion = 'earlier')
> >
> > To populate that table, an OVAL tool would have to be configured with
> > the packages and versions to test, retrieve the installed versions of
> > those packages (if they are installed), then do the comparison and
> > store the results in the table.
>
> How would the OVAL tool know which packages and versions are the queries
> interested in? Should that be an INSERT statement? That's the only thing
> that is not clear to me.

In the current approach, yes, that would mean using INSERT statements.
The query writer would also write INSERT statements for a
RedHat_RPMVersionCompare_conf table, specifying the package name and
version (and epoch if any) to test the system against.  The OVAL tool
would have to somehow read those INSERT statements before it could do
the data collection.

But as I've said a couple of times recently, I'm not really happy with
the INSERT statement approach anymore.  It makes pretty good sense if
you're thinking about an OVAL-compatible tool which is actually using
a database, and doing all of its data collection ahead of any query
evaluation.  But for any other approach, there are clearly drawbacks.
One of my major hopes for the XML encoding for OVAL vulnerability
specifications is to deal with this issue more cleanly.  So we may not
need to support INSERT statements too much longer.  Unless perhaps
there are other OVAL-compatible tools out there or under development
that are using them that I'm not aware of. :)

> > I saw your post about dpkg --compare-versions from back in May.
> > Presumably, then we could do something very similar for Debian
> > packages if you think this is the right way to go.  Any thoughts?
>
> That's precisely the way to go. We can either make a direct call to the
> command or use the provided source code to make a library similar to
> librpm. Debian does not have such a library at the moment (there is a
> library provided by apt, but not by dpkg). I could contribute effort
> into writing that API if that if that was needed.

I think it can be left up to any OVAL-compatible tool for Debian to
decide whether to call dpkg --compare-versions or to work up a library
(or just internal code) like you describe.  If it were me, I'd
probably choose to call dpkg, so that I wouldn't have to update my
tool's code if the internal semantics of version comparison in the
official dpkg changed.  Of course, if a library such as you describe
were to become available as part of the Debian distro in the future,
then we could update the OVAL Debian schema documentation to suggest
using it.

Cheers,

--Woj


Reply | Threaded
Open this post in threaded view
|

Re: Linux schema issues

Javier Fernández-Sanguino-2
>>That's precisely the way to go. We can either make a direct call to the
>>command or use the provided source code to make a library similar to
>>librpm. Debian does not have such a library at the moment (there is a
>>library provided by apt, but not by dpkg). I could contribute effort
>>into writing that API if that if that was needed.
>
>
> I think it can be left up to any OVAL-compatible tool for Debian to
> decide whether to call dpkg --compare-versions or to work up a library
> (or just internal code) like you describe.  If it were me, I'd
> probably choose to call dpkg, so that I wouldn't have to update my
> tool's code if the internal semantics of version comparison in the
> official dpkg changed.  Of course, if a library such as you describe
> were to become available as part of the Debian distro in the future,
> then we could update the OVAL Debian schema documentation to suggest
> using it.
>

There is currently a library available and that is the one implemented
by APT, it wraps the dpkg call, but is written in C++. I wouldn't like a
tool to make a system call directly, though, due to the potential
security issues, so if the APT-API is not suitable then probably
writting a libdpkg would be the best go. There has been no need for such
a library (from what I know) but I could discuss this in the developers
mailing list, if such a library was developed it would be included in
the Debian distibution.

Regards

Javi


Reply | Threaded
Open this post in threaded view
|

Re: Linux schema issues

Jay Beale
In reply to this post by Matthew N. Wojcik
I spoke with Mark today a bit about version checking schema change.
Our call got cut off, but we're hoping to open it via a conference
bridge to Woj and Darwin.  We're just trying to get a handle on that
issue.

The one bit that I did get clarified is that the librpm function,
probably rpmvercomp() or something spelled similarly, is a short
function of about 30 lines that simply takes in an RPM name, epoch,
version and release.  These may get passed in as a struct, but it'd be
simple to construct and would not require any other data from the RPM.

Linking against librpm is just providing us with a layer of abstraction
-- if we had to, we could just copy-paste the 30 lines of code! The
library is present on every RH machine.  Mark is trying to encourage Red
Hat engineering to add a --versioncompare or similar function to the rpm
command, so that users can get to this function more directly.


 - Jay



In the wise words of Matthew N. Wojcik:

> To the list: I'd sent Javier mail asking for his thoughts on the
> discussion regarding RPM version comparisons, and I think it's worth
> following up to the list now.  For one thing, I tried to summarize or
> clarify some of the details of that RedHat_RPMVersionCompare table Jay
> mentioned.  So here's the thread...
>
> On Fri, 27 Jun 2003, Javier Fernandez-Sanguino wrote:
>
> > Matthew N. Wojcik wrote:
> > >
> > > This may all be clear to you already, but the discussion on the list
> > > has been a little scattered, so maybe it will help if I recap what
> > > we've been talking about for Red Hat first.
> >
> > > It's a table that would look something like:
> > >
> > > RedHat_RPMVersionCompare -- Hold the results of specific RPM
> > >                          -- version comparisons
> > > ------------------------
> > > RPMName                  -- Package name
> > > RPMTestedVersion         -- Version string to test against
> > > RPMInstalledVersion      -- 'earlier', 'equal', or 'later'
> > >                          -- or maybe -1, 0, or 1
>
> [Note that I changed the column names a bit from what Jay had
> proposed, though I'm still not sure I'm totally happy with them yet.
> And Mark has pointed out that we need to be aware of package epochs as
> well.]
>
> > >
> > > The idea being that an OVAL-compatible tool would populate this table
> > > by using librpm API calls for specific package versions.  So if we
> > > imagine a hypothetical vulnerability in package foo that was fixed in
> > > version foo-0.17-7, we'd have a subquery like:
> >
> > In the Debian case the calls could be made either by calling dpkg
> > directly (dpkg --compare-versions) or by creating a library that
> > implemented the same comparison code in dpkg. As a matter of fact (have
> > not checked fully) apt provides a C++ library (libapt-pkg) which
> > provides a DoCmpVersion.
>
> See below for my reaction to this, after Javi's later comments about a
> library for dpkg...
>
> >
> > >
> > > (SELECT 'foo earlier than 0.17-7 installed' FROM RedHat_RPMVersionCompare
> > >   WHERE RPMName = 'foo' AND
> > >         RPMTestedVersion = '0.17-7' AND
> > >         RPMInstalledVersion = 'earlier')
> > >
> > > To populate that table, an OVAL tool would have to be configured with
> > > the packages and versions to test, retrieve the installed versions of
> > > those packages (if they are installed), then do the comparison and
> > > store the results in the table.
> >
> > How would the OVAL tool know which packages and versions are the queries
> > interested in? Should that be an INSERT statement? That's the only thing
> > that is not clear to me.
>
> In the current approach, yes, that would mean using INSERT statements.
> The query writer would also write INSERT statements for a
> RedHat_RPMVersionCompare_conf table, specifying the package name and
> version (and epoch if any) to test the system against.  The OVAL tool
> would have to somehow read those INSERT statements before it could do
> the data collection.
>
> But as I've said a couple of times recently, I'm not really happy with
> the INSERT statement approach anymore.  It makes pretty good sense if
> you're thinking about an OVAL-compatible tool which is actually using
> a database, and doing all of its data collection ahead of any query
> evaluation.  But for any other approach, there are clearly drawbacks.
> One of my major hopes for the XML encoding for OVAL vulnerability
> specifications is to deal with this issue more cleanly.  So we may not
> need to support INSERT statements too much longer.  Unless perhaps
> there are other OVAL-compatible tools out there or under development
> that are using them that I'm not aware of. :)
>
> > > I saw your post about dpkg --compare-versions from back in May.
> > > Presumably, then we could do something very similar for Debian
> > > packages if you think this is the right way to go.  Any thoughts?
> >
> > That's precisely the way to go. We can either make a direct call to the
> > command or use the provided source code to make a library similar to
> > librpm. Debian does not have such a library at the moment (there is a
> > library provided by apt, but not by dpkg). I could contribute effort
> > into writing that API if that if that was needed.
>
> I think it can be left up to any OVAL-compatible tool for Debian to
> decide whether to call dpkg --compare-versions or to work up a library
> (or just internal code) like you describe.  If it were me, I'd
> probably choose to call dpkg, so that I wouldn't have to update my
> tool's code if the internal semantics of version comparison in the
> official dpkg changed.  Of course, if a library such as you describe
> were to become available as part of the Debian distro in the future,
> then we could update the OVAL Debian schema documentation to suggest
> using it.
>
> Cheers,
>
> --Woj


Reply | Threaded
Open this post in threaded view
|

Re: Linux schema issues

Matthew N. Wojcik
[I'm CC:ing the OVAL-developer-list because we're in a grey area of
implementation issues impacting schema design.  The CC: is mostly for
archival purposes (assuming I get OVAL-dev archives online soon).
Apologies to those who get two copies of this.]

On Tue, 1 Jul 2003, Jay Beale wrote:

> The one bit that I did get clarified is that the librpm function,
> probably rpmvercomp() or something spelled similarly, is a short
> function of about 30 lines that simply takes in an RPM name, epoch,
> version and release.  These may get passed in as a struct, but it'd be
> simple to construct and would not require any other data from the RPM.

Looking at the rpmapi-4.1 documentation (http://www.rpm.org/rpmapi-4.1/)
I think an OVAL implementation would actually need to start with
rpmVersionCompare(), which is a level above rpmvercmp().  See

  http://www.rpm.org/rpmapi-4.1/rpmlib_8h.html#a468
and
  http://www.rpm.org/rpmapi-4.1/psm_8c-source.html#l00038

for the documentation and source code, respectively, for
rpmVersionCompare.  There's a link to rpmvercmp() on line 64, too.

rpmVersionCompare() does the high-level work of comparing epoches, and
calls rpmvercmp() with the version and release successively if the
epochs match.  rpmvercmp() contains the actual "logic" for comparing two
alphanumeric version or release strings, which is where most of the hard
work really is.

The problem is that rpmVersionCompare() expects two Headers as
arguments, and then calls headerGetEntry() to retrieve RPMTAG_EPOCH,
RPMTAG_VERSION, and RPMTAG_RELEASE from each Header.  Clearly the
assumption is the Headers are easily accessible in the normal state of
things, either because the info's in the local RPM database (in the case
of an installed package) or can be read directly from the .rpm file (in
the case of the package to be installed, for example).

Unfortunately, for an OVAL implementation, only one of those (the
already installed version) is actually available on the machine being
tested.  So if an OVAL implementation were to use rpmVersionCompare(),
it would have to build up a bogus Header from information provided,
either by INSERT statements or in OVAL XML.

This makes me a little nervous because building up that structure
doesn't seem completely straightforward.  Not that I need to know
exactly how to code it to be happy with the OVAL schema, of course, but
I'd prefer to write a specification which isn't too much of a pain to
implement. :)

One other possibility would be to write an rpmVersionCompare()
equivalent that takes simpler arguments, does the epoch comparison and
calls rpmvercmp() as needed with to compare the Version and Release.

> Linking against librpm is just providing us with a layer of
> abstraction -- if we had to, we could just copy-paste the 30 lines of
> code! The library is present on every RH machine.  Mark is trying to
> encourage Red Hat engineering to add a --versioncompare or similar
> function to the rpm command, so that users can get to this function
> more directly.

As I've been looking into this further, I'm more and more convinced that
using librpm API calls is the right way to go on this.  The discussion
surrounding bug #50977 on Bugzilla:

 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=50977

shows the difficulty the official maintainers have had trying to get
this stuff right; I don't want OVAL to reinvent the wheel (likely
incorrectly) and have to track rpm development for changes in the
nitty-gritty of the rpmvercmp() function.  But I think we're all pretty
much agreed on that much by now.

So I guess my only remaining question is, what exactly would we need to
provide in INSERT statements to make the comparisions possible?  It
looks to me like package name, epoch, version and release.  Am I missing
anything?

--Woj