Quantcast

XML Schema Versioning

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

XML Schema Versioning

Kirillov, Ivan A.
Hello Everyone,

For version 1.1 of the MAEC schema, we thought it would be nice (and necessary) to incorporate 'real' versioning, as this is something we didn't explicitly consider with the initial release.

With this in mind, we thought we'd solicit your feedback on how to approach this. One solution would be to incorporate namespace-based versioning, with each new schema version having its own namespace. This would mean that older instances could still validate against the older schemas, while also meaning that they would have to be updated for validation against the new schema version.

Since it's likely that we'll split off the objects and enumerations into separate schemas in the near future (version 1.2 or later, more discussion on this soon), the namespaces could take the following form:

Syntax = schema type/version

MAEC 'core' schema: http://maec.mitre.org/XMLSchema/core/1.1
MAEC 'objects' schema: http://maec.mitre.org/XMLSchema/objects/1.1
MAEC 'enumerations' schema: http://maec.mitre.org/XMLSchema/enumerations/1.1

Any thoughts on this? There are clearly other approaches to versioning that we can take (e.g. creating a 'version' element in the schema).

All in all, it seems that typically one strives to maintain a balance between retaining backwards-compatibility and the introduction of new features. This could mean creating namespaces based on the 'major' version of the schema, as with OVAL.

For those that create and update XML schemas, what do you do in terms of versioning?

Have a good weekend!

Regards,
Ivan

Ivan Kirillov
MAEC Working Group
The MITRE Corporation
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

David Ross
Good morning all,


My vote would be to use namespace to control versions as you mentioned.
Any other method will require you to understand the schema prior to determining the version of the schema, which contradicts itself to even say out loud.

We've been dealing with the schema versioning issue for a few years now and namespacing seems to be the best option.
You can always build translators to go from one namespace to another (upgrade/downgrade schema versions).


David M. Ross
MANDIANT
Technical Director
2318 Mill Road
Suite 500
Alexandria, VA 22314
703.683.3141 t
703.683.2891 f
571.435.4721 m
[hidden email]
www.mandiant.com

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Kirillov, Ivan A.
Sent: Friday, December 17, 2010 2:10 PM
To: maec-discussion-list Malware Attribute Enumeration Discussion
Subject: XML Schema Versioning

Hello Everyone,

For version 1.1 of the MAEC schema, we thought it would be nice (and necessary) to incorporate 'real' versioning, as this is something we didn't explicitly consider with the initial release.

With this in mind, we thought we'd solicit your feedback on how to approach this. One solution would be to incorporate namespace-based versioning, with each new schema version having its own namespace. This would mean that older instances could still validate against the older schemas, while also meaning that they would have to be updated for validation against the new schema version.

Since it's likely that we'll split off the objects and enumerations into separate schemas in the near future (version 1.2 or later, more discussion on this soon), the namespaces could take the following form:

Syntax = schema type/version

MAEC 'core' schema: http://maec.mitre.org/XMLSchema/core/1.1
MAEC 'objects' schema: http://maec.mitre.org/XMLSchema/objects/1.1
MAEC 'enumerations' schema: http://maec.mitre.org/XMLSchema/enumerations/1.1

Any thoughts on this? There are clearly other approaches to versioning that we can take (e.g. creating a 'version' element in the schema).

All in all, it seems that typically one strives to maintain a balance between retaining backwards-compatibility and the introduction of new features. This could mean creating namespaces based on the 'major' version of the schema, as with OVAL.

For those that create and update XML schemas, what do you do in terms of versioning?

Have a good weekend!

Regards,
Ivan

Ivan Kirillov
MAEC Working Group
The MITRE Corporation
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Schema Versioning

Julia Cheng
Hi,
    Could you explain what namespace version is ? Sorry I cannot catch the
point!

Julia Cheng


On 2010/12/20 下午10:44, "David Ross" <[hidden email]> wrote:

> Good morning all,
>
>
> My vote would be to use namespace to control versions as you mentioned.
> Any other method will require you to understand the
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

David Ross
"XML is a lot of fun; until you have to take it seriously." This is where we have to take XML seriously.


"XML namespaces provide a simple method for qualifying element and attribute names used in Extensible Markup Language documents by associating them with namespaces identified by URI references." http://www.w3.org/TR/REC-xml-names/

In short, is the location of the XML Schema Definition (XSD),..which is a place that tells you Exactly what is allowed in the XML document. So, when you get an XML document, you can use the XSD to validate it and make sure it is what you are expecting. XSD also allows you to develop tools easier to edit and make use of the XML document structure.

If English were an XML document, all the grammar rules we have could be considered the XSD. The same way you can diagram a sentence to see if it's correct, you can validate XML against it's XSD. Imagine if you had to write and accept a new grammar rule to add a new word to the English language. This is what we have to do in XML. XML is rather strict when you take it seriously.

The problem comes in when you have an XML document that is slightly different than intended (added an element, removed a required element....). It will fail XSD validation and is considered 'malformed'.

So, it is impossible to add a <version> tag in an XML document to establish the XML's version. The tools can't validate the xml in order to read the tag unless the tool already knew the version...in which case, why did we need a version tag?

So, you use these different XSD documents to establish versioning. That way, the tools don't tell you the XML is malformed, it tells you it doesn’t have an XSD for it. (It's very different to say "I don’t understand this", than to say "This is malformed".)

itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.1.xsd"
itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.2.xsd"
itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.3.xsd"
itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_2.0.xsd"

OK, that was a little long.


David M. Ross
MANDIANT
Technical Director
2318 Mill Road
Suite 500
Alexandria, VA 22314
703.683.3141 t
703.683.2891 f
571.435.4721 m
[hidden email]
www.mandiant.com


-----Original Message-----
From: Julia Cheng [mailto:[hidden email]]
Sent: Monday, December 20, 2010 10:16 AM
To: David Ross; maec-discussion-list Malware Attribute Enumeration Discussion
Subject: Re: XML Schema Versioning

Hi,
    Could you explain what namespace version is ? Sorry I cannot catch the
point!

Julia Cheng


On 2010/12/20 下午10:44, "David Ross" <[hidden email]> wrote:

> Good morning all,
>
>
> My vote would be to use namespace to control versions as you mentioned.
> Any other method will require you to understand the
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Schema Versioning

jose nazario
On Dec 20, 2010, at 10:33 AM, David Ross wrote:

> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.1.xsd 
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.2.xsd 
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.3.xsd 
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_2.0.xsd 
> "


i would suggest a less ambiguous name that this. either throw MAEC in  
the path or in the .xsd name; enumerations is a term that has been  
used by many MITRE projects in the infosec space.

-- jose
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

David Ross
You could do that "maec_enumerations_1.1.xsd" but the entire path is the qualifier.
Who's to say someone else wont make a maec_ enumerations_1.1.xsd?
The entire URL is the namespace.

The document name by itself cannot stand for validation so, it would be an un-needed step.


But, if it makes it easier to read...

David M. Ross
MANDIANT
Technical Director
2318 Mill Road
Suite 500
Alexandria, VA 22314
703.683.3141 t
703.683.2891 f
571.435.4721 m
[hidden email]
www.mandiant.com


-----Original Message-----
From: jose nazario [mailto:[hidden email]]
Sent: Monday, December 20, 2010 10:44 AM
To: David Ross
Cc: maec-discussion-list Malware Attribute Enumeration Discussion
Subject: Re: XML Schema Versioning

On Dec 20, 2010, at 10:33 AM, David Ross wrote:

> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.1.xsd
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.2.xsd
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_1.3.xsd
> "
> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_2.0.xsd
> "


i would suggest a less ambiguous name that this. either throw MAEC in
the path or in the .xsd name; enumerations is a term that has been
used by many MITRE projects in the infosec space.

-- jose
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: XML Schema Versioning

jose nazario
In reply to this post by jose nazario
On Dec 20, 2010, at 10:43 AM, jose nazario wrote:

>> itemSchemaLocation=" http://maec.mitre.org/XMLSchema/enumerations_2.0.xsd 
>> "

> i would suggest a less ambiguous name that this. either throw MAEC  
> in the path or in the .xsd name; enumerations is a term that has  
> been used by many MITRE projects in the infosec space.

i withdraw my point as the name MAEC is in the URL. i'm blind.

looks good and sane to me.

-- jose
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

Bauman, Bruce T
Hi, I'm very new to the list and not a subject domain expert; I am
however experienced in XML.  My recommendation would be to use
namespaces for major releases in which there are significant
non-backward compatible changes being introduced, and using a mandatory
XML element or attribute to signify minor backwards compatible version
releases. Internally we use the XML attribute "specificationVersion" to
do this. (We use the term specificationVersion vice schemaVersion to
signify that a "standard" is defined by much more than just the
constraints that can be expressed in an XSD). I don't like using
namespaces for minor version release because technically if you change
the namespace in your schema you are defining a totally new tag set.
This is a very heavy handed approach to deal with minor backwards
compatible version changes where you would like to assert that most of
the tags still are the same.

A couple of clarifications in terminology / concepts:

Namespaces are fundamentally just a way to reduce the chance that two
different groups creating two different tag sets don't create tags with
the same name.  By prefixing the local part of the tag name with a
namespace that itself may be based on something like a domain name means
it is much more likely that the fully qualified tag names (namespace +
local name) defined by each group won't be the same.  Then if some third
party wants to use both of these two tag sets to markup up information
that party won't run into conflicts with the tag names.  Namespaces can
be used to associate a compatible XSD to a document but this is by no
means mandatory, is not the primary purpose of a namespace, and can
indeed be limiting.

One also should keep in mind the difference between a well-formed XML
document which means that syntactically it is correct, and can be parsed
and read, and an XML document that in addition to being well-formed is
also schema-valid against one or more schemata.

Bruce Bauman, U.S. Department of Defense.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

Kirillov, Ivan A.
All,

Thanks for the comments, and Happy New Year! It certainly does appear that namespaces would be the way to go - David, thanks for the great explanation of their function, particularly in relation to validation tools.

I agree with Bruce that it would be beneficial for both ourselves and MAEC end users to have namespaces only for major releases, in order to better retain backwards compatibility among minor releases. To specify the specific (minor) version we can use the 'version' attribute of the xs:schema element. This is also the paradigm that OVAL uses, for much of the same reasons.

As such, here are the planned namespaces as of now:

MAEC 'core' schema: http://maec.mitre.org/XMLSchema/maec-core-1
MAEC 'objects' schema: http://maec.mitre.org/XMLSchema/maec-objects-1
MAEC 'enumerations' schema: http://maec.mitre.org/XMLSchema/maec-enumerations-1

As I had previously mentioned, the 'core' schema namespace will be utilized in MAEC 1.1, while the others will be used in future releases when the object and enumerations are split off.

Regards,
Ivan

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bauman, Bruce T
Sent: Tuesday, December 21, 2010 1:24 PM
To: jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration Discussion
Subject: RE: XML Schema Versioning

Hi, I'm very new to the list and not a subject domain expert; I am
however experienced in XML.  My recommendation would be to use
namespaces for major releases in which there are significant
non-backward compatible changes being introduced, and using a mandatory
XML element or attribute to signify minor backwards compatible version
releases. Internally we use the XML attribute "specificationVersion" to
do this. (We use the term specificationVersion vice schemaVersion to
signify that a "standard" is defined by much more than just the
constraints that can be expressed in an XSD). I don't like using
namespaces for minor version release because technically if you change
the namespace in your schema you are defining a totally new tag set.
This is a very heavy handed approach to deal with minor backwards
compatible version changes where you would like to assert that most of
the tags still are the same.

A couple of clarifications in terminology / concepts:

Namespaces are fundamentally just a way to reduce the chance that two
different groups creating two different tag sets don't create tags with
the same name.  By prefixing the local part of the tag name with a
namespace that itself may be based on something like a domain name means
it is much more likely that the fully qualified tag names (namespace +
local name) defined by each group won't be the same.  Then if some third
party wants to use both of these two tag sets to markup up information
that party won't run into conflicts with the tag names.  Namespaces can
be used to associate a compatible XSD to a document but this is by no
means mandatory, is not the primary purpose of a namespace, and can
indeed be limiting.

One also should keep in mind the difference between a well-formed XML
document which means that syntactically it is correct, and can be parsed
and read, and an XML document that in addition to being well-formed is
also schema-valid against one or more schemata.

Bruce Bauman, U.S. Department of Defense.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

Bauman, Bruce T
The "version" attribute of the xsd:schema element is the way to go to
associate the schema with a version, however you may also need to
directly associate an XML instance with its version. Clearly the
namespace of the instance will associate the instance with its major
version, but is that enough?  If a minor version adds in a new optional
tag the new XSD will validate any previous instance (backward
compatibility), but previous release(s) of the XSD will not necessarily
validate any new instances that contain the optional tag.  If one
receives an arbitrary XML instance how does one know which XSD minor
releases will validate it?

Bruce Bauman

-----Original Message-----
From: Kirillov, Ivan A. [mailto:[hidden email]]
Sent: Wednesday, January 05, 2011 10:19 AM
To: Bauman, Bruce T; jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration
Discussion
Subject: RE: XML Schema Versioning

All,

Thanks for the comments, and Happy New Year! It certainly does appear
that namespaces would be the way to go - David, thanks for the great
explanation of their function, particularly in relation to validation
tools.

I agree with Bruce that it would be beneficial for both ourselves and
MAEC end users to have namespaces only for major releases, in order to
better retain backwards compatibility among minor releases. To specify
the specific (minor) version we can use the 'version' attribute of the
xs:schema element. This is also the paradigm that OVAL uses, for much of
the same reasons.

As such, here are the planned namespaces as of now:

MAEC 'core' schema: http://maec.mitre.org/XMLSchema/maec-core-1
MAEC 'objects' schema: http://maec.mitre.org/XMLSchema/maec-objects-1
MAEC 'enumerations' schema:
http://maec.mitre.org/XMLSchema/maec-enumerations-1

As I had previously mentioned, the 'core' schema namespace will be
utilized in MAEC 1.1, while the others will be used in future releases
when the object and enumerations are split off.

Regards,
Ivan

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Bauman,
Bruce T
Sent: Tuesday, December 21, 2010 1:24 PM
To: jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration
Discussion
Subject: RE: XML Schema Versioning

Hi, I'm very new to the list and not a subject domain expert; I am
however experienced in XML.  My recommendation would be to use
namespaces for major releases in which there are significant
non-backward compatible changes being introduced, and using a mandatory
XML element or attribute to signify minor backwards compatible version
releases. Internally we use the XML attribute "specificationVersion" to
do this. (We use the term specificationVersion vice schemaVersion to
signify that a "standard" is defined by much more than just the
constraints that can be expressed in an XSD). I don't like using
namespaces for minor version release because technically if you change
the namespace in your schema you are defining a totally new tag set.
This is a very heavy handed approach to deal with minor backwards
compatible version changes where you would like to assert that most of
the tags still are the same.

A couple of clarifications in terminology / concepts:

Namespaces are fundamentally just a way to reduce the chance that two
different groups creating two different tag sets don't create tags with
the same name.  By prefixing the local part of the tag name with a
namespace that itself may be based on something like a domain name means
it is much more likely that the fully qualified tag names (namespace +
local name) defined by each group won't be the same.  Then if some third
party wants to use both of these two tag sets to markup up information
that party won't run into conflicts with the tag names.  Namespaces can
be used to associate a compatible XSD to a document but this is by no
means mandatory, is not the primary purpose of a namespace, and can
indeed be limiting.

One also should keep in mind the difference between a well-formed XML
document which means that syntactically it is correct, and can be parsed
and read, and an XML document that in addition to being well-formed is
also schema-valid against one or more schemata.

Bruce Bauman, U.S. Department of Defense.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: XML Schema Versioning

Kirillov, Ivan A.
Great point Bruce. I concur that there may certainly be cases where an instance will validate only against a specific minor version XSD (or range of XSDs), and so it's crucial to have the capability of specifying the minor version in an instance. OVAL actually does this too, I just happened to miss it while looking at its schema. I blame too much time staring at XML.

Anyhow, I will add a 'schema_version' attribute to the MAEC_Bundle element for use in specifying minor versions in MAEC instances.

Regards,
Ivan

Ivan Kirillov
MAEC Working Group
The MITRE Corporation

-----Original Message-----
From: Bauman, Bruce T [mailto:[hidden email]]
Sent: Thursday, January 06, 2011 2:51 PM
To: Kirillov, Ivan A.; jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration Discussion
Subject: RE: XML Schema Versioning

The "version" attribute of the xsd:schema element is the way to go to
associate the schema with a version, however you may also need to
directly associate an XML instance with its version. Clearly the
namespace of the instance will associate the instance with its major
version, but is that enough?  If a minor version adds in a new optional
tag the new XSD will validate any previous instance (backward
compatibility), but previous release(s) of the XSD will not necessarily
validate any new instances that contain the optional tag.  If one
receives an arbitrary XML instance how does one know which XSD minor
releases will validate it?

Bruce Bauman

-----Original Message-----
From: Kirillov, Ivan A. [mailto:[hidden email]]
Sent: Wednesday, January 05, 2011 10:19 AM
To: Bauman, Bruce T; jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration
Discussion
Subject: RE: XML Schema Versioning

All,

Thanks for the comments, and Happy New Year! It certainly does appear
that namespaces would be the way to go - David, thanks for the great
explanation of their function, particularly in relation to validation
tools.

I agree with Bruce that it would be beneficial for both ourselves and
MAEC end users to have namespaces only for major releases, in order to
better retain backwards compatibility among minor releases. To specify
the specific (minor) version we can use the 'version' attribute of the
xs:schema element. This is also the paradigm that OVAL uses, for much of
the same reasons.

As such, here are the planned namespaces as of now:

MAEC 'core' schema: http://maec.mitre.org/XMLSchema/maec-core-1
MAEC 'objects' schema: http://maec.mitre.org/XMLSchema/maec-objects-1
MAEC 'enumerations' schema:
http://maec.mitre.org/XMLSchema/maec-enumerations-1

As I had previously mentioned, the 'core' schema namespace will be
utilized in MAEC 1.1, while the others will be used in future releases
when the object and enumerations are split off.

Regards,
Ivan

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Bauman,
Bruce T
Sent: Tuesday, December 21, 2010 1:24 PM
To: jose nazario
Cc: David Ross; maec-discussion-list Malware Attribute Enumeration
Discussion
Subject: RE: XML Schema Versioning

Hi, I'm very new to the list and not a subject domain expert; I am
however experienced in XML.  My recommendation would be to use
namespaces for major releases in which there are significant
non-backward compatible changes being introduced, and using a mandatory
XML element or attribute to signify minor backwards compatible version
releases. Internally we use the XML attribute "specificationVersion" to
do this. (We use the term specificationVersion vice schemaVersion to
signify that a "standard" is defined by much more than just the
constraints that can be expressed in an XSD). I don't like using
namespaces for minor version release because technically if you change
the namespace in your schema you are defining a totally new tag set.
This is a very heavy handed approach to deal with minor backwards
compatible version changes where you would like to assert that most of
the tags still are the same.

A couple of clarifications in terminology / concepts:

Namespaces are fundamentally just a way to reduce the chance that two
different groups creating two different tag sets don't create tags with
the same name.  By prefixing the local part of the tag name with a
namespace that itself may be based on something like a domain name means
it is much more likely that the fully qualified tag names (namespace +
local name) defined by each group won't be the same.  Then if some third
party wants to use both of these two tag sets to markup up information
that party won't run into conflicts with the tag names.  Namespaces can
be used to associate a compatible XSD to a document but this is by no
means mandatory, is not the primary purpose of a namespace, and can
indeed be limiting.

One also should keep in mind the difference between a well-formed XML
document which means that syntactically it is correct, and can be parsed
and read, and an XML document that in addition to being well-formed is
also schema-valid against one or more schemata.

Bruce Bauman, U.S. Department of Defense.
Loading...