Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

Wheeler, David A

I think the structure of the CWEs related to CWE-119 and CWE-120 needs improving, and in particular the definition of “classic buffer overflow” in CWE-120 needs clarifying.  Below is the current situation, why I think there’s a problem, and a potential solution.

 

 

1. CURRENT SITUATION

 

CWE-119 (“Improper Restriction of Operations within the Bounds of a Memory Buffer”) has many children, including these:

- CWE-120: Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')

- CWE-125: Out-of-bounds Read

- CWE-787: Out-of-bounds Write

There are also many other children of CWE-119 that could be either  reads or writes, e.g.:

- CWE-786: Access of Memory Location Before Start of Buffer

- CWE-788: Access of Memory Location After End of Buffer

 

The extended description of CWE-120 says the following:

“A buffer overflow condition exists when a program attempts to put more data in a buffer than it can hold, or when a program attempts to put data in a memory area outside of the boundaries of a buffer. The simplest type of error, and the most common cause of buffer overflows, is the "classic" case in which the program copies the buffer without restricting how much is copied.”

 

 

2. THE PROBLEM

 

CWE-120 is about not “restricting how much is copied”, and all its examples emphasize the problem of incrementally *writing* until it writes past the end of the buffer.  It doesn’t make sense for it to parallel CWE-787 (out-of-bounds write); any case of CWE-120 would also always be an example of CWE-787.

 

The name of CWE-120 is misleading anyway: “Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')” isn’t really the issue.  Often you can’t really “check the size of the input” ahead-of-time, even though the title implies that you can.  In particular, examples in CWE-120 are given with strcpy() or gets(), and you can’t check ahead-of-time with char*s.  The fundamental issue is that there is no checking to ensure that the destination buffer hasn’t been exceeded.

 

Also, CWE-120 uses the term ‘classic buffer overflow’, but that term is not clearly defined.  You also have to read the “terminology notes” of CWE-120 carefully to realize that the term “buffer overflow” is used with a possibly-different meaning and that “buffer overflow” may apply to other CWEs as well… readers of just CWE-787 or CWE-119 would have no idea, since there’s no similar note there.

 

I’m updating my book to refer to various CWEs.  Buffer overflows are still really common; having a confusing structure and definitions in such an important area is a real problem, and makes it tricky for me to refer to CWEs when discussing buffer overflows including classic buffer overflows.  I think this can be tweaked to be better in a backwards-compatible way.

 

 

 

3. A PROPOSED SOLUTION

 

CWE-120 should not descend directly from CWE-119.  Instead, CWE-120 should descend from CWE-787.  I think CWE-120 should descend from CWE-788 as well.  That way it’s clear that a “classic buffer overflow” is a write after the end of the buffer using data from some processing or input source.

 

I suggest renaming CWE-120 to: “Copy information without checking for end of buffer (‘classic buffer overflow’)”.

Also, change the CWE-120 description summary to:

“The program incrementally copies a data source (e.g., an input buffer or processed results) to an output buffer without verifying that the size of the output buffer has not been exceeded, leading to a write after the end of the buffer.”

 

Also: What does the term “buffer overflow” mean?  It’s a widely-used term; the CWE should NOT be silent about this term.  If it means any write past the bounds, then CWE-787 should say that it means ‘buffer overflow’.  If it means any improper restriction, then CWE-119 should say that it’s the representation for the term ‘buffer overflow’.  However, if the term seems to have different meanings among practitioners – and I think THAT is the real case - then all CWEs that someone might use for ‘buffer overflow’ should note this terminological imprecision.   In other words, ALL the CWEs someone might use (say 119, 787, 788, 120) should have a “Terminology notes” section (or similar easily seen portion), with something like this: “The term *buffer overflow* is widely used by practitioners, but at this time there are some variations in the intended meaning of this term; a speaker might mean CWE-119, CWE-787, CWE-788, and/or CWE-120.   If precision is needed, use the CWE identifier instead of the term ‘buffer overflow’.  For a classic buffer overflow, use CWE-120.”  Of course, make all those CWE id’s hyperlinked.

 

Anyway, I hope this helps.

 

--- David A. Wheeler

 

JA
Reply | Threaded
Open this post in threaded view
|

Re: Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

JA
Hi,

after first quick review, I would disagree (at least for now).

You seem to say that "copy is a child of write", and so CWE-120 should
be a childof CWE-787) am I correct?

Because of the well defined relationships for CWE-120, its common
usage in the field (e.g. http://cwe.mitre.org/top25/#CWE-120), such as
in code analysis tools, etc. I would highly not recommend to "replace
it" by CWE-787

About the examples with references to strcpy() or gets() (here not
that while illustrating use of gets() in a CWE childof CWE-787 would
not fit very well), IMHO there are there to illustrate the issue, and
'misuse cases'.
However, for sure we could improve the Potential Mitigations, like In
place of gets() or using scanf to read in a string, use fgets()

I would have to review if CWE-787 would better fit as childof CWE-120,
instead of childof CWE-119. What do you think?

Regarding the term "buffer overflow', as far as I know, or remember, I
think consensus is that one of the first technical description was
http://insecure.org/stf/smashstack.html

My 2c

2015-02-22 5:07 GMT+01:00 Wheeler, David A <[hidden email]>:

> I think the structure of the CWEs related to CWE-119 and CWE-120 needs
> improving, and in particular the definition of “classic buffer overflow” in
> CWE-120 needs clarifying.  Below is the current situation, why I think
> there’s a problem, and a potential solution.
>
>
>
>
>
> 1. CURRENT SITUATION
>
>
>
> CWE-119 (“Improper Restriction of Operations within the Bounds of a Memory
> Buffer”) has many children, including these:
>
> - CWE-120: Buffer Copy without Checking Size of Input ('Classic Buffer
> Overflow')
>
> - CWE-125: Out-of-bounds Read
>
> - CWE-787: Out-of-bounds Write
>
> There are also many other children of CWE-119 that could be either  reads or
> writes, e.g.:
>
> - CWE-786: Access of Memory Location Before Start of Buffer
>
> - CWE-788: Access of Memory Location After End of Buffer
>
>
>
> The extended description of CWE-120 says the following:
>
> “A buffer overflow condition exists when a program attempts to put more data
> in a buffer than it can hold, or when a program attempts to put data in a
> memory area outside of the boundaries of a buffer. The simplest type of
> error, and the most common cause of buffer overflows, is the "classic" case
> in which the program copies the buffer without restricting how much is
> copied.”
>
>
>
>
>
> 2. THE PROBLEM
>
>
>
> CWE-120 is about not “restricting how much is copied”, and all its examples
> emphasize the problem of incrementally *writing* until it writes past the
> end of the buffer.  It doesn’t make sense for it to parallel CWE-787
> (out-of-bounds write); any case of CWE-120 would also always be an example
> of CWE-787.
>
>
>
> The name of CWE-120 is misleading anyway: “Buffer Copy without Checking Size
> of Input ('Classic Buffer Overflow')” isn’t really the issue.  Often you
> can’t really “check the size of the input” ahead-of-time, even though the
> title implies that you can.  In particular, examples in CWE-120 are given
> with strcpy() or gets(), and you can’t check ahead-of-time with char*s.  The
> fundamental issue is that there is no checking to ensure that the
> destination buffer hasn’t been exceeded.
>
>
>
> Also, CWE-120 uses the term ‘classic buffer overflow’, but that term is not
> clearly defined.  You also have to read the “terminology notes” of CWE-120
> carefully to realize that the term “buffer overflow” is used with a
> possibly-different meaning and that “buffer overflow” may apply to other
> CWEs as well… readers of just CWE-787 or CWE-119 would have no idea, since
> there’s no similar note there.
>
>
>
> I’m updating my book to refer to various CWEs.  Buffer overflows are still
> really common; having a confusing structure and definitions in such an
> important area is a real problem, and makes it tricky for me to refer to
> CWEs when discussing buffer overflows including classic buffer overflows.  I
> think this can be tweaked to be better in a backwards-compatible way.
>
>
>
>
>
>
>
> 3. A PROPOSED SOLUTION
>
>
>
> CWE-120 should not descend directly from CWE-119.  Instead, CWE-120 should
> descend from CWE-787.  I think CWE-120 should descend from CWE-788 as well.
> That way it’s clear that a “classic buffer overflow” is a write after the
> end of the buffer using data from some processing or input source.
>
>
>
> I suggest renaming CWE-120 to: “Copy information without checking for end of
> buffer (‘classic buffer overflow’)”.
>
> Also, change the CWE-120 description summary to:
>
> “The program incrementally copies a data source (e.g., an input buffer or
> processed results) to an output buffer without verifying that the size of
> the output buffer has not been exceeded, leading to a write after the end of
> the buffer.”
>
>
>
> Also: What does the term “buffer overflow” mean?  It’s a widely-used term;
> the CWE should NOT be silent about this term.  If it means any write past
> the bounds, then CWE-787 should say that it means ‘buffer overflow’.  If it
> means any improper restriction, then CWE-119 should say that it’s the
> representation for the term ‘buffer overflow’.  However, if the term seems
> to have different meanings among practitioners – and I think THAT is the
> real case - then all CWEs that someone might use for ‘buffer overflow’
> should note this terminological imprecision.   In other words, ALL the CWEs
> someone might use (say 119, 787, 788, 120) should have a “Terminology notes”
> section (or similar easily seen portion), with something like this: “The
> term *buffer overflow* is widely used by practitioners, but at this time
> there are some variations in the intended meaning of this term; a speaker
> might mean CWE-119, CWE-787, CWE-788, and/or CWE-120.   If precision is
> needed, use the CWE identifier instead of the term ‘buffer overflow’.  For a
> classic buffer overflow, use CWE-120.”  Of course, make all those CWE id’s
> hyperlinked.
>
>
>
> Anyway, I hope this helps.
>
>
>
> --- David A. Wheeler
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

Jeffrey Walton
In reply to this post by Wheeler, David A
Hi David,

> The name of CWE-120 is misleading anyway: “Buffer Copy without Checking Size
> of Input ('Classic Buffer Overflow')” isn’t really the issue.  Often you
> can’t really “check the size of the input” ahead-of-time, even though the
> title implies that you can.

My only observation here is, if you can't then its usually because of
a *bad design*. Its possible to do with a good design because good
designs provide the necessary information (i.e., a buffer and its
length).

So I'm not sure its a good idea to beg a bad design.

Or maybe its OK because it reflects the state of the art for a lot of
software... You gotta love copy/paste of terse K&R code....

> I’m updating my book to refer to various CWEs.  Buffer overflows are still
> really common;

Off topic, but ... please let me know if I can perform a read for you.
I'm fanatical about doing the Comp SCi 101 things all the time to
avoid security related defects, and I regularly cite your HowTo for
Linux platforms. (Along with Apple and Microsoft's Secure Coding
Guides).

Jeff

On Sat, Feb 21, 2015 at 11:07 PM, Wheeler, David A <[hidden email]> wrote:

> I think the structure of the CWEs related to CWE-119 and CWE-120 needs
> improving, and in particular the definition of “classic buffer overflow” in
> CWE-120 needs clarifying.  Below is the current situation, why I think
> there’s a problem, and a potential solution.
>
>
>
>
>
> 1. CURRENT SITUATION
>
>
>
> CWE-119 (“Improper Restriction of Operations within the Bounds of a Memory
> Buffer”) has many children, including these:
>
> - CWE-120: Buffer Copy without Checking Size of Input ('Classic Buffer
> Overflow')
>
> - CWE-125: Out-of-bounds Read
>
> - CWE-787: Out-of-bounds Write
>
> There are also many other children of CWE-119 that could be either  reads or
> writes, e.g.:
>
> - CWE-786: Access of Memory Location Before Start of Buffer
>
> - CWE-788: Access of Memory Location After End of Buffer
>
>
>
> The extended description of CWE-120 says the following:
>
> “A buffer overflow condition exists when a program attempts to put more data
> in a buffer than it can hold, or when a program attempts to put data in a
> memory area outside of the boundaries of a buffer. The simplest type of
> error, and the most common cause of buffer overflows, is the "classic" case
> in which the program copies the buffer without restricting how much is
> copied.”
>
>
>
>
>
> 2. THE PROBLEM
>
>
>
> CWE-120 is about not “restricting how much is copied”, and all its examples
> emphasize the problem of incrementally *writing* until it writes past the
> end of the buffer.  It doesn’t make sense for it to parallel CWE-787
> (out-of-bounds write); any case of CWE-120 would also always be an example
> of CWE-787.
>
>
>
> The name of CWE-120 is misleading anyway: “Buffer Copy without Checking Size
> of Input ('Classic Buffer Overflow')” isn’t really the issue.  Often you
> can’t really “check the size of the input” ahead-of-time, even though the
> title implies that you can.  In particular, examples in CWE-120 are given
> with strcpy() or gets(), and you can’t check ahead-of-time with char*s.  The
> fundamental issue is that there is no checking to ensure that the
> destination buffer hasn’t been exceeded.
>
>
>
> Also, CWE-120 uses the term ‘classic buffer overflow’, but that term is not
> clearly defined.  You also have to read the “terminology notes” of CWE-120
> carefully to realize that the term “buffer overflow” is used with a
> possibly-different meaning and that “buffer overflow” may apply to other
> CWEs as well… readers of just CWE-787 or CWE-119 would have no idea, since
> there’s no similar note there.
>
>
>
> I’m updating my book to refer to various CWEs.  Buffer overflows are still
> really common; having a confusing structure and definitions in such an
> important area is a real problem, and makes it tricky for me to refer to
> CWEs when discussing buffer overflows including classic buffer overflows.  I
> think this can be tweaked to be better in a backwards-compatible way.
>
>
>
>
>
>
>
> 3. A PROPOSED SOLUTION
>
>
>
> CWE-120 should not descend directly from CWE-119.  Instead, CWE-120 should
> descend from CWE-787.  I think CWE-120 should descend from CWE-788 as well.
> That way it’s clear that a “classic buffer overflow” is a write after the
> end of the buffer using data from some processing or input source.
>
>
>
> I suggest renaming CWE-120 to: “Copy information without checking for end of
> buffer (‘classic buffer overflow’)”.
>
> Also, change the CWE-120 description summary to:
>
> “The program incrementally copies a data source (e.g., an input buffer or
> processed results) to an output buffer without verifying that the size of
> the output buffer has not been exceeded, leading to a write after the end of
> the buffer.”
>
>
>
> Also: What does the term “buffer overflow” mean?  It’s a widely-used term;
> the CWE should NOT be silent about this term.  If it means any write past
> the bounds, then CWE-787 should say that it means ‘buffer overflow’.  If it
> means any improper restriction, then CWE-119 should say that it’s the
> representation for the term ‘buffer overflow’.  However, if the term seems
> to have different meanings among practitioners – and I think THAT is the
> real case - then all CWEs that someone might use for ‘buffer overflow’
> should note this terminological imprecision.   In other words, ALL the CWEs
> someone might use (say 119, 787, 788, 120) should have a “Terminology notes”
> section (or similar easily seen portion), with something like this: “The
> term *buffer overflow* is widely used by practitioners, but at this time
> there are some variations in the intended meaning of this term; a speaker
> might mean CWE-119, CWE-787, CWE-788, and/or CWE-120.   If precision is
> needed, use the CWE identifier instead of the term ‘buffer overflow’.  For a
> classic buffer overflow, use CWE-120.”  Of course, make all those CWE id’s
> hyperlinked.
>
>
>
> Anyway, I hope this helps.
>
>
>
> --- David A. Wheeler
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

Wheeler, David A
I said:
>> The name of CWE-120 is misleading anyway: “Buffer Copy without
>> Checking Size of Input ('Classic Buffer Overflow')” isn’t really the
>> issue.  Often you can’t really “check the size of the input”
>> ahead-of-time, even though the title implies that you can.

From: Jeffrey Walton [mailto:[hidden email]]:
> My only observation here is, if you can't then its usually because of a *bad design*. Its possible to do with a good design because good designs provide the necessary information (i.e., a buffer and its length).

That may be, but the CWE is not a catalog of good designs :-).  Since this is a common mistake that can lead to a vulnerability, it needs a clear CWE id.

One of my points is that the "input" leading to a classic buffer overflow may not be from a buffer; it may be from an external (network) source. In that case, there may be no input size information available. And that's expected.  Protocols that state the length at the beginning can also be a bad design; it means that you have to queue up and delay the entire transmission until you know its length, which can be disastrous if the material is dynamically generated, and such protocols CANNOT be used for real-time streaming.  CWE-120 example 3 uses "gets", yet gets does not necessarily use an input buffer at all.   I think the title of CWE-120 just needs tweaking, because the problem is a lack of *checking* of the buffer being written to, not really the size of the input.  (As long as the output buffer is big enough, it doesn't matter what the input does.)


>> I’m updating my book to refer to various CWEs.  Buffer overflows are
>> still really common;

> Off topic, but ... please let me know if I can perform a read for you.
> I'm fanatical about doing the Comp SCi 101 things all the time to avoid security related defects,
> and I regularly cite your HowTo for Linux platforms. (Along with Apple and Microsoft's Secure Coding Guides).

That is fantastic!  Thanks so much.  I'm just starting to update it (haven't done so for a while), so I'll probably take you up on that.

--- David A. Wheeler

Reply | Threaded
Open this post in threaded view
|

Re: Rationalize CWE-119 (improper buffer bound restriction) and CWE-120 (classic buffer overflow)

Pascal Meunier
In reply to this post by Wheeler, David A
David, I understand your position as approaching the CWE by looking at end
results, and from that point of view I agree with your analysis and would agree
with your proposal.  However I believe that's not what the CWE intended to do.

I understand the difference between CWE-787 and CWE-120 in how or why they
happen, not the end result which can overlap.  I believe this is correct
because the CWE is (or was originally) focused on how errors are made, and not
directly on the effects. 120 is about a copy operation not comparing, or not
being able to check ahead of time and not running a count, the size of an input
to the size of the buffer. 787 is about trying to do a copy correctly, taking
sizes into account but failing, or using other operations (array assignment)
that inadvertently write outside a buffer. The difference is in the intent and
the type of operation.

I believe it is appropriate to link leaves to a parent by comparing the end
results, but the leaves should be describing how it happened. The descriptions
and relationships for these CWEs need to make this clearer. Perhaps CWE-787
should be split into "trying to do a copy correctly but miscalculating" and
"operations other than a copy, that write out of bounds", or rewritten to
clearly define these cases. CWE-120 has a clear place in my mind as "copying
or writing sequentially into a buffer without a care", which is the classic
buffer overflow for me.

Pascal

On Sat, 21 Feb 2015 23:07:40 -0500
"Wheeler, David A" <[hidden email]> wrote:

> I think the structure of the CWEs related to CWE-119 and CWE-120 needs
> improving, and in particular the definition of "classic buffer overflow" in
> CWE-120 needs clarifying.  Below is the current situation, why I think
> there's a problem, and a potential solution.
>
>
> 1. CURRENT SITUATION
>
> CWE-119 ("Improper Restriction of Operations within the Bounds of a Memory
> Buffer") has many children, including these:
> - CWE-120: Buffer Copy without Checking Size of Input ('Classic Buffer
> Overflow')
> - CWE-125: Out-of-bounds Read
> - CWE-787: Out-of-bounds Write
> There are also many other children of CWE-119 that could be either  reads or
> writes, e.g.:
> - CWE-786: Access of Memory Location Before Start of Buffer
> - CWE-788: Access of Memory Location After End of Buffer
>
> The extended description of CWE-120 says the following:
> "A buffer overflow condition exists when a program attempts to put more data
> in a buffer than it can hold, or when a program attempts to put data in a
> memory area outside of the boundaries of a buffer. The simplest type of
> error, and the most common cause of buffer overflows, is the "classic" case
> in which the program copies the buffer without restricting how much is
> copied."
>
>
> 2. THE PROBLEM
>
> CWE-120 is about not "restricting how much is copied", and all its examples
> emphasize the problem of incrementally *writing* until it writes past the end
> of the buffer.  It doesn't make sense for it to parallel CWE-787
> (out-of-bounds write); any case of CWE-120 would also always be an example of
> CWE-787.
>
> The name of CWE-120 is misleading anyway: "Buffer Copy without Checking Size
> of Input ('Classic Buffer Overflow')" isn't really the issue.  Often you
> can't really "check the size of the input" ahead-of-time, even though the
> title implies that you can.  In particular, examples in CWE-120 are given
> with strcpy() or gets(), and you can't check ahead-of-time with char*s.  The
> fundamental issue is that there is no checking to ensure that the destination
> buffer hasn't been exceeded.
>
> Also, CWE-120 uses the term 'classic buffer overflow', but that term is not
> clearly defined.  You also have to read the "terminology notes" of CWE-120
> carefully to realize that the term "buffer overflow" is used with a
> possibly-different meaning and that "buffer overflow" may apply to other CWEs
> as well... readers of just CWE-787 or CWE-119 would have no idea, since
> there's no similar note there.
>
> I'm updating my book to refer to various CWEs.  Buffer overflows are still
> really common; having a confusing structure and definitions in such an
> important area is a real problem, and makes it tricky for me to refer to CWEs
> when discussing buffer overflows including classic buffer overflows.  I think
> this can be tweaked to be better in a backwards-compatible way.
>
>
>
> 3. A PROPOSED SOLUTION
>
> CWE-120 should not descend directly from CWE-119.  Instead, CWE-120 should
> descend from CWE-787.  I think CWE-120 should descend from CWE-788 as well.
> That way it's clear that a "classic buffer overflow" is a write after the end
> of the buffer using data from some processing or input source.
>
> I suggest renaming CWE-120 to: "Copy information without checking for end of
> buffer ('classic buffer overflow')". Also, change the CWE-120 description
> summary to: "The program incrementally copies a data source (e.g., an input
> buffer or processed results) to an output buffer without verifying that the
> size of the output buffer has not been exceeded, leading to a write after the
> end of the buffer."
>
> Also: What does the term "buffer overflow" mean?  It's a widely-used term;
> the CWE should NOT be silent about this term.  If it means any write past the
> bounds, then CWE-787 should say that it means 'buffer overflow'.  If it means
> any improper restriction, then CWE-119 should say that it's the
> representation for the term 'buffer overflow'.  However, if the term seems to
> have different meanings among practitioners - and I think THAT is the real
> case - then all CWEs that someone might use for 'buffer overflow' should note
> this terminological imprecision.   In other words, ALL the CWEs someone might
> use (say 119, 787, 788, 120) should have a "Terminology notes" section (or
> similar easily seen portion), with something like this: "The term *buffer
> overflow* is widely used by practitioners, but at this time there are some
> variations in the intended meaning of this term; a speaker might mean
> CWE-119, CWE-787, CWE-788, and/or CWE-120.   If precision is needed, use the
> CWE identifier instead of the term 'buffer overflow'.  For a classic buffer
> overflow, use CWE-120."  Of course, make all those CWE id's hyperlinked.
>
> Anyway, I hope this helps.
>
> --- David A. Wheeler
>