Process58 - command_line

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Process58 - command_line

Zbynek Moravec
Hi,
I am fixing implementation of process58_test in OpenSCAP.

Documentation (https://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation/unix-definitions-schema.html#process58_test) says "It is equivalent to parsing the output of the ps command."

Information for command_line element is available in /proc/${PID}/cmdline, that's fine. But there is problem with encoding/escaping data from this "file" - specially non-printable/non ASCII characters. Equivalent behavior with "ps command" can be reached "only" by "copy-pasting" of ps-source. It is not quite good practice.

Is there any more useful definition or better solution?


Thank you very much for your reply.

--
Zbynek Moravec

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Process58 - command_line

Steve Grubb
On Monday, July 20, 2015 10:49:40 AM Zbynek Moravec wrote:
> Hi,
> I am fixing implementation of process58_test in OpenSCAP.
>
> Documentation
> (https://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation
> /unix-definitions-schema.html#process58_test) says "It is equivalent to
> parsing the output of the ps command."

I think that is general guidance to help orient you to the big picture.


> Information for command_line element is available in /proc/${PID}/cmdline,
> that's fine. But there is problem with encoding/escaping data from this
> "file" - specially non-printable/non ASCII characters. Equivalent behavior
> with "ps command" can be reached "only" by "copy-pasting" of ps-source. It
> is not quite good practice.

You didn't give any example about what the issue actually is. So, I am going
to take a guess about what you are asking. The /proc/<pid>/cmdline output is
actually a zero delimited array of strings. This is because it corresponds to
argv[] that is passed to main. Even when you call execve, it expects that you
give it argv[].

But let's look at an example. I have this for dhclient in the output of ps:

/sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-
p4p1.pid -lf /var/lib/NetworkManager/dhclient-db96ea25-
acd9-4226-8670-3e1c8d53262d-p4p1.lease -cf /var/lib/NetworkManager/dhclient-
p4p1.conf p4p1

You can see what is actually pulled out of /proc with strace:

read(3, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 131072) = 221
write(1, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 221/sbin/dhclient-d-
q-sf/usr/libexec/nm-dhcp-helper-pf/var/run/dhclient-p4p1.pid-
lf/var/lib/NetworkManager/dhclient-db96ea25-acd9-4226-8670-3e1c8d53262d-
p4p1.lease-cf/var/lib/NetworkManager/dhclient-p4p1.confp4p1) = 221

Notice all the '\0'. Also notice that read returned 221 bytes, This means that
you cannot just printf the buffer from the read. You have to iterate over the
NUL terminated array until you have output all of it. You would want to insert
a space between each argument when outputting it, too. Hope this helps...

-Steve

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Process58 - command_line

Zbynek Moravec
----- Original Message -----

> From: "Steve Grubb" <[hidden email]>
> To: [hidden email]
> Sent: Tuesday, July 21, 2015 12:27:19 AM
> Subject: Re: [OVAL-DEVELOPER-LIST] Process58 - command_line
>
> On Monday, July 20, 2015 10:49:40 AM Zbynek Moravec wrote:
> > Hi,
> > I am fixing implementation of process58_test in OpenSCAP.
> >
> > Documentation
> > (https://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation
> > /unix-definitions-schema.html#process58_test) says "It is equivalent to
> > parsing the output of the ps command."
>
> I think that is general guidance to help orient you to the big picture.
>
>
> > Information for command_line element is available in /proc/${PID}/cmdline,
> > that's fine. But there is problem with encoding/escaping data from this
> > "file" - specially non-printable/non ASCII characters. Equivalent behavior
> > with "ps command" can be reached "only" by "copy-pasting" of ps-source. It
> > is not quite good practice.
>
> You didn't give any example about what the issue actually is. So, I am going
> to take a guess about what you are asking. The /proc/<pid>/cmdline output is
> actually a zero delimited array of strings. This is because it corresponds to
> argv[] that is passed to main. Even when you call execve, it expects that you
> give it argv[].
>
> But let's look at an example. I have this for dhclient in the output of ps:
>
> /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-
> p4p1.pid -lf /var/lib/NetworkManager/dhclient-db96ea25-
> acd9-4226-8670-3e1c8d53262d-p4p1.lease -cf /var/lib/NetworkManager/dhclient-
> p4p1.conf p4p1
>
> You can see what is actually pulled out of /proc with strace:
>
> read(3, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
> pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
> db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
> cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 131072) = 221
> write(1, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
> pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
> db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
> cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 221/sbin/dhclient-d-
> q-sf/usr/libexec/nm-dhcp-helper-pf/var/run/dhclient-p4p1.pid-
> lf/var/lib/NetworkManager/dhclient-db96ea25-acd9-4226-8670-3e1c8d53262d-
> p4p1.lease-cf/var/lib/NetworkManager/dhclient-p4p1.confp4p1) = 221
>
> Notice all the '\0'. Also notice that read returned 221 bytes, This means
> that
> you cannot just printf the buffer from the read. You have to iterate over the
> NUL terminated array until you have output all of it. You would want to
> insert
> a space between each argument when outputting it, too. Hope this helps...
>
> -Steve
>


Hi, example you gave contains only printable & ascii characters.

Problem is with processes like this:

$ ./stop.sh simpleParam "`echo -e \"\n\e\"`" "Příliš žluťoučký kůň úpěl ďábelské ódy"

$ cat /proc/9080/cmdline |hexdump -C
00000000  2f 62 69 6e 2f 62 61 73  68 00 2e 2f 73 74 6f 70  |/bin/bash../stop|
00000010  2e 73 68 00 73 69 6d 70  6c 65 50 61 72 61 6d 00  |.sh.simpleParam.|
00000020  0a 1b 00 50 c5 99 c3 ad  6c 69 c5 a1 20 c5 be 6c  |...P....li.. ..l| // contains "\n\e" as expected
00000030  75 c5 a5 6f 75 c4 8d 6b  c3 bd 20 6b c5 af c5 88  |u..ou..k.. k....|
00000040  20 c3 ba 70 c4 9b 6c 20  c4 8f c3 a1 62 65 6c 73  | ..p..l ....bels|
00000050  6b c3 a9 20 c3 b3 64 79  00                       |k.. ..dy.|
00000059

And ps output looks like

$ ps a | grep stop
 9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  ? Příliš žluťoučký kůň úp?l ďábelské ódy

$ LC_ALL=C ps a | grep stop
 9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  . P????li?? ??lu??ou??k?? k???? ??p??l ????belsk?? ??dy

'\n' is replaced by space - it is ok

'\e' is replaced by '?' or by '.'
"czech characters" are sometimes replaced by '?'

"Problem" I mentioned is how to copy this behavior without exact documentation.

Zbynek Moravec

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Process58 - command_line

David Solin-3
I presume the file must be UTF-8 encoded.  (Bytes only map directly to characters in ASCII).

   



> On Jul 21, 2015, at 1:50 AM, Zbynek Moravec <[hidden email]> wrote:
>
> ----- Original Message -----
>> From: "Steve Grubb" <[hidden email]>
>> To: [hidden email]
>> Sent: Tuesday, July 21, 2015 12:27:19 AM
>> Subject: Re: [OVAL-DEVELOPER-LIST] Process58 - command_line
>>
>> On Monday, July 20, 2015 10:49:40 AM Zbynek Moravec wrote:
>>> Hi,
>>> I am fixing implementation of process58_test in OpenSCAP.
>>>
>>> Documentation
>>> (https://oval.mitre.org/language/version5.10.1/ovaldefinition/documentation
>>> /unix-definitions-schema.html#process58_test) says "It is equivalent to
>>> parsing the output of the ps command."
>>
>> I think that is general guidance to help orient you to the big picture.
>>
>>
>>> Information for command_line element is available in /proc/${PID}/cmdline,
>>> that's fine. But there is problem with encoding/escaping data from this
>>> "file" - specially non-printable/non ASCII characters. Equivalent behavior
>>> with "ps command" can be reached "only" by "copy-pasting" of ps-source. It
>>> is not quite good practice.
>>
>> You didn't give any example about what the issue actually is. So, I am going
>> to take a guess about what you are asking. The /proc/<pid>/cmdline output is
>> actually a zero delimited array of strings. This is because it corresponds to
>> argv[] that is passed to main. Even when you call execve, it expects that you
>> give it argv[].
>>
>> But let's look at an example. I have this for dhclient in the output of ps:
>>
>> /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-
>> p4p1.pid -lf /var/lib/NetworkManager/dhclient-db96ea25-
>> acd9-4226-8670-3e1c8d53262d-p4p1.lease -cf /var/lib/NetworkManager/dhclient-
>> p4p1.conf p4p1
>>
>> You can see what is actually pulled out of /proc with strace:
>>
>> read(3, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
>> pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
>> db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
>> cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 131072) = 221
>> write(1, "/sbin/dhclient\0-d\0-q\0-sf\0/usr/libexec/nm-dhcp-helper\0-
>> pf\0/var/run/dhclient-p4p1.pid\0-lf\0/var/lib/NetworkManager/dhclient-
>> db96ea25-acd9-4226-8670-3e1c8d53262d-p4p1.lease\0-
>> cf\0/var/lib/NetworkManager/dhclient-p4p1.conf\0p4p1\0", 221/sbin/dhclient-d-
>> q-sf/usr/libexec/nm-dhcp-helper-pf/var/run/dhclient-p4p1.pid-
>> lf/var/lib/NetworkManager/dhclient-db96ea25-acd9-4226-8670-3e1c8d53262d-
>> p4p1.lease-cf/var/lib/NetworkManager/dhclient-p4p1.confp4p1) = 221
>>
>> Notice all the '\0'. Also notice that read returned 221 bytes, This means
>> that
>> you cannot just printf the buffer from the read. You have to iterate over the
>> NUL terminated array until you have output all of it. You would want to
>> insert
>> a space between each argument when outputting it, too. Hope this helps...
>>
>> -Steve
>>
>
>
> Hi, example you gave contains only printable & ascii characters.
>
> Problem is with processes like this:
>
> $ ./stop.sh simpleParam "`echo -e \"\n\e\"`" "Příliš žluťoučký kůň úpěl ďábelské ódy"
>
> $ cat /proc/9080/cmdline |hexdump -C
> 00000000  2f 62 69 6e 2f 62 61 73  68 00 2e 2f 73 74 6f 70  |/bin/bash../stop|
> 00000010  2e 73 68 00 73 69 6d 70  6c 65 50 61 72 61 6d 00  |.sh.simpleParam.|
> 00000020  0a 1b 00 50 c5 99 c3 ad  6c 69 c5 a1 20 c5 be 6c  |...P....li.. ..l| // contains "\n\e" as expected
> 00000030  75 c5 a5 6f 75 c4 8d 6b  c3 bd 20 6b c5 af c5 88  |u..ou..k.. k....|
> 00000040  20 c3 ba 70 c4 9b 6c 20  c4 8f c3 a1 62 65 6c 73  | ..p..l ....bels|
> 00000050  6b c3 a9 20 c3 b3 64 79  00                       |k.. ..dy.|
> 00000059
>
> And ps output looks like
>
> $ ps a | grep stop
> 9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  ? Příliš žluťoučký kůň úp?l ďábelské ódy
>
> $ LC_ALL=C ps a | grep stop
> 9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  . P????li?? ??lu??ou??k?? k???? ??p??l ????belsk?? ??dy
>
> '\n' is replaced by space - it is ok
>
> '\e' is replaced by '?' or by '.'
> "czech characters" are sometimes replaced by '?'
>
> "Problem" I mentioned is how to copy this behavior without exact documentation.
>
> Zbynek Moravec
>
> To unsubscribe, send an email message to [hidden email] with
> SIGNOFF OVAL-DEVELOPER-LIST
> in the BODY of the message.  If you have difficulties, write to [hidden email].

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: Process58 - command_line

Steve Grubb
In reply to this post by Zbynek Moravec
Hello,

The issue that you mention is a generic issue throughout all OVAL schemas and
not specific to process58. The only wrinkle to process58 is that you have to be
aware of the zero delimited arrays of characters. You have this problem with
file names, user names, process names, and anything a user can change like file
contents, extended attributes, directories, mount points, environmental
variables, and even remote system if you support the dns cache test. Its
everywhere.

On Tuesday, July 21, 2015 02:50:23 AM Zbynek Moravec wrote:

> Problem is with processes like this:
>
> $ ./stop.sh simpleParam "`echo -e \"\n\e\"`" "Příliš žluťoučký kůň úpěl
> ďábelské ódy"
>
> $ cat /proc/9080/cmdline |hexdump -C
> 00000000  2f 62 69 6e 2f 62 61 73  68 00 2e 2f 73 74 6f 70
> |/bin/bash../stop| 00000010  2e 73 68 00 73 69 6d 70  6c 65 50 61 72 61 6d
> 00  |.sh.simpleParam.| 00000020  0a 1b 00 50 c5 99 c3 ad  6c 69 c5 a1 20 c5
> be 6c  |...P....li.. ..l| // contains "\n\e" as expected 00000030  75 c5 a5
> 6f 75 c4 8d 6b  c3 bd 20 6b c5 af c5 88  |u..ou..k.. k....| 00000040  20 c3
> ba 70 c4 9b 6c 20  c4 8f c3 a1 62 65 6c 73  | ..p..l ....bels| 00000050  6b
> c3 a9 20 c3 b3 64 79  00                       |k.. ..dy.| 00000059
>
> And ps output looks like
>
> $ ps a | grep stop
>  9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  ? Příliš
> žluťoučký kůň úp?l ďábelské ódy

My guess is the process is created in the Czech locale and ps is using the
Czech locale.


> $ LC_ALL=C ps a | grep stop
>  9080 pts/10   T      0:00 /bin/bash ./stop.sh simpleParam  . P????li??
> ??lu??ou??k?? k???? ??p??l ????belsk?? ??dy
>
> '\n' is replaced by space - it is ok

This clearly wrong because you've lost the mapping.


> '\e' is replaced by '?' or by '.'
> "czech characters" are sometimes replaced by '?'
>
> "Problem" I mentioned is how to copy this behavior without exact
> documentation.

How is this solved everywhere else? Its not a process58 issue as long as you
account for the NUL delimited args.

-Steve

To unsubscribe, send an email message to [hidden email] with
SIGNOFF OVAL-DEVELOPER-LIST
in the BODY of the message.  If you have difficulties, write to [hidden email].