maec examples?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

maec examples?

Stuart Maclean
Hi from a newbie.

I am looking for real maec data, i.e. actual bundles that conform to the
1.1 schema.  While I have read 10 or so white papers on maec and see
merit in the ideals, I still do not know where to start with actual maec
production.  FWIW I have run Java's jaxb tools over the schema and that
part works great.  So now I can populate an object graph and have the
xml come out automagically. The sticking point is what to actually put
in that object graph.  Even for the Conficker test cases, I can't locate
any maec output.

Any help appreciated.

Stu
Reply | Threaded
Open this post in threaded view
|

RE: maec examples?

Kirillov, Ivan A.
Hi Stuart,

Welcome!

Right now there aren't any publically available repositories of MAEC data.

However, we do have several translator tools from freely available dynamic analysis tools including ThreatExpert and Anubis. These tools have lots of freely available analyses that you can then convert into MAEC.

If you wish to get access to these translators, please let me know, and I'll send you an invite to Handshake, MITRE's social collaboration portal, where they currently reside. I can also send you a few sample MAEC bundles created by these tools if you wish.

Regards,
Ivan Kirillov
MAEC Working Group
The MITRE Corporation

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Stuart Maclean
Sent: Monday, October 24, 2011 3:31 PM
To: maec-discussion-list Malware Attribute Enumeration Discussion
Subject: maec examples?

Hi from a newbie.

I am looking for real maec data, i.e. actual bundles that conform to the
1.1 schema.  While I have read 10 or so white papers on maec and see
merit in the ideals, I still do not know where to start with actual maec
production.  FWIW I have run Java's jaxb tools over the schema and that
part works great.  So now I can populate an object graph and have the
xml come out automagically. The sticking point is what to actually put
in that object graph.  Even for the Conficker test cases, I can't locate
any maec output.

Any help appreciated.

Stu
Reply | Threaded
Open this post in threaded view
|

Re: maec examples?

Wes Young
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

we take in:

malware.com.br
threatexpert
malc0de
zeus and spyeye binary feeds

and a few others:

http://code.google.com/p/collective-intelligence-framework

We currently map (translate/normalize, whatever you want to call it) these malware samples to IODEF and/or ICSG, adding a MAEC plugin (to be used instead of ICSG) probably wouldn't be that hard.

has anyone started in on a perl module?

I currently own XML::Malware, which uses XML::Compile by default. It's setup to accept any sort of schema (ICSG was the only one semi-final at the time).

http://code.google.com/p/perl-xml-malware
http://search.cpan.org/~saxjazman/XML-Malware-0.01/lib/XML/Malware.pm

we also have:

http://code.google.com/p/python-xml-malware

which is a python lib Jose built with some autogen software based on ICSG.

now that MAEC is some what final, we could lump these into the Malware libs as plugins (like we did with ICSG), then drop in some CIF Storage plugins for MAEC for those that want to use that standard.... at run-time you can pick your format.

?


On Oct 25, 2011, at 9:03 AM, Kirillov, Ivan A. wrote:

> However, we do have several translator tools from freely available dynamic analysis tools including ThreatExpert and Anubis. These tools have lots of freely available analyses that you can then convert into MAEC.

- --
Wes
claimid.com/wesyoung
[hidden email]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)

iEYEARECAAYFAk6mufMACgkQKezpZd226UbWsACePxXb3ERj49G4XxH52QmExWWB
RdAAnRev8oB3O32Nb746f3ZkRRT1amjD
=R5wI
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: maec examples?

Stuart Maclean
In reply to this post by Kirillov, Ivan A.
Hi Ivan, thanks for the intro.

I am somewhat confused by the white papers and their relationships to
the actual schema.  The papers refer to the highest level tier being
'traits' or 'mechanisms'.  Persistence is given as an example.

Yet no such construct appears in the schema.  As far as i can see, the
outermost elements (i.e. the ones whose direct parent would be the
bundle itself) are

Analyses
Behaviors
Actions
Pools

There is no place for Mechanisms.

Further, if I have nugget of knowledge A about some malware and I
formulate it in maec, and then later some other knowledge B comes along,
how does maec support the linking of A to B?  Perhaps it does not and
was never intended to.

Any advice appreciated.

Stuart
Reply | Threaded
Open this post in threaded view
|

RE: maec examples?

Kirillov, Ivan A.
Hi Stuart,

The MAEC whitepaper laid out the concepts we hope to implement in MAEC; however, the MAEC schema is still continuously being developed, and as such there are certain constructs, like mechanisms, that we have not incorporated yet. Our next major goal after the release of MAEC 2.0 is to refine the behavior type to ensure that it is flexible enough to cover the various types of high-level behavioral data; after this, we will focus on mechanisms.

In terms of linking new data, one of the features of the MAEC schema is that almost all elements are optional, so that you can add data as you acquire it. For instance, you can create a MAEC bundle for a particular malware instance and then add information on behaviors that were observed through further analysis. Of course it may be useful to have more explicit linkages for added data, so that you can link between different MAEC bundles, and also incorporate addition-specific attributes, such as the date/time a piece of data was added. We have not explored this concept in great detail yet but are certainly willing to do so. Do you have any particular examples or needs for linking up data?

Anyways, clearly there is still much we can do, so we appreciate all input and feedback :)

Regards,
Ivan

-----Original Message-----
From: Stuart Maclean [mailto:[hidden email]]
Sent: Tuesday, October 25, 2011 1:02 PM
To: Kirillov, Ivan A.
Cc: maec-discussion-list Malware Attribute Enumeration Discussion
Subject: Re: maec examples?

Hi Ivan, thanks for the intro.

I am somewhat confused by the white papers and their relationships to
the actual schema.  The papers refer to the highest level tier being
'traits' or 'mechanisms'.  Persistence is given as an example.

Yet no such construct appears in the schema.  As far as i can see, the
outermost elements (i.e. the ones whose direct parent would be the
bundle itself) are

Analyses
Behaviors
Actions
Pools

There is no place for Mechanisms.

Further, if I have nugget of knowledge A about some malware and I
formulate it in maec, and then later some other knowledge B comes along,
how does maec support the linking of A to B?  Perhaps it does not and
was never intended to.

Any advice appreciated.

Stuart
Reply | Threaded
Open this post in threaded view
|

RE: maec examples?

Kirillov, Ivan A.
In reply to this post by Wes Young
Wes,

Ah, very interesting. I need to play around with the collective intelligence framework.

I don't believe anyone has started on a MAEC perl module, but I'm sure we could come up with something. Right now everything we have is in Python; generateDS gives us Python data bindings essentially for free.

It would be pretty cool if you could add a MAEC plugin into CIF. I think once MAEC 2.0 comes out we should discuss this further.

Regards,
Ivan

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Wes Young
Sent: Tuesday, October 25, 2011 9:30 AM
To: maec-discussion-list Malware Attribute Enumeration Discussion
Subject: Re: maec examples?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

we take in:

malware.com.br
threatexpert
malc0de
zeus and spyeye binary feeds

and a few others:

http://code.google.com/p/collective-intelligence-framework

We currently map (translate/normalize, whatever you want to call it) these malware samples to IODEF and/or ICSG, adding a MAEC plugin (to be used instead of ICSG) probably wouldn't be that hard.

has anyone started in on a perl module?

I currently own XML::Malware, which uses XML::Compile by default. It's setup to accept any sort of schema (ICSG was the only one semi-final at the time).

http://code.google.com/p/perl-xml-malware
http://search.cpan.org/~saxjazman/XML-Malware-0.01/lib/XML/Malware.pm

we also have:

http://code.google.com/p/python-xml-malware

which is a python lib Jose built with some autogen software based on ICSG.

now that MAEC is some what final, we could lump these into the Malware libs as plugins (like we did with ICSG), then drop in some CIF Storage plugins for MAEC for those that want to use that standard.... at run-time you can pick your format.

?


On Oct 25, 2011, at 9:03 AM, Kirillov, Ivan A. wrote:

> However, we do have several translator tools from freely available dynamic analysis tools including ThreatExpert and Anubis. These tools have lots of freely available analyses that you can then convert into MAEC.

- --
Wes
claimid.com/wesyoung
[hidden email]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)

iEYEARECAAYFAk6mufMACgkQKezpZd226UbWsACePxXb3ERj49G4XxH52QmExWWB
RdAAnRev8oB3O32Nb746f3ZkRRT1amjD
=R5wI
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: maec examples?

Wes Young
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Oct 25, 2011, at 1:24 PM, Kirillov, Ivan A. wrote:

> I don't believe anyone has started on a MAEC perl module, but I'm sure we could come up with something. Right now everything we have is in Python; generateDS gives us Python data bindings essentially for free.

ya, that's what we did for ICSG too :)

XML::Compile sort of works the same way, cept it generates it on-the-fly based on the schema, then caches it for faster use "after the first compile".

i'll see about building it into XML::Malware as an option. Should be a no-brainder long as I have the xsd.

> It would be pretty cool if you could add a MAEC plugin into CIF. I think once MAEC 2.0 comes out we should discuss this further.

easy once XML::Malware has the functions. Just a snap-in to the storage stack in CIF::Archive.

- --
Wes
claimid.com/wesyoung
[hidden email]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)

iEYEARECAAYFAk6m8vIACgkQKezpZd226UZEPwCfaUgIH9Qj6rhVH+U9Wl/2/Rjh
47IAniFFMyiTouJA0+33q8fWMRSJfhd5
=c1h1
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: maec examples?

sprabhu
In reply to this post by Wes Young
Hello Wes --

From the list of tools you have mentioned, the below links for
perl-xml-malware and python-xml-malware

http://code.google.com/p/perl-xml-malware/
http://code.google.com/p/python-xml-malware/

are inaccessible,  can you tell me how can I get these tools?
Any other links where I can find these?


-- 
Thanks !!
Prabhu S A



 
On 10/25/2011 07:00 PM, Wes Young wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

we take in:

malware.com.br
threatexpert
malc0de
zeus and spyeye binary feeds

and a few others:

http://code.google.com/p/collective-intelligence-framework

We currently map (translate/normalize, whatever you want to call it) these malware samples to IODEF and/or ICSG, adding a MAEC plugin (to be used instead of ICSG) probably wouldn't be that hard. 

has anyone started in on a perl module?

I currently own XML::Malware, which uses XML::Compile by default. It's setup to accept any sort of schema (ICSG was the only one semi-final at the time).

http://code.google.com/p/perl-xml-malware
http://search.cpan.org/~saxjazman/XML-Malware-0.01/lib/XML/Malware.pm

we also have:

http://code.google.com/p/python-xml-malware

which is a python lib Jose built with some autogen software based on ICSG.

now that MAEC is some what final, we could lump these into the Malware libs as plugins (like we did with ICSG), then drop in some CIF Storage plugins for MAEC for those that want to use that standard.... at run-time you can pick your format.

?


On Oct 25, 2011, at 9:03 AM, Kirillov, Ivan A. wrote:

  
However, we do have several translator tools from freely available dynamic analysis tools including ThreatExpert and Anubis. These tools have lots of freely available analyses that you can then convert into MAEC. 
    
- --
Wes
claimid.com/wesyoung
[hidden email]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)

iEYEARECAAYFAk6mufMACgkQKezpZd226UbWsACePxXb3ERj49G4XxH52QmExWWB
RdAAnRev8oB3O32Nb746f3ZkRRT1amjD
=R5wI
-----END PGP SIGNATURE-----

  
Reply | Threaded
Open this post in threaded view
|

Re: maec examples?

Wes Young
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey,

I've spent the last few days re-factoring/consolidating some of my various repo's (too much cruft in too many places...)

http://code.google.com/p/collective-intelligence-framework/source/browse/#svn%2Fcif-malware

I've moved the perl-xml-malware and python-xml-malware under CIF for now. They'll still be published under CPAN.

On Nov 23, 2011, at 9:04 AM, Prabhu S A wrote:

> From the list of tools you have mentioned, the below links for
> perl-xml-malware and python-xml-malware
>
> http://code.google.com/p/perl-xml-malware/
> http://code.google.com/p/python-xml-malware/
>
> are inaccessible,  can you tell me how can I get these tools?
> Any other links where I can find these?

- --
Wes
claimid.com/wesyoung
[hidden email]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)

iEYEARECAAYFAk7NApUACgkQKezpZd226UbfxACePXdiPcCuu25pYmlHShBkar52
WnkAn2h0yE4zq367k2C+46C0DdCEGfBJ
=Rv8C
-----END PGP SIGNATURE-----