Tuesday, June 26, 2018

Attacking Machine Learning Detectors: the state of the art review

Machine learning (ML) is a great approach to detect Malware. It is widely used among technical community and scientific community with two different perspectives: Performance V.S Robustness. The technical community tries to improve ML performances in order to increase the usability on large scale while scientific community is focusing on robustness by meaning how easy it would be to attack a ML detector engine. Today I'd like to focus our attention a little bit on the second perspective pointing up how to attack ML detector engines.

We might start by classifying machine learning attacks in three main sets:

  1. Direct Gradient-Based Attack. The attacker needs to know the ML Model. The attacker needs to know model structure and model weights in order to make direct queries to the Machine Learning Model and figure out what is the best way to evade the it.
  2. Score Model Attack. This attack set is based on the score systems. The attacker does not know the Machine Learning Model nor its own weights but he has direct access to the detector engine so that he can probe the machine learning model. The model will return a score and based on such a score, the attacker would be able to guess how to minimise it by forcing specific and crafted inputs.
  3. Binary Black Box Attack.  The attacker has no idea about the Machine Learning Model and the applied Weights, he has also no idea about the scoring system but he have unlimited access to probe the Machine Learning Model. 
Direct Gradient-Based Attack

Direct gradient based attack could be implemented in at least two ways. A first and most used way, is to apply small changes to the original sample in order to reduce the given score. The changes must be limited to a specific domain, for example: valid Windows PE file or  valid PDF files, and so forth. The changes must be little and they should be generated in order to minimise a scoring function derived by weights (which are know fro Direct Gradient-Based Attack). A second way is to connect the targeted model (the mode which is under attack) to a generator model in a generative adversarial network (GAN). Unlike the previous set, the  GAN generator learns how to generate a complete new sample derived by a given seed able to minimise the scoring function. 

I.Goodfellow et Al. in their work "Explaining and Harnessing Adversial Examples" (here) showed how little changes targeted to minimise the resulting weights on a given sample X would be effective in ML evasion. Another great work is written by K.Grosse et Al. titles: "Adversial Perturbations against deep neural networks for malware classification" (here). The authors attacked a deep learning Android malware model, based on DREBIN Android Malware data set, by apply a imperceptible perturbation on the feature vector. They had very interesting results getting from 50% to 84% of evasion rate.   I.Goodfellow et Al. in their work titled "Generative Adversial Nets" (here) developed a GAN able to iterate a series of adversarial rounds to generate samples that were classified as "ham" from the targeted model but that really were not. The following image shows a generative adversarial nets are trained by simultaneously updating the discriminative distribution (D, blue, dashed line) so that it discriminates between samples from the data generating distribution (black,dotted line) px from those of the generative distribution pg (G) (green, solid line).

Image from: "Generative Adversial Nets"

Score Model Attack

The attacker posture on that attack set is considered as "myope". The attacker does not know exactly how the ML model works and he has no idea about how the weights changes inside the ML algorithm but he has the chances to test his sample and getting back a score so that he is able to measure the effect of the input perturbation.

W. Xu, Y. Qi and D. Evans in their work titled: "Automatically evading classifiers" (here) implemented a "fitness function" which gives a fitness score of each generated variant. A variant with a positive fitness score is evasive. The fitness score holds the logic behind the targeted model classified as benign the current sample but retains a malicious behaviour. Once the sample gets high fitness score it is used a seed into a more general genetic algorithm which starts to manipulate the seed in order to make different species. To assure that those mutations preserve the desired malicious behaviour according to the original seed the authors used an oracle. In that case they used cuckoo sandbox.

Image from: "Automatically evading classifiers"

After one week of execution the genetic algorithm found nearly more then 15k evasive variants from 500 circa malicious seeds, getting the 100% of evasion rate on PDFrate classifier.

Binary black-box attacks

Binary black-box attacks are the most general one since attacker does not know anything about the used model and the anti malware engine just says: True or False (it's a Malware or it is not a Malware). In 2017 W.Hu and Y.Tan made a great work described in "Generating Adversarial Malware Examples for Malware Classification" (here). The authors developed MalGAN an Adversial Malware generator able to generate valid PE Malware to evade static black-box PE malware engine. The idea behind MalGAN is simple. First the attacker maps the Black-Box outputs by providing specific and Known Samples (Malware and Good PE). After the mapping phase the attacker builds a Model that behaves as the black-box Model. It is a simple Model trained to behave as the targeted one. Then the built Model is used as target model in a gradient computation GAN to produce evasive Malware. The authors reported 100% efficacy in bypassing the target Model. H. S. Anderson et Al. in "Evading Machine Learning Malware Detection" (here) adopted a Reinforced Learning Approach. The following image shows the Markov decision process formulation of the malware evasion reinforcement learning problem.

Image from: Evading Machine Learning Malware Detection

The agent is the function who manipulate the sample depending on the environment state. Both the reward and a the state are used as input from the agent in order to get decisions on next actions. The agent learns by the reward which depends about the reached state. For example the reward could be higher if the reached state is close to the desired one or vice-versa. The authors use a Q-Learning technique in order to underestimate a negative reward given for an action which would be significant in medium long term.

"In our framework, the actions space A consists of a set of modifications to the PE file that (a) don’t break the PE file format, and (b) don’t alter the intended functionality of the malware sample. The reward function is measured by the anti-malware engine, which is converted to a reward: 0 if the modified malware sample is judged to be benign, and 1 if it is deemed to be malicious. The reward and state are then fed back into the agent."

Final Considerations

Machine Learning, but more generally speaking Artificial Intelligence, would be useful to detect Cyber Attacks but unfortunately - as widely proved on this post - it would not be enough per se. Attackers would use the same techniques such as Adversarial Machine learning  to evade Machine Learning detectors. Cyber Security Analysts would still play a fundamental role in Cyber Security Science and Technology for many years from now. A technology who promises to assure cyber security protection without human interaction is not going to work.

Friday, June 8, 2018

DMOSK Malware Targeting Italian Companies

Today I'd like to share another interesting analysis made by my colleagues and I. It would be a nice and interesting analysis since it targeted many Italian and European companies. Fortunately the attacker forgot the LOG.TXT freely available on the dropping URL letting us know the IP addresses who clicked on the first stage analysed stage (yes, we know the companies who might be infected) . Despite what we did with TaxOlolo we will not disclose the victims IP addresses and so the companies which might be infected. National CERTs have been involved and they've got alerted.  Since we believe the threat could radically increase its magnitude in the following hours, we decided to write up this quick'n dirty analysis focusing on speed rather than on details. So please forgive some quick and undocumented steps.

Everything started from an eMail (how about that ?!). The eMail we've got had the following body.

Attack Path
A simple link to a drive ( drive.carlsongracieanaheim.com ) is beginning our first stage of infection. An eMail address is given as one parameter to the doc.php script which would record the IP address and the "calling" email  address belonging to the victim. The script forces the browser to download a .zip file which uncompressed presents to the victim a JSE file called: scan.jse.  The file is hard obfuscated. It was quite difficult to be able to decode the following stage of infection since the JavaScript was obfuscated through, at least, 3 different techniques. The following image shows the Obfuscated sample.

Second Stage: Obfuscated JSE
Unfortunately the second stage is not the final one. Indeed once de-obfuscated it we figured out that it was dropping and executing another file having the .SCR mimetype. From this stage it's interesting to observe that only one dropping URL was called. It's a strange behaviour, usually the attackers use multiple dropping URLs in order to get more chances to infect the victims. The found URL was the following one:

"url": "https://drive.carlsongracieanaheim.com/x/gate.php"

The JSE file dropped the Third Stage into \User\User\AppData\Local\Temp\38781520.scr having the following  hash: 77ad9ce32628d213eacf56faebd9b7f53e6e33a1a313b11814265216ca2c4745 which has been previously analysed by 68 AV but only 9 of them recognised as malicious generic file. The following image shows the VirusTotal analysis.

Third Stage: Executable SCR file

Unfortunately we are still not at the end of the infection Stage. The Third stage drops and executes another payload. It does not download and execute from a different dropping website but it drops from a special and crafted memory address (fixed from .txt:0x400000). The following image shows the execution of the Fourth Stage payload directly from the victim's memory

Fourth Stage: Dropped PE File
Following the analysis it has been possible to figure out that the final payload is something very close to ursnif which grabs victims email information and credentials. The following image shows the temporary file built before sending out information to Command and Controls servers.

Temporary File Before Sending data to Command and Control

Like any other ursnif the malware tries to reach a command and control network located both on the clearnet and on the TOR network. A following section will expose the recorded IoCs.

An interesting approach that was adopted by attackers is the black listing. We observed at least 3 black lists. The first one was based on victims IP. We guess (but we have not evidences on that) that the attacker would filtering responses based on Country in order to make possible a country targeted attack by blacklisting not-targeted countries. The following image shows the used temporary file to store Victim IP. The attacker could use this information in order to respond or not to a specific malware request.

Temporary File Storing IP Victim IP Address

A second black list that we found was on the dropping URL web site which was trained to do not drop files to specific IP addresses. The main reasons found to deny the dropping payload were three:
  • geo (Out of geographical scope). The threat is mainly focused to hit italy.
  • asn (internet service providers and/or cloud providers). The threat is mainly focused on clients and not on servers, so it would have no sense to give payload to cloud providers.
  • MIT. THe attacker does not want the dropping payload ends up to MIT folks, this is quite funny, isn't it ?
A small section of black listing drop payload  

The black lists are an interesting approach to reduce the chance to be analysed, in fact the black listed IPs belong to pretty known CyberSecurity Companies (Yoroi is included) which often use specific cloud providers to run emulations and/or sandboxes. 

Personal note: This is a reverse targeting attack, where the attacker wants to attack an entire set of victims but not some specific ones, so it introduces a blocking delivery of payload technique. End personal note.

Now we know how the attack works, so lets try to investigate a little bit what the attacker messed out. For example lets try to analyse the content of the Dropping URL. Quite fun to figure out the attacker let freely available his private key ! I will not disclose it .... let's say... for respect to the attacker (? really ?) 

Attacker Private Key !

While the used public certificate is the following one:

Attacker Certificate

By decoding the fake certificate the analyst would take the following information, of course none of these informations would be valuable, but make a nice shake of analysis .

Common Name: test.dmosk.local
Organization: Global Security
Organization Unit: IT Department
Locality: SPb
State: SPb
Country: RU
Valid From: June 5, 2018
Valid To: June 5, 2022
Issuer: Global Security
Serial Number: 12542837396936657430 (0xae111c285fe50a16

Maybe the most "original string", by meaning of being written without thinking too much from the attacker, on the entire malware analysis would be the string  "dmosk" (in the decoded certificate), from here the Malware name.

As today we observed: 6617 eMail addresses that potentially could be compromised since they clicked on First stage (evidences on dropping url). We have evidences that many organisations have been hit from this malware able to bypass most of the known security protections since it was behind CloudFlare and with not a specific bad reputation. We decided to not disclose the "probably infected" companies. Nation Wide CERTs have been alerted (June 7 2018) and together we will contact the "probably infected" companies to help them to mitigate the threat. 

Please update your rules, signature and whatever you have to block the infection.

PS: the threat is quite a bit bigger than what I described, there are several additional components including APK (Android Malware), base ciphers, multi stage obfuscators and a complete list of "probably infected" users, but again, we decided to encourage the notification speed rather than analysis details. 

Hope you might find it helpful.

  • Dropurl:
    • https:// drive[.carlsongracieanaheim[.com/doc.php
    • https:// drive[.carlsongracieanaheim[.com/doc1.php
    • https:// drive[.carlsongracieanaheim[.com/x/gate.php
    • https:// drive[.carlsongracieanaheim[.com/1/gate.php
  • C2 (tor):
    • https:// 4fsq3wnmms6xqybt[.onion/wpapi
    • https:// em2eddryi6ptkcnh[.onion/wpapi
    • https:// nap7zb4gtnzwmxsv[.onion/wpapi
    • https:// t7yz3cihrrzalznq[.onion/wpapi
  • C2:
    • https:// loop.evama.[at/wpapi
    • https:// torafy[.cn/wpapi
    • https:// u55.evama[.at/wpapi
    • https:// yraco[.cn/wpapi
    • https:// inc.robatop.[at/wpapi
    • https:// poi.robatop.[at/wpapi
    • https:// arh.mobipot.[at/wpapi
    • https:// bbb.mobipot.[at/wpapi
    • https:// takhak.[at/wpapi
    • https:// kerions.[at/wpapi
    • https:// j11.evama[.at/wpapi
    • https:// clocktop[.at/wpapi
    • https:// harent.[cn/wpapi
  • Hash:
    • 067b39632f093821852889b1e4bb8b2a48afd94d1e348702a608a70bb7b00e54 zip
    • 77ad9ce32628d213eacf56faebd9b7f53e6e33a1a313b11814265216ca2c4745 jse
    • 8d3d37c9139641e817bcf0fad8550d869b9f68bc689dbbf4b4d3eb2aaa3cf361 scr
    • 1fdc0b08ad6afe61bbc2f054b205b2aab8416c48d87f2dcebb2073a8d92caf8d exe
    • afd98dde72881d6716270eb13b3fdad2d2863db110fc2b314424b88d85cd8e79 exe
  • Cert: