In this blog,we are going to discuss about how ML is integrated with some of the industrial use-cases to solve the issues may be threats related to security mostly.
So lets understand first ,
What is Cyber Security ?
💻Cyber security refers to the body of technologies, processes, and practices designed to protect networks, devices, programs, and data from attack, damage, or unauthorized access. Cyber security may also be referred to as information technology security.
❗ THE IMPORTANCE OF CYBER SECURITY
Cyber security is important because government, military, corporate, financial, and medical organizations collect, process, and store unprecedented amounts of data on computers and other devices. A significant portion of that data can be sensitive information, whether that be intellectual property, financial data, personal information, or other types of data for which unauthorized access or exposure could have negative consequences. Organizations transmit sensitive data across networks and to other devices in the course of doing businesses, and cyber security describes the discipline dedicated to protecting that information and the systems used to process or store it.
As the volume and sophistication of cyber attacks grow, companies and organizations, especially those that are tasked with safeguarding information relating to national security, health, or financial records, need to take steps to protect their sensitive business and personnel information. As early as March 2013, the nation’s top intelligence officials cautioned that cyber attacks and digital spying are the top threat to national security, eclipsing even terrorism.
How ML is assisting to detect cyber-threats ?
In principle, machine learning can help businesses better analyze threats and respond to attacks and security incidents. It could also help to automate more menial tasks previously carried out by stretched and sometimes under-skilled security teams.
Subsequently, machine learning in security is a fast-growing trend. Analysts at ABI Research estimate that machine learning in cybersecurity will boost spending in big data, artificial intelligence (AI) and analytics to $96 billion by 2021, while some of the world’s technology giants are already taking a stand to better protect their own customers.
Google is using machine learning to analyze threats against mobile endpoints running on Android — as well as identifying and removing malware from infected handsets, while cloud infrastructure giant Amazon has acquired start-up harvest.AI and launched Macie, a service that uses machine learning to uncover, sort and classify data stored on the S3 cloud storage service.
Simultaneously, enterprise security vendors have been working towards incorporating machine learning into new and old products, largely in a bid to improve malware detection. “Most of the major companies in security have moved from a purely “signature-based” system of a few years ago used to detect malware, to a machine learning system that tries to interpret actions and events and learns from a variety of sources what is safe and what is not,” says Jack Gold, president and principal analyst at J. Gold Associates. “It’s still a nascent field, but it is clearly the way to go in the future. Artificial intelligence and machine learning will dramatically change how security is done.”
Though this transformation won’t happen overnight, machine learning is already emerging in certain areas. “AI — as a wider definition which includes machine learning and deep learning — is in its early phase of empowering cyber defense where we mostly see the obvious use cases of identifying patterns of malicious activities whether on the endpoint, network, fraud or at the SIEM,” says Dudu Mimran, CTO of Deutsche Telekom Innovation Laboratories (and also of the Cyber Security Research Center at Israel’s Ben-Gurion University). “I believe we will see more and more use cases, in the areas of defense against service disruptions, attribution and user behavior modification.”
Confusion matrix
Confusion matrix is a very popular measure used while solving classification problems. It can be applied to binary classification as well as for multiclass classification problems. An example of a confusion matrix for binary classification is shown below
Confusion matrices represent counts from predicted and actual values. The output “TN” stands for True Negative which shows the number of negative examples classified accurately. Similarly, “TP” stands for True Positive which indicates the number of positive examples classified accurately. The term “FP” shows False Positive value, i.e., the number of actual negative examples classified as positive; and “FN” means a False Negative value which is the number of actual positive examples classified as negative. One of the most commonly used metrics while performing classification is accuracy. The accuracy of a model (through a confusion matrix) is calculated using the given formula below.
Accuracy=TN+TP/TN+FP+FN+TP
Accuracy can be misleading if used with imbalanced datasets, and therefore there are other metrics based on confusion matrix which can be useful for evaluating performance. In Python, confusion matrix can be obtained using “confusion_matrix()” function which is a part of “sklearn” library [17]. This function can be imported into Python using “from sklearn.metrics import confusion_matrix.” To obtain confusion matrix, users need to provide actual values and predicted values to the function.
KDD CUP 1999 :
KDD CUP 1999 is an extremely large dataset that used with intrusion detection experiments, the complete dataset has approximately 4.5 million records. This dataset has 41 features that can be classified into three main Categories: TCP connections features, Content features, and Traffic features.
KDD CUP 99 dataset has not noise or missing value so it is a clear dataset, but it has numerical and Text values, the numerical value having large numbers, it will delay the training and complicate the processing. In addition, the text values cannot be processed in the operations of the deep neural network algorithm.
Hence, the dataset must be preprocessed. The preprocessing in this model can be classified into two main steps: the normalization process and text mapping. Z-score normalization stated in Equation 1 has been used for normalizing the numerical attributes to lessen its values and reduce the training processes (time and space).
In another way, Converting the text attributes to numerical values have been executed using One Hot encoder easy handling with math equations.
Binary Classification(Normal and Attack) for testing data performed using the dataset.(Confusion Matrix)
This table means that there are 291903 normal packets detected true as normal from 292011 normal packets, and the error obtains on 108 packets, they detected as an attack, same case in attacks, there are 1177519 packets in testing data was an attack, detected 1177312 as an attack, and the error obtained on 207 packets.
Confusion matrix for training data:
Lets discuss the use-case:
37% of cyberattacks are discovered on servers, making them the most likely place to identify an attack in an organization. That’s one of the alarming stats taken from a recent Sophos survey of 2,700 IT managers around the world.
But why are servers such tempting targets for hackers?
1. Servers are high value
Servers often contain an organization’s most valuable data. For example, personally identifiable information (PII) such as employee and customer records could be stolen if they’re not adequately secured (for example, with encryption) on the server.
Regulations, such as the recently introduced GDPR that protects EU citizens’ data, levy significant fines for non-compliance. Attackers know this and will threaten to release sensitive data if their demands are not met.
2. Server downtime is costly
Servers are the lifeblood of organizations and are critical to their day-to-day functions. Unexpected downtime can seriously impact productivity by removing access to important files or communication tools such as Microsoft Teams or Skype. Ransomware attacks can cause organizations to grind to a halt unless a ransom is paid.
In instances where an organization is reliant on servers for commercial function (e.g. an e-commerce site) downtime can be even more severe.
3. Servers are the perfect staging ground
Servers are usually well connected in an organization’s network. They are also online and running 24/7, which makes them an ideal platform for launching further attacks and performing reconnaissance looking for weak spots to exploit across the network. If you can’t identify a compromised server, the gates to your IT kingdom could be wide open.
So what needs to be done to secure your organization’s servers? The answer is in the right combination of advanced protection, visibility with powerful tools like Endpoint Detection and Response (EDR) and server specific features such as File Integrity Monitoring.
Sophos Intercept X for Server has them all, keeping your organization secure against advanced threats including ransomware and exploits. It gives you the tools you need to hunt down evasive threats and it locks down your servers so they can’t be tampered with.
MICROSOFT :
Microsoft uses its own cybersecurity platform, Windows Defender Advanced Threat Protection (ATP), for preventative protection, breach detection, automated investigation and response. Windows Defender ATP IS built into Windows 10 devices, automatically updates and employs cloud AI and multiple levels of machine learning algorithms to spot threats.
In Windows Defender AV’s layered approach to defense, if the first layer doesn’t detect a threat, we move on to the next level of inspection. As we move down the layers, the amount of time required increases. However, we catch the vast majority of malware at the first (fastest) protection layers and only need to move on to a more sophisticated (but slower) level of inspection for rarer/more advanced threats.
For example, the vast majority of scanned objects are evaluated by the local Windows Defender client machine learning models, behavior-based detection algorithms, generic and heuristic classifications, and more. This helps ensure that users get the best possible performance. In rare cases where local intelligence can’t reach a definitive verdict, Windows Defender AV will use the cloud for deeper analysis.
Human analysts are extremely capable of carving out heuristics that alert on breach activities based on their expertise. However, an analyst can consider only a limited set of signals when creating heuristic rules. By taking into account thousands of signals, ML can slice through data more precisely while being guided by manually created heuristics. Based on our analysis of actual alerts, our ML technologies are at least 20% more precise than manually crafted heuristics.
Machine learning technologies are also able to operate with more generic artifacts. As a result, ML technologies can generalize from various shades of data to detect new and previously unseen threats. Our ML models optimize the use of the vast amounts of data and computational resources available to Windows Defender ATP.
Conclusion : New machine learning and AI techniques, in combination with both static and dynamic analysis, gives the ability to block more and more malware threats at first sight and, if that fails, learn as quickly as possible that something is bad and start blocking it. Using a layered approach, with different ML models at each layer, gives us the ability to target a wide variety of threats quickly while maintaining low false positive rates.
References :
- Network intrusion detection system using deep neural networks
Mohammed Maithem and Ghadaa A. Al-sultany 2021 J. Phys.: Conf
https://iopscience.iop.org/article/10.1088/1742-6596/1804/1/012138/pdf - Windows Defender Antivirus and layered machine learning defenses
Microsoft Defender Security Research Team
https://www.microsoft.com/security/blog/2017/12/11/detonating-a-bad-rabbit-windows-defender-antivirus-and-layered-machine-learning-defenses/
Happy reading 🤗🤗