CBS

• Rules for identifying known penetrations or penetrations that would exploit known weaknesses
• Rules can also be defined that identify suspicious behavior, even when the behavior is within the bounds of established patterns of usage
• Specific to the machine and OS
• Fruitful approach for developing these rules is to analyze attack toolkits
• SNORT is an example of a rule-based NIDS

• Statistical
    • Analysis of the observed behavior using univariate, multivariate, or time-series models of observed metrics
• Knowledge based
    • Approaches use an expert system that classifies observed behavior according to a set of rules that model legitimate behavior
• Machine learning
    • Approaches automatically determine a suitable classification model from the training data using data mining techniques

• Signature-based detection
    + Can quickly and efficiently identify known attacks
    + Low false positive (false alarms)
    - Expensive (need up-to-date signatures)
    - Cannot find new attacks (e.g., zero-day attacks)
• Anomaly-based detection
    + Can detect unknown, zero-day attacks
    - High false positive (false alarms)
    - Training on legitimate/benign data only, reducing the effectiveness of such approaches

achine learning [...] gives "computers the ability to learn without being
explicitly programmed" [and] [...] explores the study and construction of
algorithms that can learn from and make predictions on data – such
algorithms overcome following strictly static program instructions by making data-
driven predictions or decisions, through building a model from sample inputs.

Yes (even cars)

Supervised
    • Labeled data
    • Direct feedback
    • Predict outcome/future
    • Example: estimate a house price

Unsupervised
    • No labels
    • No feedback
    • Find hidden structures
    • Example: customer segmentation

Reinforcement
    • Decision process
    • Reward system
    • Learn series of actions
    • Example: learning to play a board game (e.g., chess)

Image

• Confidentiality of the model itself (e.g., intellectual property)
• Privacy of the training or test data (e.g., medical records)

• Integrity of the predictions (w.r.t. expected outcome)
• Availability of the system deploying machine learning

• Black-box attacks
    • Model architecture and parameters unknown
    • Can only interact blindly with the model
• Grey-box attacks
    • Model architecture known, parameters unknown
    • Can only interact with the model, but has information about the type of model
• White-box attacks
    • Known model architecture and parameters
    • Can replicate the model and use the model’s internal parameters in the attac

Bottom-up: More threatening
Up-bottom: Higher success rates

• Evasion attacks
    • Craft adversarial data that prevent an ML model from correctly identifying the contents of the data
    • E.g., to evade ML-based virus/malware detection
• Poisoning attacks
    • Poison the dataset used to train an ML model by modifying the underlying data or its labels (allowing the adversary to embed vulnerabilities in the ML model)
• Backdoor attacks
    • Operates as expected under typical conditions, but produces the adversary’s desired output when a trigger is used as input data
• Denial of ML service attacks
    • Target ML systems with a flood of requests for the purpose of degrading or shutting down the service

• Membership inference attacks
    • Try to determine whether an input sample was part of the training set
    • Most popular category of privacy attacks
• Reconstruction attacks
    • Try to recreate one or more training samples (and/or their labels)
    • Also known as attribute inference or model inversion
• Property inference attacks
    • Try to extract dataset properties which were not explicitly encoded as features or were not correlated to the learning task
    • E.g., ratio of women and men in a patient dataset
• Model stealing (or model extraction) attacks
    • Extract information and potentially fully reconstruct an ML model

Goal: “steal” the expensive model by observing its outputs, with less cost than obtaining the data and training the model

• Threatens the confidentiality of the model itself
    • Intellectual property
    • Also called model extraction
• Good ML models require considerable investment
    • Collecting data takes time and money
    • Training infrastructure is expensive

When used for classification, a linear model employs a linear function (separating hyperplane) to produce a decision
• E.g., SVM with linear kernel
• If f(x) > t, the output class is 1, otherwise it is 0

For two-dimensional x, we only need three queries!

In general, if a linear model uses d features, the adversary needs d+1 different queries to steal the model
• Solving the linear system for w, b:

\(w*x^i+b = f(x^i)\)

RETRAINING ATTACK
Idea: observe many queries Z = (x, f(x)), and fit the model on Z like on any other training data
• It takes many queries: for a neural network with 2,000 parameters, we need 11,000 queries to get 99.9% similarity between the models

• Output perturbation [1]
    • Add noise to the probabilities output by the model to hinder reconstruction, but not accuracy
• Detect suspicious queries [2]
    • Identify deviations from expected behavior on distribution of successive queries from a client

Privacy Threats
– Collection of sensitive personal data
– Re-identification
– Inference attacks

– Do trained models leak sensitive information?
– Can we train good privacy-preserving models?

Training: Information leaks about data used for training ?
Testing: Testing data (or the result) might be sensitive in MLaaS

Train a model to…recognize the difference … without knowing the parameters of the actual model!

Exmaple:

Input from the training set
then
Input not from the training set

Train a model to detect

Overfitting

Privacy: Does the model leak information about data in the training set ? 
Learning: Does the model generalize to data outside the treaning set ? 

Overfitting is the common enemy (Privacy and Learning)
    • Overfitted models leak training data
    • Overfitted models lack predictive power
• Need generalizability and accuracy

• ML needs (a lot of) data to learn!
• Machine learning is based on data to
    • Find features
    • Train the model
• Data is highly unique → allows many inferences
    • Anonymizing may not work…
    • Aggregation affects utility and requires careful evaluation
• Hide data
    • Noise → differential privacy
    • Encryption → homomorphic encryption, secure multiparty computation, …

• To obtain value, you must give data!
• To use the ML model, you need to provide a sample…
    • If the model is remote (e.g., MLaaS), you are giving this sample out
• We can do privacy-preserving model evaluation:
    • Predict on noisy data: affects utility
    • Use advanced cryptography: affects performance

• The output reveals information!
• Membership inference
    • Given the answer of the classifier, infer whether the queried example was used in training
• Attribute inference
    • Given the answer of the classifier, infer whether a training sample had a particular attribute

• Coined by Szegedy et al. in 2014 [1] with respect to computer vision but
goes back further to work on spam and malware evasion by Srndic &
Laskov [2], Biggio et al. [3], and Dalvi et al. [4] in 2013

Inputs to a model that an attacker has designed to cause the model to make a mistake
• Set of techniques that aim to fool ML model

• Evasion attacks
    • Most prevalent type of attacks
    • Perturbation of test data, no influence over training data
    • E.g., spammers trying to evade detection by obfuscating the content of spam e-mails
        • Samples are modified to be classified as legitimate
        • E.g., image-based spam: spam content embedded within an attached image to evade textual analysis by anti-spam filters
• Poisoning attacks
    • Adversarial contamination of training data
    • ML systems can be re-trained using data collected during operations
        • E.g., intrusion detection systems (IDSs) often re-trained using such data
        • An attacker may poison this data by injecting malicious samples during operation that subsequently disrupt re-training

elf-driving cars:
Stop sign → speed-limit sign with scotch

Objective: find model parameters that minimize empirical loss

image