CBS

• ML needs (a lot of) data to learn!
• Machine learning is based on data to
    • Find features
    • Train the model
• Data is highly unique → allows many inferences
    • Anonymizing may not work…
    • Aggregation affects utility and requires careful evaluation
• Hide data
    • Noise → differential privacy
    • Encryption → homomorphic encryption, secure multiparty computation, …

• To obtain value, you must give data!
• To use the ML model, you need to provide a sample…
    • If the model is remote (e.g., MLaaS), you are giving this sample out
• We can do privacy-preserving model evaluation:
    • Predict on noisy data: affects utility
    • Use advanced cryptography: affects performance

• The output reveals information!
• Membership inference
    • Given the answer of the classifier, infer whether the queried example was used in training
• Attribute inference
    • Given the answer of the classifier, infer whether a training sample had a particular attribute

• Coined by Szegedy et al. in 2014 [1] with respect to computer vision but
goes back further to work on spam and malware evasion by Srndic &
Laskov [2], Biggio et al. [3], and Dalvi et al. [4] in 2013

Inputs to a model that an attacker has designed to cause the model to make a mistake
• Set of techniques that aim to fool ML model

• Evasion attacks
    • Most prevalent type of attacks
    • Perturbation of test data, no influence over training data
    • E.g., spammers trying to evade detection by obfuscating the content of spam e-mails
        • Samples are modified to be classified as legitimate
        • E.g., image-based spam: spam content embedded within an attached image to evade textual analysis by anti-spam filters
• Poisoning attacks
    • Adversarial contamination of training data
    • ML systems can be re-trained using data collected during operations
        • E.g., intrusion detection systems (IDSs) often re-trained using such data
        • An attacker may poison this data by injecting malicious samples during operation that subsequently disrupt re-training

elf-driving cars:
Stop sign → speed-limit sign with scotch

Objective: find model parameters that minimize empirical loss

image

Adversary’s objective: find a perturbation that maximizes the loss

One approach: Fast Gradient Sign Method (FGSM

The similarity relation is often represented as adversarial cost constraint.
If the goal is to be imperceptible, the common cost is a norm of the perturbation

• Constraint on perturbation norm assumes that similar images are close in, e.g., Euclidean distance…
    • But this is not always true! [1]
• For example, similarity could be defined as small affine transformations [2] :

We can only defend against a particular threat model (e.g., perturbations up to epsilon norm), and normally no guarantees

Training on simulated adversarial examples, i.e., include adversarial examples in the training set

Adversarial samples are not restricted to computer vision
    • Twitter bot detection [1]
        • Detection tools can easily be fooled by tweaking the number of replies or retweets
    • Text classification [2]
    • Audio [3]
       • Misleading an automatic speech recognition neural network with some noise
        • E.g., transcribing "without the dataset the article is useless" into
"okay google, browse to evil.com"

• Data or infrastructure as targets
• Attacks against confidentiality, integrity or availability
• Examples of incident description:
• Man-in-the-Middle attack when data was being transferred through the
public Internet
• Phishing
• People running crypto mining on our infrastructure through our APIs
• Server was compromised
• Botnet communication

• Relation to AI vague (beyond data related)
• Attacks against confidentiality, targeting data, not ML model
• Examples of incident description:
• Patient data should not be shared
• Data privacy
• Incorrect use of temporary storage on AWS lambda led to a potential leak of
raw inference [...] potentially containing PII [...]

• Attacks against integrity
• Incident descriptions:
• ML systems being retrained to provide false outputs
• Client/partner employees [...] feel threatened by automation, and either stall
or sabotage the labeling effort, harming the models

Evasion → 4
• Attacks against integrity
• Incident descriptions:
• Intentional circumventions
• Users spam to optimize their strategy for job search
• A speed-sign as advertisement put onto a driving-school car was interpreted
by the camera as a valid street sign
• Autonomous vehicle image recognition errors leading to dangerous path
planning

Confidentiality
Integrity
Availability
 

Data confidentiality: private or confidential information not made available or
disclosed to unauthorized individuals

Privacy: individuals control or influence what information related to them may be
collected and stored and by whom and to whom that information may be disclosed

Data integrity: information and programs are changed only in a specified and
authorized manner

• System integrity: a system performs its intended function in an unimpaired manner,
free from deliberate or inadvertent unauthorized manipulation of the system

Systems work promptly and service is not denied to authorized users

. Preserving authorized restrictions on information access and disclosure, incl. means for protecting privacy and proprietary information
 

Preventing improper information modification or destruction, incl. ensuring information non-repudiation and authenticity
 

Availability
• Ensuring timely and reliable access to and use of information

unauthorized disclosure of information

unauthorized modification or destruction of information

disruption of access to or use of information (denial of service)

• Authenticity
   • Verifying that users are who they say they are
• Accountability (non-repudiation)
   • Entity cannot deny having done an action
• Anonymity
   • Not being recognizable among a group of people

1. Computer security is not as simple as it might first appear to the novice
2. In developing a particular security mechanism or algorithm, one must always consider potential attacks on those security features
3. Security mechanisms typically involve more than one particular protocol and also require that participants be in possession of some secret information which raises questions about the creation, distribution, and protection of that secret information
4. Attackers only need to find a single weakness, while the designer must find and eliminate all weaknesses to achieve perfect security
5. Security is still too often an afterthought to be incorporated into a system after the design is complete, rather than being an integral part of the design process
6. Security requires regular and constant monitoring
7. There is a natural tendency on the part of users and system managers to perceive little benefit from security investment until a security failure occurs
8. Many users and even security administrators view strong security as an impediment to efficient and user-friendly operation of an information system

• Asset
• Hardware, software, data, communication networks, …
• E.g., a major application, high impact program, physical plant, mission critical system, …

Individual, group, organization, or government that conducts detrimental activities

Any kind of malicious activity that attempts to collect, disrupt, deny, degrade, or destroy information system resources or the information itself

Any event with the potential to adversely impact organizational operations, organizational assets, individuals, other organizations, or the Nation through an information system via unauthorized access, destruction, disclosure, modification of information, and/or denial of service

Technical term that defines the adversary’s capabilities (more about it next)