CBS
CBS
CBS
What is the PRIVACY AND ML: PROBLEM #1 ?
• ML needs (a lot of) data to learn!
• Machine learning is based on data to
• Find features
• Train the model
• Data is highly unique → allows many inferences
• Anonymizing may not work…
• Aggregation affects utility and requires careful evaluation
• Hide data
• Noise → differential privacy
• Encryption → homomorphic encryption, secure multiparty computation, …
What is the PRIVACY AND ML: PROBLEM #2 ?
• To obtain value, you must give data!
• To use the ML model, you need to provide a sample…
• If the model is remote (e.g., MLaaS), you are giving this sample out
• We can do privacy-preserving model evaluation:
• Predict on noisy data: affects utility
• Use advanced cryptography: affects performance
What is the PRIVACY AND ML: PROBLEM #3 ?
• The output reveals information!
• Membership inference
• Given the answer of the classifier, infer whether the queried example was used in training
• Attribute inference
• Given the answer of the classifier, infer whether a training sample had a particular attribute
Give a definition of the term adversarial example
• Coined by Szegedy et al. in 2014 [1] with respect to computer vision but
goes back further to work on spam and malware evasion by Srndic &
Laskov [2], Biggio et al. [3], and Dalvi et al. [4] in 2013
What is a Working definition for adversarial example
Inputs to a model that an attacker has designed to cause the model to make a mistake
• Set of techniques that aim to fool ML model
What are the 2 ADVERSARIAL ML ATTACKS ?
• Evasion attacks
• Most prevalent type of attacks
• Perturbation of test data, no influence over training data
• E.g., spammers trying to evade detection by obfuscating the content of spam e-mails
• Samples are modified to be classified as legitimate
• E.g., image-based spam: spam content embedded within an attached image to evade textual analysis by anti-spam filters
• Poisoning attacks
• Adversarial contamination of training data
• ML systems can be re-trained using data collected during operations
• E.g., intrusion detection systems (IDSs) often re-trained using such data
• An attacker may poison this data by injecting malicious samples during operation that subsequently disrupt re-training
Give some example
elf-driving cars:
Stop sign → speed-limit sign with scotch
What is the Objective of MACHINE LEARNING 101 ?
Objective: find model parameters that minimize empirical loss
In MACHINE LEARNING 101 How do we find those parameters/weights?
image
What is the Adversary’s objective ?
Adversary’s objective: find a perturbation that maximizes the loss
HOW TO SOLVE THE OPTIMIZATION PROBLEM?
One approach: Fast Gradient Sign Method (FGSM
HOW TO DEFINE SIMILARITY?
The similarity relation is often represented as adversarial cost constraint.
If the goal is to be imperceptible, the common cost is a norm of the perturbation
OTHER SIMILARITY RELATIONS?
• Constraint on perturbation norm assumes that similar images are close in, e.g., Euclidean distance…
• But this is not always true! [1]
• For example, similarity could be defined as small affine transformations [2] :
Why Defending in general is hard AGAINST ADVERSARIAL EXAMPLES ?
We can only defend against a particular threat model (e.g., perturbations up to epsilon norm), and normally no guarantees
What is the Standard approachfor DEFENDING AGAINST ADVERSARIAL EXAMPLES
Training on simulated adversarial examples, i.e., include adversarial examples in the training set
Is Adversarial samples restricted to computer vision ? Give 3 examples
Adversarial samples are not restricted to computer vision
• Twitter bot detection [1]
• Detection tools can easily be fooled by tweaking the number of replies or retweets
• Text classification [2]
• Audio [3]
• Misleading an automatic speech recognition neural network with some noise
• E.g., transcribing "without the dataset the article is useless" into
"okay google, browse to evil.com"
Give exabple of CYBERSECURITY INCIDENTS
• Data or infrastructure as targets
• Attacks against confidentiality, integrity or availability
• Examples of incident description:
• Man-in-the-Middle attack when data was being transferred through the
public Internet
• Phishing
• People running crypto mining on our infrastructure through our APIs
• Server was compromised
• Botnet communication
Give example of PRIVACY INCIDENTS
• Relation to AI vague (beyond data related)
• Attacks against confidentiality, targeting data, not ML model
• Examples of incident description:
• Patient data should not be shared
• Data privacy
• Incorrect use of temporary storage on AWS lambda led to a potential leak of
raw inference [...] potentially containing PII [...]
Give example of: POISONING INCIDENTS
• Attacks against integrity
• Incident descriptions:
• ML systems being retrained to provide false outputs
• Client/partner employees [...] feel threatened by automation, and either stall
or sabotage the labeling effort, harming the models
Give examples of EVASION INCIDENTS
Evasion → 4
• Attacks against integrity
• Incident descriptions:
• Intentional circumventions
• Users spam to optimize their strategy for job search
• A speed-sign as advertisement put onto a driving-school car was interpreted
by the camera as a valid street sign
• Autonomous vehicle image recognition errors leading to dangerous path
planning
WHAT IS COMPUTER SECURITY?
“Measures and controls that ensure confidentiality, integrity, and availability of information system assets including hardware, software, firmware, and information being processed, stored, and communicated.”
What are the 3 KEY SECURITY OBJECTIVES and explain them ?
Confidentiality
Integrity
Availability
What is Data confidentiality: ?
Data confidentiality: private or confidential information not made available or
disclosed to unauthorized individuals
What is Privacy?
Privacy: individuals control or influence what information related to them may be
collected and stored and by whom and to whom that information may be disclosed
What is Data integrity ?
Data integrity: information and programs are changed only in a specified and
authorized manner
What is system integrity ?
• System integrity: a system performs its intended function in an unimpaired manner,
free from deliberate or inadvertent unauthorized manipulation of the system
What is availability ?
Systems work promptly and service is not denied to authorized users
In terms of requirements and loss security (NIST standard) what i s Confidentiality ?
. Preserving authorized restrictions on information access and disclosure, incl. means for protecting privacy and proprietary information
In terms of requirements and loss security (NIST standard) what is Integrity?
Preventing improper information modification or destruction, incl. ensuring information non-repudiation and authenticity
In terms of requirements and loss security (NIST standard) what is Availability?
Availability
• Ensuring timely and reliable access to and use of information
What is Loss of confidentiality ?
unauthorized disclosure of information
What is Loss of integrity ?
unauthorized modification or destruction of information
What is Loss of availability?
disruption of access to or use of information (denial of service)
What are the 3 OTHER SECURITY OBJECTIVES and explain them
• Authenticity
• Verifying that users are who they say they are
• Accountability (non-repudiation)
• Entity cannot deny having done an action
• Anonymity
• Not being recognizable among a group of people
Give the 8 CHALLENGES IN SECURITY
1. Computer security is not as simple as it might first appear to the novice
2. In developing a particular security mechanism or algorithm, one must always consider potential attacks on those security features
3. Security mechanisms typically involve more than one particular protocol and also require that participants be in possession of some secret information which raises questions about the creation, distribution, and protection of that secret information
4. Attackers only need to find a single weakness, while the designer must find and eliminate all weaknesses to achieve perfect security
5. Security is still too often an afterthought to be incorporated into a system after the design is complete, rather than being an integral part of the design process
6. Security requires regular and constant monitoring
7. There is a natural tendency on the part of users and system managers to perceive little benefit from security investment until a security failure occurs
8. Many users and even security administrators view strong security as an impediment to efficient and user-friendly operation of an information system
What is an Asset ?
• Asset
• Hardware, software, data, communication networks, …
• E.g., a major application, high impact program, physical plant, mission critical system, …
What is an Adversary (or attacker, threat agent)
Individual, group, organization, or government that conducts detrimental activities
What is an Attack
Any kind of malicious activity that attempts to collect, disrupt, deny, degrade, or destroy information system resources or the information itself
What is a Threat ?
Any event with the potential to adversely impact organizational operations, organizational assets, individuals, other organizations, or the Nation through an information system via unauthorized access, destruction, disclosure, modification of information, and/or denial of service
What is aThreat model (or adversary model) ?
Technical term that defines the adversary’s capabilities (more about it next)
-
- 1 / 491
-
Lernkarten
What is an example of a passive attack?
Eavesdropping on online communications.
What is an active attack?
An attack that alters system resources or affects their operation.
What does an active attack usually involve?
Modification of a data stream or creation of a false stream.
What are the four categories of active attacks in network security?
Replay, masquerade, message modification, and denial of service.
What is an example of an active attack?
A DDoS attack against governmental websites.
What is the main difference between passive and active attacks?
Passive attacks only observe; active attacks modify or disrupt.
What are the 8 SECURITY DESIGN PRINCIPLES ?
1. Economy of mechanism
2. Fail-safe defaults
3. Complete mediation
4. Open design
5. Separation of privilege
6. Least privilege
7. Least common mechanism
8. Psychological acceptability
What is ECONOMY OF MECHANISM ? and why to use it ?
Keep the design as simple and small as possible.
Easier to test and verify
More complex mechanism ¨ more likely to have exploitable flaws
E.g., complicated code difficult to review
Updating or replacing a simple mechanism is less intensive
What is FAIL-SAFE DEFAULTS ? and why to use it ?
“ Base access decision on permission rather than exclusion”
• Unless a subject is given explicit access to an object, it should be denied access to that object
• Main design implications
• Default situation: lack of access to the system
• Protection scheme identifies conditions under which access is permitted
• Most file access systems work on this principle and virtually all protected services on client/server systems work this way
• Example: use whitelists, not blacklists
What is COMPLETE MEDIATION ? and why to use it ?
“ Every access to every object must be checked for authority”
• Every access must be checked against the access control mechanism
• Should not rely on access decisions retrieved from a cache
• Every time a user reads a field or record in a file, the system must exercise access control
• Difficult to implement and rarely used (too resource-intensive)
What is OPEN DESIGN ?
“ The design should not be secret”
• The design of a security mechanism should be open rather than secret
• Design of doors:
• We do not construct secret mechanisms that are only known by authorized persons, who then know how to open and close the systems
• We use standardized doors and locks and rely on the protection of the associated key
Who Open Design is known in cryptography ?
• Known as Kerckhoff’s principle in cryptography
• Encryption keys: secret; but encryption algorithms: open to public scrutiny
• Algorithms can be reviewed by many experts
• Examples: Certification processes of DES (1977) and AES (2001) by the NIST
What is SEPARATION OF PRIVILEGE `? (give also an example an d the downside)
“ No single accident, deception, or breach of trust is sufficient to compromise the protected information”
• A system should not grant permission based on a single condition
• Requires multiple attributes to execute an action or access a restricted resource
• Examples:
• Two-factor authentication (password + smart card)
• Company checks for more than $75’000 must be signed by two officers of the company
• Downsides: availability, responsibility, complexity
What is LEAST PRIVILEGE ? (Give an example)
"Every program and every user of the system should operate using the least set of privileges necessary to complete the jobh
. Privileges should be reduced to the (absolute) minimum
. Permissions and rights added as needed, discarded after use
. Subjects not allowed to access objects other than those really needed to complete their job
. Examples:
. Employees not having access in general to other employeesf files
. But for some jobs, e.g., human resources, access is granted since it is needed
-> Role-based access control (RBAC)
. Temporal aspect: system administrators with special privileges only when necessary
What is LEAST COMMON MECHANISM ? (and give an example and the downside)
“ Minimize the amount of mechanism common to more than one user and depended on by all users”
• The design should minimize the functions shared by different users
• Reduce the number of unintended communication paths and the amount of hardware and software on which all users depend
• Easier to verify if there are any undesirable security implications
• Example: DoS attack against bank web servers because the servers and the attackers share the same Internet
• This principle is also restrictive because it limits sharing of resources
What is COMPOSITION OF SECURITY MECHANISMS? And what are the 2 approaches and give examples?
• If only composition was linear…
• Two approaches:
• Defense in depth (or layering): as long as one remains, security policy is
• The strongest mechanism defines the strength of the system
example: Two-factor authentication
• Weakest link: if anyone fails, then security policy is
• The weakest mechanism defines the strength of the system
example: Password recovery via security question
Is true that Secury policies may also be breached without breaking the security mechanisms ? answer and explain and give examples
Yes, Users are principals in the system, and as such they are potential targets for attacks in order to get information to break into the system
Examples:
• Phishing attacks
• Social engineering
• Bad use of passwords
• Weak, written down, etc.
Whats is an ATTACK SURFACEand give examples
Consists of the reachable and exploitable vulnerabilities in a system
Examples:
• Open ports on outward facing Web
• Code that processes incoming data, email, XML, office documents, …
• An employee with access to sensitive information vulnerable to a social engineering attack
• SQL databases, Web forms, …
When does a threat materialize
When an attack succeeds
What is an attack tree?
A data structure representing potential techniques for exploiting security vulnerabilities.
What does the root node of an attack tree represent?
The security incident or goal of the attack.
What do nodes in an attack tree represent?
Subgoals needed for the attack to succeed.
What do leaf nodes in an attack tree represent?
Different ways to initiate an attack.
What can branches in an attack tree be labeled with?
Difficulty, cost, or other attack attributes.
Why label branches in an attack tree?
To compare different attacks.
Why are attack trees useful for security analysts?
They document attacks in a structured form.
What can attack trees help guide?
System design and the choice of countermeasures.
What is the motivation for using attack trees?
To effectively exploit information available on attack patterns.
How can security analysts use attack trees?
To document attacks in a structured form.
What decisions can attack trees guide?
System/application design and the choice and strength of countermeasures.
Represent an Attack tree
see img
Why Standards have been developed ? (give examples)
Standards have been developed to cover management practices
and the overall architecture of security mechanisms and services
• E.g., ISO 27001 and 27002, NIST Cybersecurity Framework
What are Main organizations involved in development or promotion of these standards and descivbe them
National Institute of Standards and Technology (NIST)
• US federal agency that deals with measurement science, standards, and technology related to US government use and to the promotion of US private sector innovation
Internet Society (ISOC)
• Professional membership society that provides leadership in addressing issues that confront the future of the Internet, and is the organization home for the groups responsible for Internet infrastructure standards
International Telecommunication Union (ITU-T)
• United Nations agency in which governments and the private sector coordinate global telecom networks and services
International Organization for Standardization (ISO)
• Nongovernmental organization whose work results in international agreements that are published as International Standards
European Union Agency for Cybersecurity (ENISA)
• EU agency that assists the EU commission and the member states in meeting the requirements of network and information security
• First Cybersecurity Certification Scheme just adopted: https://www.enisa.europa.eu/news/an-eu-prime-eu-adopts-first-cybersecurity-certification-scheme
• AgenceNationalede la Sécuritédes Systèmesd’Information(ANSSI)
• National Cyber Security Center (NCSC)
What are the Two main settings of crryptography:
• Protectingdata at rest
• Hardware thatdoesnot includea CPU, e.g., USB sticks or externalhard drive (HD)
• Lost or stolen laptops: if hard drive not encrypted, data availableto the thief
• Protectingdata in transit
• Internet protocols: HTTPS/SSL, SSH, S/MIME, …
• Wireless communications: WEP, WPA2, WPA3, ...
• Instant messaging (withend-to-end encryption)
What is the KEY TERMINOLOGY of Cryptography ?
science of securing communicationin the presence of potentially malicious third parties
• Common usage: “cryptanalysis of X” = “breaking X”
• Common abbreviation: “crypto”
What is the KERCKHOFF’S PRINCIPLE ?
The ciphermust not berequiredto besecret, and itmust beable to fallintothe hands of the enemywithoutinconvenience.’
• Everythingshouldbepublic knowledge, exceptthe key
What are the motivation of Krerckhoff's principle ?
• In commercial productsitis not realisticto assume thatt he details remain secret (e.g., reverse engineering)
• Keys are easier to protect, generate, and replace
• The design of the schemecanbediscussedand analyzed in public
• No security by obscurity
How works Caesar cipher(50 BC):?
.
Choosea shiftk (secret key) and rotatethe alphabet
.
{A, B, c, Z} .{0, 1, c, 25}
.
Encryption: Ek(x)= x + k mod26
.
Decryption: Dk(c)= c -k mod26
.
Julius Caesar usedthiskindof cipherwithk= 3 or 4
What is Vigenère cipher?
• Vigenère’s idea (polyalphabetic substitution cipher):
• Shift different letters in the sentence with different keys (shift values)
• This prevents the plaintext letters to be always transformed into the same ciphertext letter
If cipertext is long enough, repeated patterns will appear at multiples of the keyword length
How works ONE-TIME PAD (OTP) ?
• Encryption: Ek(m) = m ⊕ k
• Decryption: Dk(c) = c ⊕ k
• One-time pad provides perfect secrecy
