CBS
CBS
CBS
What is the PRIVACY AND ML: PROBLEM #1 ?
• ML needs (a lot of) data to learn!
• Machine learning is based on data to
• Find features
• Train the model
• Data is highly unique → allows many inferences
• Anonymizing may not work…
• Aggregation affects utility and requires careful evaluation
• Hide data
• Noise → differential privacy
• Encryption → homomorphic encryption, secure multiparty computation, …
What is the PRIVACY AND ML: PROBLEM #2 ?
• To obtain value, you must give data!
• To use the ML model, you need to provide a sample…
• If the model is remote (e.g., MLaaS), you are giving this sample out
• We can do privacy-preserving model evaluation:
• Predict on noisy data: affects utility
• Use advanced cryptography: affects performance
What is the PRIVACY AND ML: PROBLEM #3 ?
• The output reveals information!
• Membership inference
• Given the answer of the classifier, infer whether the queried example was used in training
• Attribute inference
• Given the answer of the classifier, infer whether a training sample had a particular attribute
Give a definition of the term adversarial example
• Coined by Szegedy et al. in 2014 [1] with respect to computer vision but
goes back further to work on spam and malware evasion by Srndic &
Laskov [2], Biggio et al. [3], and Dalvi et al. [4] in 2013
What is a Working definition for adversarial example
Inputs to a model that an attacker has designed to cause the model to make a mistake
• Set of techniques that aim to fool ML model
What are the 2 ADVERSARIAL ML ATTACKS ?
• Evasion attacks
• Most prevalent type of attacks
• Perturbation of test data, no influence over training data
• E.g., spammers trying to evade detection by obfuscating the content of spam e-mails
• Samples are modified to be classified as legitimate
• E.g., image-based spam: spam content embedded within an attached image to evade textual analysis by anti-spam filters
• Poisoning attacks
• Adversarial contamination of training data
• ML systems can be re-trained using data collected during operations
• E.g., intrusion detection systems (IDSs) often re-trained using such data
• An attacker may poison this data by injecting malicious samples during operation that subsequently disrupt re-training
Give some example
elf-driving cars:
Stop sign → speed-limit sign with scotch
What is the Objective of MACHINE LEARNING 101 ?
Objective: find model parameters that minimize empirical loss
In MACHINE LEARNING 101 How do we find those parameters/weights?
image
What is the Adversary’s objective ?
Adversary’s objective: find a perturbation that maximizes the loss
HOW TO SOLVE THE OPTIMIZATION PROBLEM?
One approach: Fast Gradient Sign Method (FGSM
HOW TO DEFINE SIMILARITY?
The similarity relation is often represented as adversarial cost constraint.
If the goal is to be imperceptible, the common cost is a norm of the perturbation
OTHER SIMILARITY RELATIONS?
• Constraint on perturbation norm assumes that similar images are close in, e.g., Euclidean distance…
• But this is not always true! [1]
• For example, similarity could be defined as small affine transformations [2] :
Why Defending in general is hard AGAINST ADVERSARIAL EXAMPLES ?
We can only defend against a particular threat model (e.g., perturbations up to epsilon norm), and normally no guarantees
What is the Standard approachfor DEFENDING AGAINST ADVERSARIAL EXAMPLES
Training on simulated adversarial examples, i.e., include adversarial examples in the training set
Is Adversarial samples restricted to computer vision ? Give 3 examples
Adversarial samples are not restricted to computer vision
• Twitter bot detection [1]
• Detection tools can easily be fooled by tweaking the number of replies or retweets
• Text classification [2]
• Audio [3]
• Misleading an automatic speech recognition neural network with some noise
• E.g., transcribing "without the dataset the article is useless" into
"okay google, browse to evil.com"
Give exabple of CYBERSECURITY INCIDENTS
• Data or infrastructure as targets
• Attacks against confidentiality, integrity or availability
• Examples of incident description:
• Man-in-the-Middle attack when data was being transferred through the
public Internet
• Phishing
• People running crypto mining on our infrastructure through our APIs
• Server was compromised
• Botnet communication
Give example of PRIVACY INCIDENTS
• Relation to AI vague (beyond data related)
• Attacks against confidentiality, targeting data, not ML model
• Examples of incident description:
• Patient data should not be shared
• Data privacy
• Incorrect use of temporary storage on AWS lambda led to a potential leak of
raw inference [...] potentially containing PII [...]
Give example of: POISONING INCIDENTS
• Attacks against integrity
• Incident descriptions:
• ML systems being retrained to provide false outputs
• Client/partner employees [...] feel threatened by automation, and either stall
or sabotage the labeling effort, harming the models
Give examples of EVASION INCIDENTS
Evasion → 4
• Attacks against integrity
• Incident descriptions:
• Intentional circumventions
• Users spam to optimize their strategy for job search
• A speed-sign as advertisement put onto a driving-school car was interpreted
by the camera as a valid street sign
• Autonomous vehicle image recognition errors leading to dangerous path
planning
WHAT IS COMPUTER SECURITY?
“Measures and controls that ensure confidentiality, integrity, and availability of information system assets including hardware, software, firmware, and information being processed, stored, and communicated.”
What are the 3 KEY SECURITY OBJECTIVES and explain them ?
Confidentiality
Integrity
Availability
What is Data confidentiality: ?
Data confidentiality: private or confidential information not made available or
disclosed to unauthorized individuals
What is Privacy?
Privacy: individuals control or influence what information related to them may be
collected and stored and by whom and to whom that information may be disclosed
What is Data integrity ?
Data integrity: information and programs are changed only in a specified and
authorized manner
What is system integrity ?
• System integrity: a system performs its intended function in an unimpaired manner,
free from deliberate or inadvertent unauthorized manipulation of the system
What is availability ?
Systems work promptly and service is not denied to authorized users
In terms of requirements and loss security (NIST standard) what i s Confidentiality ?
. Preserving authorized restrictions on information access and disclosure, incl. means for protecting privacy and proprietary information
In terms of requirements and loss security (NIST standard) what is Integrity?
Preventing improper information modification or destruction, incl. ensuring information non-repudiation and authenticity
In terms of requirements and loss security (NIST standard) what is Availability?
Availability
• Ensuring timely and reliable access to and use of information
What is Loss of confidentiality ?
unauthorized disclosure of information
What is Loss of integrity ?
unauthorized modification or destruction of information
What is Loss of availability?
disruption of access to or use of information (denial of service)
What are the 3 OTHER SECURITY OBJECTIVES and explain them
• Authenticity
• Verifying that users are who they say they are
• Accountability (non-repudiation)
• Entity cannot deny having done an action
• Anonymity
• Not being recognizable among a group of people
Give the 8 CHALLENGES IN SECURITY
1. Computer security is not as simple as it might first appear to the novice
2. In developing a particular security mechanism or algorithm, one must always consider potential attacks on those security features
3. Security mechanisms typically involve more than one particular protocol and also require that participants be in possession of some secret information which raises questions about the creation, distribution, and protection of that secret information
4. Attackers only need to find a single weakness, while the designer must find and eliminate all weaknesses to achieve perfect security
5. Security is still too often an afterthought to be incorporated into a system after the design is complete, rather than being an integral part of the design process
6. Security requires regular and constant monitoring
7. There is a natural tendency on the part of users and system managers to perceive little benefit from security investment until a security failure occurs
8. Many users and even security administrators view strong security as an impediment to efficient and user-friendly operation of an information system
What is an Asset ?
• Asset
• Hardware, software, data, communication networks, …
• E.g., a major application, high impact program, physical plant, mission critical system, …
What is an Adversary (or attacker, threat agent)
Individual, group, organization, or government that conducts detrimental activities
What is an Attack
Any kind of malicious activity that attempts to collect, disrupt, deny, degrade, or destroy information system resources or the information itself
What is a Threat ?
Any event with the potential to adversely impact organizational operations, organizational assets, individuals, other organizations, or the Nation through an information system via unauthorized access, destruction, disclosure, modification of information, and/or denial of service
What is aThreat model (or adversary model) ?
Technical term that defines the adversary’s capabilities (more about it next)
-
- 1 / 491
-
Cartes-fiches
What kind of network level defenses exist against worms?
- Limit the number of outgoing connections: limit worm spreading
- Personal firewall: e.g., block outgoing or incoming SMTP connections from unknown applications
- Intrusion detection systems (next lecture)
What is a trojan?
Useful or apparently useful program containing hidden code that performs malicious activities.
What is the difference between a trojan and a virus?
Trojans cannot replicate by themselves, but can do any malicious activity.
How can you defend against trojans?
- Train users to not run programs from untrusted sources.
- Running untrusted programs with least privilege principle.
What kind of malicious activity can trojans do?
- Spy on sensitive user data (spyware).
- Allow remote access (backdoor).
- Base for further attacks, e.g., act as email relay (for spammers).
- Damage routines (corrupting files).
Give some examples of trojan horses.
Tiny banker Trojan (2012) and Zeus banking Trojan (2013).
What are the two modes of operation of these trojans?
First mode of operation:
- Sniff packets to learn when a user visits a banking website.
- Steal credentials before they are sent → send to malware server.
Second mode of operation:
- Sniff packets to learn when a user visits a banking website.
- Steal appearance from website (Man-in-the-Browser attack).
- Ask security details to user on a pop-up → send answer to malware server.
What is phishing?
Mimicking some communication from a trusted source (social engineering).
Give a generic example of a phishing attack.
- Include a URL in a spam e-mail that links to a fake website that mimics the login page of a banking or similar site.
- Suggests that urgent action is required by the user to authenticate their account.
- Attacker exploits the account using the captured credentials.
What is spear-phishing?
Spear-phishing works like classic phishing, but recipients are carefully researched by the attacker. The e-mail is crafted to specifically suit its recipient, often quoting a range of information to convince them of its authenticity.
Give a generic example of spear-phishing.
CEO fraud: sender pretending to be a senior executive to make other employees at an organization to perform a specific action, e.g., the wiring of money to an offshore account.
What are the defenses against phishing?
User education and training: Essential part of anti-phishing strategy is to actively educate its users so that they can identify phishing scams and act accordingly.
- Run regular simulated phishing campaigns.
- Encourage users to modify their browsing habits.
- Legitimate e-mails generally contain an item of information not readily available to the attackers.
Technical solution: phishing is a technological medium to exploit human weaknesses, and technology cannot fully compensate for human weaknesses.
- Filtering out phishing e-mail with spam filters.
- Browsers alerting users to fraudulent websites.
- Monitoring and takedown of phishing websites.
- Multifactor authentication.
What is a backdoor?
Secret entry point into a program allowing the attacker to gain access and bypass the security access procedures.
What would be a legitimate use for a backdoor?
Backdoors are used by programmers to debug and test programs (they are called maintenance hook in this legitimate case). They are used for applications with authentication procedure or long setup.
What is a rootkit?
Malicious code installed in the core of the OS to maintain covert access to the system.
Why are rootkits difficult to detect?
A rootkit hides its presence by subverting the mechanisms (in the OS) that monitor and report on the processes, files, and registries on a computer.
Give an example of a famous rootkit.
Stuxnet (2010) against Iran’s nuclear power plants.
What is a bot?
(AKA zombie) compromised host connected to the Internet used to launch or manage attacks.
What is a botnet?
Multiple (e.g., millions) bots under the control of a single entity and capable of acting in a coordinated manner.
What is a botnet command & control (C&C) center?
A system to keep track of bots and send commands to them.
What is the problem with the botnet star topology?
The C&C machine is a single point of failure: hacker loses control of entire network if it's taken down.
What is the problem with the botnet P2P topology?
It's difficult to manage. If security defenders gain control of enough bots in the botnet, they can counter the malicious commands issues by the attacker (sybil attacks).
Given the problems with the star topology and the P2P topology, what is the architecture to adopt? Draw a schema.
The hybrid architecture: C&C machines are in P2P, while the bots are in star with their respective machines.
What are the usages of botnets?
Distributed denial-of-service (DDoS) attacks, spamming, sniffing traffic, spreading new malware, manipulating online polls, …
What are the monetization avenues for botnets?
- Rental: “Pay me money and I’ll let you use my botnet”
- DDoS extortion: “Pay me or I take down your legitimate business”
- Bulk traffic selling: “Pay me to boost visit counts on your website”
- Click fraud: “Simulate clicks on advertised links to generate revenue”
- Distribute ransomware: “I’ve encrypted your hard drive, pay!”
- Advertise products: “Pay me to leave comments all around the web”
- Bitcoin mining
Give an example of a botnet.
MIRAI (2016) that targeted IoT devices.
What are the defenses against botnets?
Attacking C&C infrastructure:
- Take communication channel offline.
- Hijack/poison the DNS domain of the C&C.
- But botnet takedown is challenging: requires a combination of technical and legal methods and often the collaboration of numerous entities with varying jurisdictions.
Honeypots:
- Vulnerable computer that serves no purpose other than to attract attackers and study their behavior in controlled environments.
- Useful tool to discover where the C&C is.
- Study botnet behavior to find appropriate defense.
What is an Advanced Persistent Threat (APT)?
Well-resourced, persistent application of a wide variety of intrusion technologies and malware to selected targets (usually business or political).
Who are the typical authors of APT?
Typically attributed to state-sponsored organizations and criminal enterprises.
How do APT differ from other types of attacks?
Differ from other types of attack by their careful target selection and stealthy intrusion efforts over extended periods.
Give an example of an APT.
The RUAG espionage case.
Break down the characteristics of APT:
Advanced:
- Wide variety of intrusion technologies and malware including the development of custom malware if required.
- Individual components not necessarily technically advanced but carefully selected to suit the chosen target.
Persistent:
- Determined application of the attacks over an extended period against the chosen target in order to maximize the chance of success.
- Variety of attacks progressively applied until the target is compromised.
Threats:
- Threats to the targets as a result of the organized, capable, and well-funded attackers intending to compromise the specifically chosen targets.
- The active involvement of people in the process greatly raises the threat level from that due to automated attacks tools.
What are the aims of APT attacks?
Varies from theft of intellectual property (IP) or security and infrastructure related data to the physical disruption of infrastructure.
What are the techniques used?
- Social engineering.
- Spear-phishing email.
- Drive-by-downloads from selected compromised websites likely to be visited by personnel in the target organization.
What is the intent with an APT attack?
Infect the target with sophisticated malware with multiple propagation mechanisms and payloads. Once they have gained initial access to systems in the target organization a further range of attack tools are used to maintain and extend their access.
What are the steps of an APT attack?
- Initial compromise.
- Establish foothold.
- Escalate privileges.
- Move laterally.
- Maintain presence.
- Complete mission.
What are the challenges of defending against APT attacks?
- Much harder to defend against due to targeting and persistence.
- Requires a combination of technical countermeasures and user training.
- Even with best practice countermeasures, zero-day attacks can still succeed.
- Multiple layers of defenses are needed.
From the previous examples of attacks, which qualifies as an APT attack?
Stuxnet.
What is the NIST DoS definition ?
NIST (SP 800-61) definesdenial-of-service (DoS) attacksas:“An action that prevents or impairs the authorized use of networks, systems, or applications by exhausting resources such as CPU, memory, bandwidth, and disk space.”
What CIA attack DoS ?
Attack on the availabilityof some service
