-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create AudioDynamicTrigger #2328
base: dev_1.18.0
Are you sure you want to change the base?
Conversation
This code is designed to use the DynamicTrigger class to create a dynamic backdoor attack by generating a trigger and inserting it into an audio signal. If the generate_dynamic_trigger function correctly returns an instance of the DynamicTrigger class, then the attack works
Hi @OrsonTyphanel93 Thank you very much for your pull request! Could you please change the target branch to one of the dev branches? |
Codecov ReportAttention:
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## dev_1.18.0 #2328 +/- ##
===============================================
- Coverage 85.62% 70.70% -14.93%
===============================================
Files 324 324
Lines 29323 29331 +8
Branches 5405 5028 -377
===============================================
- Hits 25108 20738 -4370
- Misses 2837 7446 +4609
+ Partials 1378 1147 -231
|
from sklearn.preprocessing import QuantileTransformer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
def anonymize_speaker(self, spectrogram, noise_std=0.1):
# Create a model for differentially private feature extraction
input_layer = Input(shape=(spectrogram.shape[1],))
hidden_layer = Dense(128, activation='relu')(input_layer)
output_layer = Dense(spectrogram.shape[1])(hidden_layer)
autoencoder = Model(input_layer, output_layer)
autoencoder.compile(optimizer='adam', loss='mean_squared_error')
# Train the autoencoder with noise layers
noisy_spectrogram = spectrogram + np.random.normal(0, noise_std, spectrogram.shape)
autoencoder.fit(noisy_spectrogram, spectrogram, epochs=10, batch_size=32)
# Use the autoencoder to extract features from the spectrogram
features = autoencoder.predict(spectrogram)
# Apply quantization-based transformation
transformer = QuantileTransformer(n_quantiles=100, random_state=0)
quantized_features = transformer.fit_transform(features)
# Reconstruct the spectrogram from the quantized features
reconstructed_spectrogram = autoencoder.predict(quantized_features)
return reconstructed_spectrogram |
Hi @beat-buesser please , next time you test codecov, please try DynamicTrigger with this new anonymiser_speaker function, I think it will make code optimization faster. Thanks ! To make the anonymize_speaker method complex, I have incorporated several techniques and concepts such as : **Differentially private feature extraction: We can introduce differentially private feature extractors based on an autoencoder and an automatic speech recognizer, trained using noise layers. This approach was proposed in the paper (Differentially Private Speaker Anonymization see arxiv link ) and has been shown to obtain private utterances with a provable upper bound on the speaker information they contain. ** Quantization-based transformation: We can promote anonymization algorithms based on quantization-based transformation as an alternative to the most widely used and well-known noise-based approach. This approach was proposed in the paper (Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques see arxiv link link and can reduce the speaker's PPI (privacy preserving information) while maintaining utility. ** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update DynamicTrigger
I'll change the target branch to |
Description : Dynamic Stacking of triggers :
last Update ( 26, may 2024) : https://github.com/OrsonTyphanel93/adversarial-robustness-toolbox/blob/dev_1.14.0/The_END_Last_update_ART_dynamic_backdoor_attacks_trigger_stacking_(_Audio).ipynb
https://github.com/OrsonTyphanel93/adversarial-robustness-toolbox/blob/dev_1.14.0/Update_ART_dynamic_backdoor_attacks_trigger_stacking_(_Audio).ipynb
"DynamicTrigger", exploits a "trigger stacking" technique combining numerous triggers to make detection more difficult. The model can learn to correlate the combined trigger with the desired output by stacking triggers. This means that even if the input has only one trigger, the model can anticipate the expected result. As a result, the model has the ability to generate identical samples with similar class titles, or to assign the label designated by the attacker for each sample, depending on its objectives.
This code is designed to use the DynamicTrigger class to create a dynamic backdoor attack by generating a trigger and inserting it into an audio signal. If the generate_dynamic_trigger function correctly returns an instance of the DynamicTrigger class, then the attack works
Test Configuration: