
############################## Experiment details ##############################

dataset: speech_commands
dataset_dir: ../data/speech_commands/
output_dir: output/fedavg-speech_commands-2022-08-15_18:29:19/
model: m5
num_rounds: 100
eval_every: 10
ServerType: <class 'rayleaf.entities.server.Server'>
client_types: [(<class 'rayleaf.entities.client.Client'>, 10)]
clients_per_round: 2
client_lr: 0.05
batch_size: 64
seed: 0
use_val_set: False
num_epochs: 10
gpus_per_client_cluster: 1
num_client_clusters: 4
save_model: False

############################## Simulation ##############################

Spawning 4 ClientClusters using cuda device (this may take a while)
10 total clients: 10 Clients

>>> Training Accuracy: Round 0 <<<
+-----------+-----------+-------------------+-------------------+
|   average |    median |   10th percentile |   90th percentile |
|-----------+-----------+-------------------+-------------------|
| 0.0353006 | 0.0371496 |          0.022366 |         0.0473295 |
+-----------+-----------+-------------------+-------------------+
>>> Test Accuracy: Round 0 <<<
+-----------+-----------+-------------------+-------------------+
|   average |    median |   10th percentile |   90th percentile |
|-----------+-----------+-------------------+-------------------|
| 0.0479303 | 0.0364583 |          0.025571 |         0.0778537 |
+-----------+-----------+-------------------+-------------------+
--- Round 1 of 100: Training 2 clients: 2 Clients ---
--- Round 2 of 100: Training 2 clients: 2 Clients ---
--- Round 3 of 100: Training 2 clients: 2 Clients ---
--- Round 4 of 100: Training 2 clients: 2 Clients ---
--- Round 5 of 100: Training 2 clients: 2 Clients ---
--- Round 6 of 100: Training 2 clients: 2 Clients ---
--- Round 7 of 100: Training 2 clients: 2 Clients ---
--- Round 8 of 100: Training 2 clients: 2 Clients ---
--- Round 9 of 100: Training 2 clients: 2 Clients ---
--- Round 10 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 10 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.289024 | 0.193987 |          0.071746 |          0.641024 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 10 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.20915 |  0.16997 |         0.0465353 |          0.374216 |
+-----------+----------+-------------------+-------------------+
--- Round 11 of 100: Training 2 clients: 2 Clients ---
--- Round 12 of 100: Training 2 clients: 2 Clients ---
--- Round 13 of 100: Training 2 clients: 2 Clients ---
--- Round 14 of 100: Training 2 clients: 2 Clients ---
--- Round 15 of 100: Training 2 clients: 2 Clients ---
--- Round 16 of 100: Training 2 clients: 2 Clients ---
--- Round 17 of 100: Training 2 clients: 2 Clients ---
--- Round 18 of 100: Training 2 clients: 2 Clients ---
--- Round 19 of 100: Training 2 clients: 2 Clients ---
--- Round 20 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 20 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.413127 | 0.344112 |          0.259733 |          0.788782 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 20 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.294118 | 0.218108 |          0.144408 |          0.503691 |
+-----------+----------+-------------------+-------------------+
--- Round 21 of 100: Training 2 clients: 2 Clients ---
--- Round 22 of 100: Training 2 clients: 2 Clients ---
--- Round 23 of 100: Training 2 clients: 2 Clients ---
--- Round 24 of 100: Training 2 clients: 2 Clients ---
--- Round 25 of 100: Training 2 clients: 2 Clients ---
--- Round 26 of 100: Training 2 clients: 2 Clients ---
--- Round 27 of 100: Training 2 clients: 2 Clients ---
--- Round 28 of 100: Training 2 clients: 2 Clients ---
--- Round 29 of 100: Training 2 clients: 2 Clients ---
--- Round 30 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 30 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.639272 | 0.621539 |          0.418373 |          0.861429 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 30 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.477124 | 0.418919 |          0.224607 |           0.75975 |
+-----------+----------+-------------------+-------------------+
--- Round 31 of 100: Training 2 clients: 2 Clients ---
--- Round 32 of 100: Training 2 clients: 2 Clients ---
--- Round 33 of 100: Training 2 clients: 2 Clients ---
--- Round 34 of 100: Training 2 clients: 2 Clients ---
--- Round 35 of 100: Training 2 clients: 2 Clients ---
--- Round 36 of 100: Training 2 clients: 2 Clients ---
--- Round 37 of 100: Training 2 clients: 2 Clients ---
--- Round 38 of 100: Training 2 clients: 2 Clients ---
--- Round 39 of 100: Training 2 clients: 2 Clients ---
--- Round 40 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 40 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.630447 |  0.64453 |           0.29881 |          0.925732 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 40 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.494553 | 0.501469 |            0.1525 |          0.804992 |
+-----------+----------+-------------------+-------------------+
--- Round 41 of 100: Training 2 clients: 2 Clients ---
--- Round 42 of 100: Training 2 clients: 2 Clients ---
--- Round 43 of 100: Training 2 clients: 2 Clients ---
--- Round 44 of 100: Training 2 clients: 2 Clients ---
--- Round 45 of 100: Training 2 clients: 2 Clients ---
--- Round 46 of 100: Training 2 clients: 2 Clients ---
--- Round 47 of 100: Training 2 clients: 2 Clients ---
--- Round 48 of 100: Training 2 clients: 2 Clients ---
--- Round 49 of 100: Training 2 clients: 2 Clients ---
--- Round 50 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 50 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.75786 | 0.774469 |          0.416777 |             0.991 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 50 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.568627 | 0.512045 |          0.239674 |          0.944437 |
+-----------+----------+-------------------+-------------------+
--- Round 51 of 100: Training 2 clients: 2 Clients ---
--- Round 52 of 100: Training 2 clients: 2 Clients ---
--- Round 53 of 100: Training 2 clients: 2 Clients ---
--- Round 54 of 100: Training 2 clients: 2 Clients ---
--- Round 55 of 100: Training 2 clients: 2 Clients ---
--- Round 56 of 100: Training 2 clients: 2 Clients ---
--- Round 57 of 100: Training 2 clients: 2 Clients ---
--- Round 58 of 100: Training 2 clients: 2 Clients ---
--- Round 59 of 100: Training 2 clients: 2 Clients ---
--- Round 60 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 60 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.788196 | 0.834567 |             0.595 |          0.978571 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 60 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.601307 |  0.49009 |          0.332895 |          0.923122 |
+-----------+----------+-------------------+-------------------+
--- Round 61 of 100: Training 2 clients: 2 Clients ---
--- Round 62 of 100: Training 2 clients: 2 Clients ---
--- Round 63 of 100: Training 2 clients: 2 Clients ---
--- Round 64 of 100: Training 2 clients: 2 Clients ---
--- Round 65 of 100: Training 2 clients: 2 Clients ---
--- Round 66 of 100: Training 2 clients: 2 Clients ---
--- Round 67 of 100: Training 2 clients: 2 Clients ---
--- Round 68 of 100: Training 2 clients: 2 Clients ---
--- Round 69 of 100: Training 2 clients: 2 Clients ---
--- Round 70 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 70 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.769443 | 0.719595 |          0.560714 |                 1 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 70 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.620915 | 0.571498 |          0.439929 |          0.867187 |
+-----------+----------+-------------------+-------------------+
--- Round 71 of 100: Training 2 clients: 2 Clients ---
--- Round 72 of 100: Training 2 clients: 2 Clients ---
--- Round 73 of 100: Training 2 clients: 2 Clients ---
--- Round 74 of 100: Training 2 clients: 2 Clients ---
--- Round 75 of 100: Training 2 clients: 2 Clients ---
--- Round 76 of 100: Training 2 clients: 2 Clients ---
--- Round 77 of 100: Training 2 clients: 2 Clients ---
--- Round 78 of 100: Training 2 clients: 2 Clients ---
--- Round 79 of 100: Training 2 clients: 2 Clients ---
--- Round 80 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 80 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.88417 | 0.892559 |          0.801643 |                 1 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 80 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.67756 | 0.653216 |          0.501734 |          0.799314 |
+-----------+----------+-------------------+-------------------+
--- Round 81 of 100: Training 2 clients: 2 Clients ---
--- Round 82 of 100: Training 2 clients: 2 Clients ---
--- Round 83 of 100: Training 2 clients: 2 Clients ---
--- Round 84 of 100: Training 2 clients: 2 Clients ---
--- Round 85 of 100: Training 2 clients: 2 Clients ---
--- Round 86 of 100: Training 2 clients: 2 Clients ---
--- Round 87 of 100: Training 2 clients: 2 Clients ---
--- Round 88 of 100: Training 2 clients: 2 Clients ---
--- Round 89 of 100: Training 2 clients: 2 Clients ---
--- Round 90 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 90 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.937121 | 0.984756 |          0.873119 |                 1 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 90 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.736383 | 0.719772 |          0.522336 |          0.926438 |
+-----------+----------+-------------------+-------------------+
--- Round 91 of 100: Training 2 clients: 2 Clients ---
--- Round 92 of 100: Training 2 clients: 2 Clients ---
--- Round 93 of 100: Training 2 clients: 2 Clients ---
--- Round 94 of 100: Training 2 clients: 2 Clients ---
--- Round 95 of 100: Training 2 clients: 2 Clients ---
--- Round 96 of 100: Training 2 clients: 2 Clients ---
--- Round 97 of 100: Training 2 clients: 2 Clients ---
--- Round 98 of 100: Training 2 clients: 2 Clients ---
--- Round 99 of 100: Training 2 clients: 2 Clients ---
--- Round 100 of 100: Training 2 clients: 2 Clients ---
>>> Training Accuracy: Round 100 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.974628 | 0.996951 |          0.935714 |                 1 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 100 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.729847 | 0.631007 |          0.529883 |          0.962437 |
+-----------+----------+-------------------+-------------------+

############################## Post-Simulation ##############################

Total Experiment time: 0:03:06.732196
