
############################## Experiment details ##############################

dataset: femnist
dataset_dir: /home/huberkeley/rayleaf/data/femnist/
output_dir: output/femnist/fedavg/10clients-5cpr-0.05lr-25epochs-100rounds-2022-09-18_16:17:40
model: cnn
num_rounds: 100
eval_every: 1
ServerType: <class 'rayleaf.entities.server.Server'>
client_types: [(<class 'rayleaf.entities.client.Client'>, 10)]
clients_per_round: 5
client_lr: 0.05
batch_size: 64
seed: 0
use_val_set: False
num_epochs: 25
gpus_per_client_cluster: 1
num_client_clusters: 1
save_model: False

############################## Simulation ##############################

Spawning 1 ClientClusters using cuda device (this may take a while)
10 total clients: 10 Clients

>>> Training Accuracy: Round 0 <<<
+------------+------------+-------------------+-------------------+
|    average |     median |   10th percentile |   90th percentile |
|------------+------------+-------------------+-------------------|
| 0.00232897 | 0.00277027 |                 0 |        0.00299917 |
+------------+------------+-------------------+-------------------+
>>> Test Accuracy: Round 0 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|         0 |        0 |                 0 |                 0 |
+-----------+----------+-------------------+-------------------+
--- Round 1 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 1 <<<
+-----------+-----------+-------------------+-------------------+
|   average |    median |   10th percentile |   90th percentile |
|-----------+-----------+-------------------+-------------------|
| 0.0637555 | 0.0674539 |         0.0528529 |         0.0741964 |
+-----------+-----------+-------------------+-------------------+
>>> Test Accuracy: Round 1 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
| 0.0516796 | 0.050641 |         0.0219512 |          0.097619 |
+-----------+----------+-------------------+-------------------+
--- Round 2 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 2 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.149927 | 0.140622 |          0.089226 |          0.254404 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 2 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.136951 | 0.146429 |           0.05625 |          0.239286 |
+-----------+----------+-------------------+-------------------+
--- Round 3 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 3 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.358661 | 0.345127 |          0.161749 |          0.564728 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 3 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.330749 | 0.368132 |          0.118016 |          0.526159 |
+-----------+----------+-------------------+-------------------+
--- Round 4 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 4 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.50655 |  0.48531 |           0.40591 |          0.634001 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 4 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.447028 | 0.481707 |          0.286667 |          0.578571 |
+-----------+----------+-------------------+-------------------+
--- Round 5 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 5 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.60524 | 0.618225 |            0.4937 |           0.76528 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 5 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.565891 | 0.581731 |          0.394737 |               0.7 |
+-----------+----------+-------------------+-------------------+
--- Round 6 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 6 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.653857 | 0.610574 |          0.510884 |          0.796969 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 6 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.630491 | 0.650407 |          0.439474 |          0.814286 |
+-----------+----------+-------------------+-------------------+
--- Round 7 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 7 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.65968 | 0.586461 |          0.533068 |          0.830438 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 7 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.614987 | 0.613948 |          0.444298 |          0.792857 |
+-----------+----------+-------------------+-------------------+
--- Round 8 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 8 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.71179 | 0.663169 |          0.569803 |          0.862126 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 8 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.645995 | 0.617388 |          0.447076 |          0.795238 |
+-----------+----------+-------------------+-------------------+
--- Round 9 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 9 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.72198 | 0.689068 |          0.641964 |          0.846383 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 9 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.664083 | 0.628289 |          0.512573 |          0.864286 |
+-----------+----------+-------------------+-------------------+
--- Round 10 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 10 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.745852 | 0.690996 |          0.645195 |            0.8723 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 10 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.69509 | 0.721474 |          0.497222 |          0.861905 |
+-----------+----------+-------------------+-------------------+
--- Round 11 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 11 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.724017 | 0.737513 |          0.541514 |           0.85606 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 11 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.674419 | 0.675439 |          0.429861 |          0.864286 |
+-----------+----------+-------------------+-------------------+
--- Round 12 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 12 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.740902 | 0.729083 |          0.625071 |          0.867706 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 12 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.664083 | 0.692763 |          0.488889 |          0.790476 |
+-----------+----------+-------------------+-------------------+
--- Round 13 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 13 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.767394 | 0.748201 |           0.64466 |          0.876709 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 13 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.700258 | 0.705263 |          0.491667 |          0.883333 |
+-----------+----------+-------------------+-------------------+
--- Round 14 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 14 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.778457 | 0.764724 |           0.69759 |          0.843426 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 14 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.702842 | 0.728354 |          0.473538 |          0.859524 |
+-----------+----------+-------------------+-------------------+
--- Round 15 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 15 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.796215 | 0.774949 |          0.740387 |          0.868921 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 15 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.713178 | 0.715854 |           0.57383 |          0.857143 |
+-----------+----------+-------------------+-------------------+
--- Round 16 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 16 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|   0.80786 | 0.809084 |          0.732324 |          0.857322 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 16 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.715762 |  0.73117 |          0.597515 |          0.795238 |
+-----------+----------+-------------------+-------------------+
--- Round 17 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 17 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.813683 | 0.805648 |          0.757896 |          0.903031 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 17 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.710594 | 0.724828 |          0.576608 |          0.811905 |
+-----------+----------+-------------------+-------------------+
--- Round 18 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 18 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.821252 | 0.841119 |          0.720797 |          0.897953 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 18 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.710594 | 0.706996 |          0.559028 |          0.859524 |
+-----------+----------+-------------------+-------------------+
--- Round 19 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 19 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.822707 |  0.81388 |           0.78057 |          0.869929 |
+-----------+----------+-------------------+-------------------+
>>> Test Accuracy: Round 19 <<<
+-----------+----------+-------------------+-------------------+
|   average |   median |   10th percentile |   90th percentile |
|-----------+----------+-------------------+-------------------|
|  0.723514 | 0.721474 |          0.580263 |          0.859524 |
+-----------+----------+-------------------+-------------------+
--- Round 20 of 100: Training 5 clients: 5 Clients ---
>>> Training Accuracy: Round 20 <<<
