Hyperparameters, training graph, tail of training log, and csv with values in the log can be found in details for each model.

ResNet

Layers	Version	Scaled?	Best Top1 Acc	Best Top5 Acc	Details
18	v1	no	87.9	99.7	here

Layers	Version	Scaled?	Best Top1 Acc	Best Top5 Acc	Details	Commment
18	v1	no	(49.2)	(73.3)	here	no LR update
18	v1	yes	51.9	75.5	here
18	v2	no	53.2	76.5	here
18	v2	yes	49.5	73.4	here

DenseNet

Layers	Growth-Rate	Init-Features	Reduction	Model Size	FP SC	Relu SC	Best Top1 Acc	Best Top5 Acc	Details
21	128	128	1,1.4,1.4	1.4 MB	no	no	87.8	99.5	here
21	128	128	2,2,2		yes	no	86.4	99.4	here
21	128	128	2,2,2		yes	yes	90.3	99.7	here
21	128	128	1,1.4,1.4		no	no	87.6	99.5	here

Layers	Growth-Rate	Init-Features	Reduction	Model Size	FP SC	Relu SC	Best Top1 Acc	Best Top5 Acc	Details
13	256	64	1,1.4,1.4	3.31 MB	no	no	50.2	73.7	here
21	128	64	1,1.4,1.4	3.39 MB	no	no	52.7	75.7	here
37	64	64	1,1.4,1.4	3.45 MB	no	no	54.3	77.3	here
13	256	128	1,1.4,1.4		no	no	51.9	75.2	here
21	128	128	1,1.4,1.4		no	no	54.1	77.1	here
21	128	128	1.5,1.4,1.4(+sc)		no	no	52.7	76.2	here
37	64	64	2,2,2	3.08 MB	yes	yes	57.1	80.0	here
21	128	64	2,2,2	3.03 MB	yes	yes	55.9	78.5	here
21	128	128	2,2,2	3.48 MB	yes	yes	57.2	79.6	here
21	160	128	2,2,2	4.4 MB	yes	yes	59.3	81.4	here
21	160	128	2.2,2.2,2.2	3.99 MB	yes	yes	58.6	81.0	here

Layers	Version	Scaled?	FP SC	Pool SC	Approx Sign	Best Top1 Acc	Best Top5 Acc	Details
18	1	no	no	yes	yes	84.1	99.3	here
18	1	no	no	yes	no	85.6	99.3	here
18	1	no	yes	yes	yes	84.9	99.3	here
18	1	no	yes	yes	no	86.1	99.3	here
18	2	no	no	yes	yes	84.9	99.3	here
18	2	no	no	yes	no	87.2	99.5	here
18	2	no	yes	yes	yes	86.1	99.4	here
18	2	no	yes	yes	no	87.6	99.5	here
18	1	yes	no	yes	yes	83.7	99.2	here
18	1	yes	no	yes	no	85.6	99.3	here
18	1	yes	yes	yes	yes	83.8	99.1	here
18	1	yes	yes	yes	no	86.1	99.2	here
18	2	yes	no	yes	yes	84.2	99.2	here
18	2	yes	no	yes	no	83.6	99.2	here
18	2	yes	yes	yes	yes	84.4	99.3	here
18	2	yes	yes	yes	no	84.7	99.2	here
34	1	no	yes	yes	no	86.2	99.3	here

Layers	Version	Scaled?	FP SC	Pool SC	Approx Sign	Best Top1 Acc	Best Top5 Acc	Details
18	1	no	no	yes	yes	54.3	77.6	here
18	1	no	no	yes	no	54.4	77.5	here
18	1	no	yes	yes	yes	56.6	79.3	here
18	1	no	yes	yes	no	56.7	79.2	here
18	1	yes	yes	yes	yes	55.3	78.3	here
18	1	yes	yes	yes	no	55.6	78.4	here
18	1	yes	no	yes	yes	53.3	76.4	here
18	1	yes	no	yes	no	52.7	76.1	here
18	2	no	yes	yes	no	54.5	77.5	here
18	2	yes	yes	yes	no	52.1	75.4	here
34	1	no	yes	yes	no	59.5	81.5	here
34	1	no	no	yes	no	58.1	80.6	here

Extended trainings of above models:

Layers	Version	Scaled?	FP SC	Pool SC	Approx Sign	Best Top1 Acc	Best Top5 Acc	Details
18	1	no	yes	yes	no	56.9	79.7	here
34	1	no	yes	yes	no	60.0	82.0	here