You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You shouldn't base your settings on information provided in this section. Perform your
own measurements, this section acts more as a rule of thumb.
Below matrices show latest performance measurements gathered from automated tests from last
week.
Around 3000 tests are performed daily which gives us around 30000 tests in total
used for creation of this matrix
Each entry has the following format:
DurationType: Trait1 x Trait2
DurationType:
init - how long it took to initialize Lambda function
duration - how long the request took
billed - same as duration but rounded to the next 100ms. How much you would
be charged for running this function. See how it translates to money on
AWS Lambda pricing documentation
Trait1 - setting value by which analysis was taken. For grad it could
be True and False. It could be model_name (one of five tested torchvision neural networks).
Trait2 - same as Trait1 but different axis
See concrete comparisons below
init: grad x type
base64
byte
char
short
int
long
float
double
True
262.794
268.382
272.844
272.637
270.627
272.486
279.914
283.939
False
264.209
269.987
271.258
271.682
277.495
275.894
285.3
279.747
init: grad x payload
128x128
256x256
512x512
1024x1024
True
254.866
259.86
269.602
307.477
False
258.506
261.383
272.267
306.083
init: grad x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
True
225.229
297.9
271.317
278.306
293.555
False
226.373
296.043
271.768
278.985
298.575
init: type x payload
128x128
256x256
512x512
1024x1024
base64
257.167
257.679
258.895
281.198
byte
259.657
249.893
267.662
298.845
char
255.026
259.244
270.006
304.967
short
260.231
261.807
266.182
300.049
int
257.553
257.348
274.804
307.239
long
250.143
281.303
268.109
300.064
float
257.498
261.89
280.604
329.155
double
256.516
257.542
282.262
332.019
init: type x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
base64
212.903
288.637
263.232
265.673
289.939
byte
221.264
286.998
272.218
277.232
290.041
char
222.558
290.604
262.26
282.246
303.873
short
221.904
307.45
271.704
272.425
290.27
int
225.391
305.489
275.713
273.626
291.12
long
230.288
294.743
272.685
286.979
286.703
float
235.465
300.092
274.945
289.429
311.165
double
237.995
302.315
279.179
282.466
305.656
init: payload x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
128x128
205.434
281.052
253.675
264.836
279.517
256x256
214.859
284.189
261.805
257.333
284.805
512x512
228.727
294.48
260.142
277.278
295.323
1024x1024
254.235
327.505
312.312
316.925
324.586
duration: grad x type
base64
byte
char
short
int
long
float
double
True
432.529
696.713
710.694
680.162
715.483
691.227
709.192
743.032
False
345.182
601.445
609.604
617.736
601.549
612.657
639.977
597.784
duration: grad x payload
128x128
256x256
512x512
1024x1024
True
142.531
221.485
522.29
1803.21
False
120.025
193.554
453.129
1560.27
duration: grad x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
True
468.485
556.825
723.169
719.833
888.704
False
414.174
509.882
637.426
594.925
731.131
duration: type x payload
128x128
256x256
512x512
1024x1024
base64
120.622
165.184
311.129
995.249
byte
133.315
203.628
498.391
1738.97
char
128.331
209.306
517.482
1822.56
short
135.859
207.389
497.818
1740.91
int
133.42
216.163
519.502
1783.02
long
126.979
228.497
516.98
1760.93
float
135.825
223.045
515.245
1802.25
double
137.281
209.267
536.959
1811.83
duration: type x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
base64
219.756
278.199
442.88
437.775
571.823
byte
449.97
571.625
679.552
695.264
865.326
char
450.345
562.572
692.297
722.42
884.605
short
462.636
600.807
706.776
660.66
815.31
int
473.651
592.709
725.122
712.483
790.561
long
482.671
546.883
741.208
648.576
827.792
float
508.803
557.018
710.671
712.434
879.32
double
500.988
556.12
754.763
682.386
850.336
duration: payload x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
128x128
111.577
88.7494
150.38
143.482
163.189
256x256
168.288
153.559
231.17
218.247
263.759
512x512
366.953
408.622
504.222
527.545
629.891
1024x1024
1113.89
1459.65
1890.06
1798.75
2174.21
billed: grad x type
base64
byte
char
short
int
long
float
double
True
466.967
731.787
745.211
714.972
750.411
726.061
743.148
777.318
False
379.825
636.793
643.841
651.828
636.077
647.913
675.096
632.645
billed: grad x payload
128x128
256x256
512x512
1024x1024
True
179.779
254.078
556.66
1837.33
False
152.366
229.676
489.205
1594.87
billed: grad x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
True
496.569
595.265
759.582
757.506
921.477
False
449.249
550.232
670.066
626.155
765.432
billed: type x payload
128x128
256x256
512x512
1024x1024
base64
157.573
197.474
344.856
1030.27
byte
169.224
238.06
533.519
1774.34
char
162.523
243.762
553.036
1855.85
short
170.518
242.243
532.019
1775
int
167.975
249.366
555.378
1818.43
long
160.381
266.062
552.608
1794.81
float
170.45
257.971
549.952
1836.1
double
171.375
242.654
573.975
1845.74
billed: type x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
base64
249.296
314.235
479.132
472.509
608.277
byte
481.623
610.922
714.977
730.62
899.642
char
480.69
601.661
725.461
757.319
919.109
short
492.568
642.494
742.809
693.365
847.569
int
506.9
634.132
758.885
745.333
823.212
long
515.759
587.212
775.285
684.278
860.044
float
541.204
594.824
744.811
748.252
911.824
double
533.217
595.767
788.475
716.133
883.799
billed: payload x model_name
shufflenet_v2_x1_0
resnet18
mobilenet_v2
mnasnet1_0
mnasnet1_3
128x128
136.208
136.215
185.992
178.347
194.905
256x256
201.749
193.313
264.962
249.489
297.343
512x512
402.269
444.255
538.008
563.571
665.273
1024x1024
1146.93
1494.11
1925.24
1834.52
2207.75
This file was auto-generated on 2020-05-30-11:22:06 based on 34415 tests