Sample Image

Detailed Analysis of the D-LORD Face Recognition Dataset

The D-LORD face recognition dataset serves as a public benchmark for face verification tasks. It encompasses a variety of face recognition models, each tested at varying distances and evaluated across multiple False Positive Rate (FPR) settings, ranging from 0.01 to 0.000001. The dataset facilitates the analysis of model performance through graphs depicting Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) metrics for each model. Additionally, a custom graph is provided, allowing users to select a specific model and its corresponding True Positive Rate (TPR) and FPR settings via a dropdown list.

View dynamic graphs depicting TPR and distance variations across different models. Click here

Frames and graphs at distance 5 meters

Model TPR@0.01FAR TPR@0.001FAR TPR@0.0001FAR TPR@0.00001FAR TPR@0.000001FAR
arcface - r100[1] 93.40 88.19 81.93 72.97 66.52
facenet - vggface2[2] 62.77 40.09 22.50 10.25 6.08
elasticface-arc[3] 92.46 86.83 80.48 72.61 57.78
arcface - r 18[1] 87.65 79.95 70.76 62.48 51.75
arcface - r 50[1] 93.37 88.15 81.75 73.14 67.70
arcface-r 34[1] 90.76 84.62 76.14 68.99 58.39
facenet-casia webface[2] 42.46 23.44 12.42 5.60 2.58
ghostface[4] 87.34 77.28 66.21 58.18 37.69
cosface r 18[5] 90.26 84.03 76.25 68.05 62.76
cosface r 34[5] 92.81 87.69 81.19 73.74 63.15
cosface r50[5] 94.45 90.43 85.26 79.95 70.18
cosface r100[5] 95.19 91.98 87.86 82.11 70.11
adaface-r100(12wb)[6] 95.06 90.88 85.69 77.58 74.21
adaface -r100(4wb)[6] 94.42 89.91 83.07 73.72 63.37
adaface-r100(ms1mv2)[6] 92.48 86.28 78.25 71.24 64.55
adaface -r100(ms1mv3)[6] 92.59 86.86 79.69 72.12 57.89
adaface-r50(casia)[6] 80.72 67.29 54.86 45.46 31.90
adaface -r50(wb4)[6] 92.58 86.59 78.36 68.27 59.10
adaface-r18(wb)[6] 88.07 80.08 71.59 60.93 42.12
adaface-r18(casia)[6] 73.17 57.48 43.02 28.71 19.36
adaface-r18(VGG)[6] 83.03 71.24 59.10 44.93 38.17
adaface-r50-ms1mv2[2] 91.19 84.67 76.37 67.24 66.11
elasticface-cos[3] 93.23 88.67 82.33 73.42 63.00
elasticface-arc+[3] 92.28 86.74 80.05 72.14 58.79
elasticface-cos+[3] 93.48 88.71 82.62 77.87 57.23
Graph Image
Graph Image

Frames and graphs at distance 7 meters

Model TPR@0.01FAR TPR@0.001FAR TPR@0.0001FAR TPR@0.00001FAR TPR@0.000001FAR
arcface-r34[1] 82.40 73.20 63.68 55.19 46.46
arcface-r100[1] 85.70 77.40 68.88 62.15 45.12
facenet-vggface2[2] 55.75 34.08 18.35 11.60 7.55
elasticface-arc[3] 83.87 75.70 67.93 59.44 48.20
arcface-r18[1] 79.41 69.37 59.23 48.02 42.60
arcface-r50[1] 84.53 75.22 66.11 57.82 44.44
facenet-casia-webface[2] 36.50 19.89 10.27 5.92 3.94
ghostface[4] 79.55 68.12 56.72 46.38 39.08
cosface-r18[5] 82.10 73.02 64.28 53.99 45.98
cosface-r34[5] 85.21 77.33 68.88 61.13 51.23
cosface-r50[5] 87.49 80.28 72.73 66.32 61.27
cosface-r100[5] 88.67 82.34 76.00 68.65 59.61
adaface-r100(12wb)[6] 88.42 81.34 73.91 65.62 61.51
adaface-r100(4wb)[6] 87.40 79.49 71.12 63.86 61.33
adaface-r100(ms1mv2)[6] 84.40 74.90 66.05 57.85 48.30
adaface-r100(ms1mv3)[6] 85.42 76.78 67.16 59.62 55.73
adaface-r50(casia)[6] 71.10 57.35 45.18 35.41 23.85
adaface-r50(wb4)[6] 85.23 75.84 66.69 58.02 44.88
adaface-r18(wb)[6] 80.25 70.29 60.39 50.44 31.87
adaface-r18(casia)[6] 62.88 48.14 34.33 22.12 10.02
adaface-r18(vgg)[6] 73.76 60.96 48.89 38.32 31.58
adaface-r50-ms1mv2[6] 82.88 73.87 65.07 57.54 49.93
elasticface-arc+[3] 83.71 75.76 67.36 60.76 51.77
elasticface-cos+[3] 85.23 77.54 76.76 63.16 58.22
elasticface-cos[3] 85.23 77.54 70.76 63.11 58.21
Graph Image
lGraph Image

Frames and graphs at distance 10 meters

Model TPR@0.01FAR TPR@0.001FAR TPR@0.0001FAR TPR@0.00001FAR TPR@0.000001FAR
arcface - r100[1] 80.11 70.11 62.00 54.77 43.62
facenet - vggface2[2] 50.18 31.80 19.15 10.44 6.73
elasticface-arc[3] 78.06 68.24 59.62 50.40 38.63
arcface-r 18[1] 73.72 62.80 54.02 46.40 38.18
arcface-r 50[1] 78.60 68.23 59.36 50.03 32.10
arcface-r 34[1] 76.24 65.71 56.73 47.37 40.03
facenet-casia webface[2] 33.10 18.01 9.01 5.01 2.31
ghostface[4] 74.15 61.50 50.09 40.14 25.05
cosface r 18[5] 77.23 66.86 58.37 50.15 45.91
cosface r 34[5] 80.09 70.71 62.49 54.05 42.00
cosface r50[5] 82.29 73.47 65.59 59.22 54.82
cosface r100[5] 83.46 75.55 68.14 61.16 52.05
adaface -r100(12wb)[6] 83.60 74.62 66.99 60.40 38.53
adaface -r100(4wb)[6] 81.94 72.42 64.34 58.18 45.42
adaface-r100(ms1mv2)[6] 79.06 67.77 59.32 50.14 36.12
adaface-r100(ms1mv3)[6] 80.57 69.89 61.27 53.10 40.62
adaface-r50(casia)[6] 66.03 52.56 40.74 31.27 25.40
adaface -r50(wb4)[6] 79.70 69.47 60.62 53.55 34.72
adaface-r18(wb)[6] 75.92 64.80 55.05 47.12 32.59
adaface-r18(casia)[6] 57.54 43.99 31.59 17.98 12.54
adaface-r18(vgg)[6] 69.23 55.56 43.47 35.96 30.38
elasticface-arc+[3] 77.67 68.14 59.71 51.63 43.74
elasticface-cos+[3] 79.53 70.01 62.59 56.02 47.01
elasticface-cos[3] 79.56 70.12 63.27 55.99 50.09
adaface-r50 (ms1mv2)[6] 77.72 67.27 58.80 50.64 29.61
Graph Image
Graph Image

Frames and graphs at distance 15 meters

Model TPR@0.01FAR TPR@0.001FAR TPR@0.0001FAR TPR@0.00001FAR TPR@0.000001FAR
arcface - r100[1] 61.95 51.90 44.67 37.06 28.90
facenet - vggface2[2] 36.66 21.76 12.61 7.64 2.74
elasticface-arc[3] 59.69 50.36 42.73 34.79 30.51
arcface-r 18[1] 55.90 46.26 38.58 31.05 19.69
arcface-r50[1] 59.75 50.40 43.30 32.96 23.26
arcface-r34[1] 58.18 48.53 40.61 31.90 26.23
facenet-casia webface[2] 23.24 11.57 5.56 3.01 1.66
cosface r 18[5] 60.41 49.71 42.62 34.35 28.44
cosface r 34[5] 62.65 52.64 45.79 39.03 29.15
cosface r50[5] 65.61 55.30 48.55 42.67 36.19
cosface r100[5] 66.71 56.98 50.42 44.37 35.42
adaface-r100(12wb)[6] 67.08 56.55 49.56 44.55 41.13
adaface -r100(4wb)[6] 64.11 54.15 46.84 40.57 37.08
adaface-r100(ms1mv2)[6] 60.31 50.22 42.38 36.58 31.12
adaface -r100(ms1mv3)[6] 62.85 52.26 44.26 39.65 33.98
adaface-r50(casia)[6] 49.02 38.16 29.60 21.73 18.50
adaface -r50(wb4)[6] 62.18 52.02 44.83 35.68 30.95
adaface-r18(wb)[6] 59.47 49.07 40.37 32.32 24.85
adaface -r18(casia)[6] 42.80 31.61 21.97 14.08 10.89
adaface-r18(vgg)[6] 53.04 41.02 31.12 22.36 13.24
elasticface-cos[1] 60.89 51.97 45.08 39.47 25.29
ghostface[4] 56.62 45.51 36.36 29.50 24.06
adaface-r50-ms1mv2[5] 59.74 50.30 42.06 34.71 27.13
elasticface-arc+[3] 59.19 50.12 42.58 36.23 29.30
elasticface-cos+[3] 61.22 51.68 44.90 37.90 29.32
Graph Image
Graph Image

Model Graph Selector

References

  1. [1] Title: ArcFace: Additive Angular Margin Loss for Deep Face Recognition
    https://arxiv.org/abs/1801.07698
  2. [2] Title: FaceNet: A Unified Embedding for Face Recognition and Clustering
    https://arxiv.org/abs/1503.03832
  3. [3] Title: ElasticFace: Elastic Margin Loss for Deep Face Recognition
    https://arxiv.org/pdf/2109.09416
  4. [4] Title: GhostFaceNets: Lightweight Face Recognition Model From Cheap Operations
    https://ieeexplore.ieee.org/document/10098610
  5. [5] Title: CosFace: Large Margin Cosine Loss for Deep Face Recognition
    https://arxiv.org/pdf/1801.09414
  6. [6] Title: AdaFace: Quality Adaptive Margin for Face Recognition
    https://arxiv.org/abs/2204.00964

Appendix

Format of models used: [model_name]_[backbone/loss function]_[dataset]_[distance]

Note: If a convention has less than 4 fields mentioned, either the model has been trained on a single dataset or single backbone.

Models:

adaf: AdaFace

af: ArcFace

cf: CosFace

ef: ElasticFace

fn: FaceNet

gf: GhostFaceNets

Backbone:

r100/ir101: Resnet100

r50/ir50: Resnet50

r18/ir18: Resnet18

r34: Resnet34

Loss Function:

arc: arcface

cos: cosface

arc+: arcface+

cos+: cosface+

Dataset:

ms1mv2: ms1mv2

ms1mv3: ms1mv3

wf12/wb12: webface 12

wf4/wb4: webface 4

casia: casia-webface

vgg2: vgg face2