Preliminary in silico assessment of the specificity of published molecular assays and design of new assays using the available whole genome sequences of 2019-nCoV

SARS-CoV-2 coronavirus nCoV-2019 Diagnostics and Vaccines
37

Updated to 81,375 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt).

identifier type PT TP FN Percent True PF FP TN
Noblis.12 Probe 80629 375 371 99.54% 0 0 680
Noblis.40 Probe 79217 1929 229 99.72% 0 1 1420
Noblis.42 Probe 80711 279 385 99.53% 0 2 1136
Noblis.44 Probe 80193 416 766 99.06% 0 1 1786
Noblis.57 Probe 79696 795 884 98.91% 0 1 853
ncov_e_gene Probe 80662 253 460 99.43% 361 15 53
ncov_n_gene Probe 79688 1040 647 99.20% 0 340 72
ncov_rdrp_1 Probe 0 81109 266 99.67% 433 105 5967
ncov_rdrp_2 Probe 6 81129 240 99.71% 2 73 5916
cdc_n1 Probe 78614 2517 244 99.70% 0 2 381
cdc_n2 Probe 79527 773 1075 98.68% 0 1 372
cdc_n3 Probe 79854 1095 426 99.48% 1 353 36
China_N Probe 53206 1218 26951 66.88% 0 2 691
China_ORF1ab Probe 78024 586 2765 96.60% 1 0 367
France_nCoV_IP2 Probe 80744 326 305 99.63% 1 0 367
France_nCoV_IP4 Probe 80688 521 166 99.80% 0 0 1190
HKU-N Probe 79290 1012 1073 98.68% 327 33 34
HKU-ORF1b-nsp14 Probe 80555 585 235 99.71% 324 29 452
Japan_NIID_2019-nCOV_N Probe 0 80250 1125 98.62% 0 0 384
Japan_NIID_WH-1_F24381 Outer 79663 1175 537 99.34% 0 1 601
Japan_NIID_WH-1_F501 Outer 79542 1149 684 99.16% 0 1 370
Japan_NIID_WH-1_F509 Outer 79641 953 781 99.04% 1 0 369
Japan_NIID_WH-1_Seq_F24383 Outer 79624 1200 551 99.32% 0 0 606
Japan_NIID_WH-1_Seq_F519 Outer 79428 769 1178 98.55% 0 1 369
Japan_WuhanCoV-spk1 Outer 79824 1003 548 99.33% 1 1 594
Thailand_WH-NIC_N Probe 80166 938 271 99.67% 0 2 375
Chan-N Probe 79664 622 1089 98.66% 4 3 353
Chan-ORF1ab Probe 0 80465 910 98.88% 0 5 2
Chan-S Probe 79578 317 1480 98.18% 5 0 2
Niu-E Probe 80483 387 505 99.38% 26 359 0
Won-E Outer 2 80967 406 99.50% 0 384 1
Won-N Outer 79037 1767 571 99.30% 8 7 358
Won-ORF1ab Outer 80690 385 300 99.63% 5 2 313
Won-S Outer 77946 263 3166 96.11% 5 1 0
Yip-ORF1ab Outer 80055 433 887 98.91% 5 1 1
Young-N Probe 0 80875 500 99.39% 0 5 1
Young-ORF1ab Probe 80786 362 227 99.72% 5 1 0
Young-S Probe 79676 802 897 98.90% 4 1 0
38

Updated to 155,090 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt).

identifier type PT TP FN Percent True PF FP TN
Noblis.12 Probe 151389 3124 577 99.63% 0 0 680
Noblis.40 Probe 151690 2939 461 99.70% 0 1 1420
Noblis.42 Probe 153741 685 664 99.57% 0 2 1136
Noblis.44 Probe 152825 1123 1142 99.26% 0 1 1786
Noblis.57 Probe 151126 1667 2297 98.52% 0 1 853
ncov_e_gene Probe 153740 484 866 99.44% 361 15 53
ncov_n_gene Probe 149783 3679 1628 98.95% 0 340 72
ncov_rdrp_1 Probe 0 154278 812 99.48% 433 105 5967
ncov_rdrp_2 Probe 8 154308 774 99.50% 2 73 5916
cdc_n1 Probe 150613 4099 378 99.76% 0 2 381
cdc_n2 Probe 149756 3617 1717 98.89% 0 1 372
cdc_n3 Probe 152268 2067 755 99.51% 1 353 36
China_N Probe 89180 3746 62164 59.92% 0 2 691
China_ORF1ab Probe 149836 960 4294 97.23% 1 0 367
France_nCoV_IP2 Probe 153767 779 544 99.65% 1 0 367
France_nCoV_IP4 Probe 153580 1190 320 99.79% 0 0 1190
HKU-N Probe 151013 2353 1724 98.89% 327 33 34
HKU-ORF1b-nsp14 Probe 152940 1810 340 99.78% 324 29 452
Japan_NIID_2019-nCOV_N Probe 0 153281 1809 98.83% 0 0 384
Japan_NIID_WH-1_F24381 Outer 151755 1856 1479 99.05% 0 1 601
Japan_NIID_WH-1_F501 Outer 152321 1804 965 99.38% 0 1 370
Japan_NIID_WH-1_F509 Outer 151285 2736 1069 99.31% 1 0 369
Japan_NIID_WH-1_Seq_F24383 Outer 151705 1890 1495 99.04% 0 0 606
Japan_NIID_WH-1_Seq_F519 Outer 151813 1536 1741 98.88% 0 1 369
Japan_WuhanCoV-spk1 Outer 152152 1440 1498 99.03% 1 1 594
Thailand_WH-NIC_N Probe 152416 2258 416 99.73% 0 2 375
Chan-N Probe 150115 3213 1762 98.86% 4 3 353
Chan-ORF1ab Probe 2 153697 1391 99.10% 0 5 2
Chan-S Probe 152099 677 2314 98.51% 5 0 2
Niu-E Probe 152976 1191 923 99.40% 26 359 0
Won-E Outer 5 154304 781 99.50% 0 384 1
Won-N Outer 149757 3865 1468 99.05% 8 7 358
Won-ORF1ab Outer 153513 745 832 99.46% 5 2 313
Won-S Outer 147646 679 6765 95.64% 5 1 0
Yip-ORF1ab Outer 152311 1110 1669 98.92% 5 1 1
Young-N Probe 0 154189 901 99.42% 0 5 1
Young-ORF1ab Probe 151420 2980 690 99.56% 5 1 0
Young-S Probe 150084 2610 2396 98.46% 4 1 0
40

Updated to 429,535 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Assay type PT TP FN Percent True PF FP TN
Noblis.12 Probe 418132 9890 1513 99.65% 0 0 680
Noblis.40 Probe 422357 6189 989 99.77% 0 1 1420
Noblis.42 Probe 425135 3027 1373 99.68% 0 2 1136
Noblis.44 Probe 423566 3726 2243 99.48% 0 1 1786
Noblis.57 Probe 420120 4728 4687 98.91% 0 1 853
ncov_e_gene Probe 425568 1533 2434 99.43% 361 15 53
ncov_n_gene Probe 412931 12717 3887 99.10% 0 340 72
ncov_rdrp_1 Probe 0 427853 1682 99.61% 433 105 5967
ncov_rdrp_2 Probe 15 428021 1499 99.65% 2 73 5916
cdc_n1 Probe 413781 14665 1089 99.75% 0 2 381
cdc_n2 Probe 413697 12867 2971 99.31% 0 1 372
cdc_n3 Probe 415446 12633 1456 99.66% 1 353 36
China_N Probe 250112 23943 155480 63.80% 0 2 691
China_ORF1ab Probe 417104 3245 9186 97.86% 1 0 367
France_nCoV_IP2 Probe 424068 3058 2409 99.44% 1 0 367
France_nCoV_IP4 Probe 423557 5197 781 99.82% 0 0 1190
HKU-N Probe 416560 9882 3093 99.28% 327 33 34
HKU-ORF1b-nsp14 Probe 425082 3616 837 99.81% 324 29 452
Japan_NIID_2019-nCOV_N Probe 0 426277 3258 99.24% 0 0 384
Japan_NIID_WH-1_F24381 Outer 421243 5282 3010 99.30% 0 1 601
Japan_NIID_WH-1_F501 Outer 422571 4379 2585 99.40% 0 1 370
Japan_NIID_WH-1_F509 Outer 419030 7716 2789 99.35% 1 0 369
Japan_NIID_WH-1_Seq_F24383 Outer 421039 5450 3046 99.29% 0 0 606
Japan_NIID_WH-1_Seq_F519 Outer 418816 4946 5773 98.66% 0 1 369
Japan_WuhanCoV-spk1 Outer 423033 3482 3020 99.30% 1 1 594
Thailand_WH-NIC_N Probe 420302 8063 1170 99.73% 0 2 375
Chan-N Probe 414334 11996 3205 99.25% 4 3 353
Chan-ORF1ab Probe 5 424942 4588 98.93% 0 5 2
Chan-S Probe 420124 3359 6052 98.59% 5 0 2
Niu-E Probe 422447 4543 2545 99.41% 26 359 0
Won-E Outer 6 427327 2202 99.49% 0 384 1
Won-N Outer 416703 9242 3590 99.16% 8 7 358
Won-ORF1ab Outer 425905 2027 1603 99.63% 5 2 313
Won-S Outer 411918 2402 15215 96.46% 5 1 0
Yip-ORF1ab Outer 421949 3155 4431 98.97% 5 1 1
Young-N Probe 1 428083 1451 99.66% 0 5 1
Young-ORF1ab Probe 420062 8220 1253 99.71% 5 1 0
Young-S Probe 367410 13478 48647 88.67% 4 1 0
C1_COV_ORF1a Probe 411789 6109 11637 97.29% 0 0 0
C2_COV_ORF1b Probe 417280 10607 1648 99.62% 0 0 0
C3_COV_ORF3a Probe 385590 31374 12571 97.07% 0 0 0
C4_COV_ORF8 Probe 371318 51082 7135 98.34% 0 0 0
C5_COV_S_gene Probe 421619 5116 2800 99.35% 0 0 0
41

Updated to 466,524 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Assay PT TP FN Percent True PF FP TN
Noblis.12 454125 10499 1900 99.59% 0 0 680
Noblis.40 458704 6644 1176 99.75% 0 1 1420
Noblis.42 461350 3393 1781 99.62% 0 2 1136
Noblis.44 459680 4097 2747 99.41% 0 1 1786
Noblis.57 455564 5232 5728 98.77% 0 1 853
ncov_e_gene 461926 1752 2846 99.39% 361 15 53
ncov_n_gene 448606 13640 4278 99.08% 0 340 72
ncov_rdrp_1 0 464006 2518 99.46% 433 105 5967
ncov_rdrp_2 16 464254 2254 99.52% 2 73 5916
cdc_n1 449212 15919 1393 99.70% 0 2 381
cdc_n2 449423 13796 3305 99.29% 0 1 372
cdc_n3 448548 16074 1902 99.59% 1 353 36
China_N 263858 28328 174338 62.63% 0 2 691
China_ORF1ab 452935 3603 9986 97.86% 1 0 367
France_nCoV_IP2 460117 3537 2870 99.38% 1 0 367
France_nCoV_IP4 459488 5955 1081 99.77% 0 0 1190
HKU-N 452261 10734 3529 99.24% 327 33 34
HKU-ORF1b-nsp14 461355 3951 1218 99.74% 324 29 452
Japan_NIID_2019-nCOV_N 0 462783 3741 99.20% 0 0 384
Japan_NIID_WH-1_F24381 456883 5750 3891 99.17% 0 1 601
Japan_NIID_WH-1_F501 458829 4663 3032 99.35% 0 1 370
Japan_NIID_WH-1_F509 455000 8305 3219 99.31% 1 0 369
Japan_NIID_WH-1_Seq_F24383 456677 5914 3933 99.16% 0 0 606
Japan_NIID_WH-1_Seq_F519 454746 5480 6298 98.65% 0 1 369
Japan_WuhanCoV-spk1 458563 4051 3910 99.16% 1 1 594
Thailand_WH-NIC_N 456149 8902 1473 99.68% 0 2 375
Chan-N 449760 13098 3666 99.21% 4 3 353
Chan-ORF1ab 5 461374 5145 98.90% 0 5 2
Chan-S 455718 3978 6828 98.54% 5 0 2
Niu-E 458497 5053 2974 99.36% 26 359 0
Won-E 7 463939 2578 99.45% 0 384 1
Won-N 452543 10040 3941 99.16% 8 7 358
Won-ORF1ab 461759 2445 2320 99.50% 5 2 313
Won-S 447723 2713 16088 96.55% 5 1 0
Yip-ORF1ab 458076 3477 4971 98.93% 5 1 1
Young-N 1 464658 1865 99.60% 0 5 1
Young-ORF1ab 456396 8564 1564 99.66% 5 1 0
Young-S 392575 15258 58691 87.42% 4 1 0
C1_COV_ORF1a 447079 6777 12668 97.28% 0 0 0
C2_COV_ORF1b 452758 11713 2053 99.56% 0 0 0
C3_COV_ORF3a 415896 37202 13426 97.12% 0 0 0
C4_COV_ORF8 397705 60858 7961 98.29% 0 0 0
C5_COV_S_gene 457226 5598 3700 99.21% 0 0 0
42

Updated to 507,323 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 494230 11101 1992 99.61% 0 0 680
Noblis.40 499141 6954 1228 99.76% 0 1 1420
Noblis.42 501587 3845 1891 99.63% 0 2 1136
Noblis.44 499889 4450 2984 99.41% 0 1 1786
Noblis.57 495554 5690 6079 98.80% 0 1 853
ncov_e_gene 502382 1902 3039 99.40% 361 15 53
ncov_n_gene 488006 14617 4700 99.07% 0 340 72
ncov_rdrp_1 0 504677 2646 99.48% 433 105 5967
ncov_rdrp_2 16 504957 2350 99.54% 2 73 5916
cdc_n1 488471 17396 1456 99.71% 0 2 381
cdc_n2 488978 14940 3405 99.33% 0 1 372
cdc_n3 488204 17106 2013 99.60% 1 353 36
China_N 279059 31342 196922 61.18% 0 2 691
China_ORF1ab 492603 4055 10665 97.90% 1 0 367
France_nCoV_IP2 500117 3999 3207 99.37% 1 0 367
France_nCoV_IP4 498969 7209 1145 99.77% 0 0 1190
HKU-N 491804 11889 3630 99.28% 327 33 34
HKU-ORF1b-nsp14 501877 4181 1265 99.75% 324 29 452
Japan_NIID_2019-nCOV_N 0 503471 3852 99.24% 0 0 384
Japan_NIID_WH-1_F24381 496921 6295 4107 99.19% 0 1 601
Japan_NIID_WH-1_F501 499044 5044 3235 99.36% 0 1 370
Japan_NIID_WH-1_F509 494915 8978 3430 99.32% 1 0 369
Japan_NIID_WH-1_Seq_F24383 496682 6491 4150 99.18% 0 0 606
Japan_NIID_WH-1_Seq_F519 494624 6040 6659 98.69% 0 1 369
Japan_WuhanCoV-spk1 498900 4290 4133 99.19% 1 1 594
Thailand_WH-NIC_N 495941 9848 1534 99.70% 0 2 375
Chan-N 489511 14036 3776 99.26% 4 3 353
Chan-ORF1ab 6 501750 5567 98.90% 0 5 2
Chan-S 495106 4758 7459 98.53% 5 0 2
Niu-E 498719 5429 3175 99.37% 26 359 0
Won-E 7 504559 2757 99.46% 0 384 1
Won-N 492335 10637 4351 99.14% 8 7 358
Won-ORF1ab 502141 2742 2440 99.52% 5 2 313
Won-S 487318 3020 16985 96.65% 5 1 0
Yip-ORF1ab 498221 3849 5253 98.96% 5 1 1
Young-N 1 505333 1989 99.61% 0 5 1
Young-ORF1ab 496665 8997 1661 99.67% 5 1 0
Young-S 411307 16905 79111 84.41% 4 1 0
C1_COV_ORF1a 485703 7396 14224 97.20% 0 0 0
C2_COV_ORF1b 492279 12891 2153 99.58% 0 0 0
C3_COV_ORF3a 450634 41962 14727 97.10% 0 0 0
C4_COV_ORF8 417307 81555 8461 98.33% 0 0 0
C5_COV_S_gene 496985 6416 3922 99.23% 0 0 0
43

Updated to 565,366 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 550666 12329 2371 99.58% 0 0 680
Noblis.40 556093 7876 1397 99.75% 0 1 1420
Noblis.42 557860 4643 2863 99.49% 0 2 1136
Noblis.44 555976 5236 4154 99.27% 0 1 1786
Noblis.57 550426 6426 8514 98.49% 0 1 853
ncov_e_gene 559501 2135 3730 99.34% 361 15 53
ncov_n_gene 543949 15931 5486 99.03% 0 340 72
ncov_rdrp_1 0 560329 5037 99.11% 433 105 5967
ncov_rdrp_2 16 560651 4699 99.17% 2 73 5916
cdc_n1 542650 19854 2862 99.49% 0 2 381
cdc_n2 542621 17193 5552 99.02% 0 1 372
cdc_n3 543000 19063 3303 99.42% 1 353 36
China_N 301247 39356 224763 60.24% 0 2 691
China_ORF1ab 547989 4790 12587 97.77% 1 0 367
France_nCoV_IP2 556064 4682 4620 99.18% 1 0 367
France_nCoV_IP4 554800 8835 1731 99.69% 0 0 1190
HKU-N 545445 14138 5783 98.98% 327 33 34
HKU-ORF1b-nsp14 558319 4608 2439 99.57% 324 29 452
Japan_NIID_2019-nCOV_N 0 559362 6004 98.94% 0 0 384
Japan_NIID_WH-1_F24381 551867 7153 6346 98.88% 0 1 601
Japan_NIID_WH-1_F501 555798 5516 4052 99.28% 0 1 370
Japan_NIID_WH-1_F509 551045 10081 4240 99.25% 1 0 369
Japan_NIID_WH-1_Seq_F24383 551598 7371 6397 98.87% 0 0 606
Japan_NIID_WH-1_Seq_F519 548791 6863 9712 98.28% 0 1 369
Japan_WuhanCoV-spk1 554212 4769 6385 98.87% 1 1 594
Thailand_WH-NIC_N 551265 11160 2941 99.48% 0 2 375
Chan-N 543747 15662 5957 98.95% 4 3 353
Chan-ORF1ab 9 558836 6521 98.85% 0 5 2
Chan-S 550675 5931 8760 98.45% 5 0 2
Niu-E 555439 6058 3869 99.32% 26 359 0
Won-E 7 561958 3401 99.40% 0 384 1
Won-N 548247 12021 5098 99.10% 8 7 358
Won-ORF1ab 557321 3243 4802 99.15% 5 2 313
Won-S 543572 3507 18287 96.77% 5 1 0
Yip-ORF1ab 554920 4405 6041 98.93% 5 1 1
Young-N 1 562014 3351 99.41% 0 5 1
Young-ORF1ab 553503 9608 2255 99.60% 5 1 0
Young-S 442944 19378 103044 81.77% 4 1 0
C1_COV_ORF1a 541039 8523 15804 97.20% 0 0 0
C2_COV_ORF1b 547608 14381 3377 99.40% 0 0 0
C3_COV_ORF3a 497339 51945 16082 97.16% 0 0 0
C4_COV_ORF8 449151 105492 10723 98.10% 0 0 0
C5_COV_S_gene 551794 7443 6129 98.92% 0 0 0
44

Updated to 612,842 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 596946 13235 2661 99.57% 0 0 680
Noblis.40 602916 8385 1541 99.75% 0 1 1420
Noblis.42 604096 5144 3602 99.41% 0 2 1136
Noblis.44 602167 5733 4942 99.19% 0 1 1786
Noblis.57 593816 7120 11906 98.06% 0 1 853
ncov_e_gene 606416 2315 4111 99.33% 361 15 53
ncov_n_gene 589936 17083 5823 99.05% 0 340 72
ncov_rdrp_1 0 604088 8754 98.57% 433 105 5967
ncov_rdrp_2 16 604437 8389 98.63% 2 73 5916
cdc_n1 586474 21564 4804 99.22% 0 2 381
cdc_n2 587540 18886 6416 98.95% 0 1 372
cdc_n3 587035 20532 5275 99.14% 1 353 36
China_N 318368 45327 249147 59.35% 0 2 691
China_ORF1ab 592185 5331 15326 97.50% 1 0 367
France_nCoV_IP2 600416 5429 6997 98.86% 1 0 367
France_nCoV_IP4 600711 10173 1958 99.68% 0 0 1190
HKU-N 590470 15731 6641 98.92% 327 33 34
HKU-ORF1b-nsp14 603677 4979 4186 99.32% 324 29 452
Japan_NIID_2019-nCOV_N 0 606145 6697 98.91% 0 0 384
Japan_NIID_WH-1_F24381 595210 7927 9705 98.42% 0 1 601
Japan_NIID_WH-1_F501 602499 6049 4294 99.30% 0 1 370
Japan_NIID_WH-1_F509 596590 11765 4487 99.27% 1 0 369
Japan_NIID_WH-1_Seq_F24383 594996 8081 9765 98.41% 0 0 606
Japan_NIID_WH-1_Seq_F519 594146 7711 10985 98.21% 0 1 369
Japan_WuhanCoV-spk1 597924 5169 9749 98.41% 1 1 594
Thailand_WH-NIC_N 595999 11955 4888 99.20% 0 2 375
Chan-N 588910 17108 6824 98.89% 4 3 353
Chan-ORF1ab 14 605287 7541 98.77% 0 5 2
Chan-S 596453 7013 9376 98.47% 5 0 2
Niu-E 601941 6656 4245 99.31% 26 359 0
Won-E 7 609067 3768 99.39% 0 384 1
Won-N 594453 12997 5392 99.12% 8 7 358
Won-ORF1ab 600687 3632 8523 98.61% 5 2 313
Won-S 589287 3934 19621 96.80% 5 1 0
Yip-ORF1ab 601244 4865 6733 98.90% 5 1 1
Young-N 1 607495 5346 99.13% 0 5 1
Young-ORF1ab 600138 10183 2521 99.59% 5 1 0
Young-S 465509 21589 125744 79.48% 4 1 0
C1_COV_ORF1a 586139 9589 17114 97.21% 0 0 0
C2_COV_ORF1b 591979 15595 5268 99.14% 0 0 0
C3_COV_ORF3a 534775 61016 17051 97.22% 0 0 0
C4_COV_ORF8 471704 127922 13216 97.84% 0 0 0
C5_COV_S_gene 595102 8416 9324 98.48% 0 0 0
45

Updated to 669,911 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Row Labels PT TP FN Percent True PF FP TN
Noblis.12 652737 14167 3007 99.55% 0 0 680
Noblis.40 659076 9152 1683 99.75% 0 1 1420
Noblis.42 660034 5746 4131 99.38% 0 2 1136
Noblis.44 657855 6362 5694 99.15% 0 1 1786
Noblis.57 648798 8136 12977 98.06% 0 1 853
ncov_e_gene 662714 2536 4661 99.30% 361 15 53
ncov_n_gene 644728 18601 6582 99.02% 0 340 72
ncov_rdrp_1 1 660834 9076 98.65% 433 105 5967
ncov_rdrp_2 17 661222 8672 98.71% 2 73 5916
cdc_n1 641063 23804 5044 99.25% 0 2 381
cdc_n2 641469 21187 7255 98.92% 0 1 372
cdc_n3 641024 23232 5655 99.16% 1 353 36
China_N 337390 53714 278807 58.38% 0 2 691
China_ORF1ab 647227 5924 16760 97.50% 1 0 367
France_nCoV_IP2 655931 6583 7397 98.90% 1 0 367
France_nCoV_IP4 655759 11939 2213 99.67% 0 0 1190
HKU-N 644999 17401 7511 98.88% 327 33 34
HKU-ORF1b-nsp14 659881 5560 4470 99.33% 324 29 452
Japan_NIID_2019-nCOV_N 0 662510 7401 98.90% 0 0 384
Japan_NIID_WH-1_F24381 650617 8728 10566 98.42% 0 1 601
Japan_NIID_WH-1_F501 658512 6520 4879 99.27% 0 1 370
Japan_NIID_WH-1_F509 651610 13233 5068 99.24% 1 0 369
Japan_NIID_WH-1_Seq_F24383 650396 8886 10629 98.41% 0 0 606
Japan_NIID_WH-1_Seq_F519 648904 8640 12367 98.15% 0 1 369
Japan_WuhanCoV-spk1 653563 5734 10614 98.42% 1 1 594
Thailand_WH-NIC_N 651646 13129 5136 99.23% 0 2 375
Chan-N 643184 18978 7749 98.84% 4 3 353
Chan-ORF1ab 24 661441 8446 98.74% 0 5 2
Chan-S 650869 8827 10215 98.48% 5 0 2
Niu-E 657851 7272 4788 99.29% 26 359 0
Won-E 7 665621 4283 99.36% 0 384 1
Won-N 649249 14520 6142 99.08% 8 7 358
Won-ORF1ab 656774 4244 8893 98.67% 5 2 313
Won-S 643506 4541 21864 96.74% 5 1 0
Yip-ORF1ab 656804 5396 7711 98.85% 5 1 1
Young-N 1 664199 5711 99.15% 0 5 1
Young-ORF1ab 656255 10854 2802 99.58% 5 1 0
Young-S 495902 24739 149270 77.72% 4 1 0
C1_COV_ORF1a 640819 10816 18276 97.27% 0 0 0
C2_COV_ORF1b 647087 17247 5577 99.17% 0 0 0
C3_COV_ORF3a 578321 73005 18585 97.23% 0 0 0
C4_COV_ORF8 504638 150508 14765 97.80% 0 0 0
C5_COV_S_gene 650533 9356 10022 98.50% 0 0 0
46

Updated to 752,038 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 732527 15800 3711 99.51% 0 0 680
Noblis.40 739757 10398 1883 99.75% 0 1 1420
Noblis.42 740307 6668 5063 99.33% 0 2 1136
Noblis.44 737939 7338 6761 99.10% 0 1 1786
Noblis.57 726361 9491 16186 97.85% 0 1 853
ncov_e_gene 743808 2907 5323 99.29% 361 15 53
ncov_n_gene 723754 20850 7434 99.01% 0 340 72
ncov_rdrp_1 1 739666 12371 98.36% 433 105 5967
ncov_rdrp_2 18 740163 11857 98.42% 2 73 5916
cdc_n1 718432 27037 6569 99.13% 0 2 381
cdc_n2 719964 24044 8030 98.93% 0 1 372
cdc_n3 718854 25880 7304 99.03% 1 353 36
China_N 369207 63326 319505 57.51% 0 2 691
China_ORF1ab 725167 7166 19705 97.38% 1 0 367
France_nCoV_IP2 734414 8080 9544 98.73% 1 0 367
France_nCoV_IP4 734686 14779 2573 99.66% 0 0 1190
HKU-N 723266 20386 8386 98.88% 327 33 34
HKU-ORF1b-nsp14 739700 6517 5821 99.23% 324 29 452
Japan_NIID_2019-nCOV_N 743740 8298 98.90% 0 0 384
Japan_NIID_WH-1_F24381 728025 10519 13494 98.21% 0 1 601
Japan_NIID_WH-1_F501 739150 7536 5352 99.29% 0 1 370
Japan_NIID_WH-1_F509 731464 15006 5568 99.26% 1 0 369
Japan_NIID_WH-1_Seq_F24383 727819 10645 13574 98.20% 0 0 606
Japan_NIID_WH-1_Seq_F519 728918 9847 13273 98.24% 0 1 369
Japan_WuhanCoV-spk1 731615 6892 13531 98.20% 1 1 594
Thailand_WH-NIC_N 730306 15058 6674 99.11% 0 2 375
Chan-N 721969 21393 8676 98.85% 4 3 353
Chan-ORF1ab 29 742643 9366 98.75% 0 5 2
Chan-S 729213 10863 11962 98.41% 5 0 2
Niu-E 737961 8640 5437 99.28% 26 359 0
Won-E 7 747123 4908 99.35% 0 384 1
Won-N 728953 16147 6938 99.08% 8 7 358
Won-ORF1ab 734716 4747 12575 98.33% 5 2 313
Won-S 722231 5920 23887 96.82% 5 1 0
Yip-ORF1ab 736795 6217 9026 98.80% 5 1 1
Young-N 1 744671 7366 99.02% 0 5 1
Young-ORF1ab 737278 11578 3182 99.58% 5 1 0
Young-S 538001 29767 184270 75.50% 4 1 0
C1_COV_ORF1a 719324 12714 20000 97.34% 0 0 0
C2_COV_ORF1b 725254 19702 7082 99.06% 0 0 0
C3_COV_ORF3a 644109 87439 20490 97.28% 0 0 0
C4_COV_ORF8 549217 184807 18014 97.60% 0 0 0
C5_COV_S_gene 728246 10728 13064 98.26% 0 0 0
47

Updated to 771,942 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Row Labels PT TP FN Percent True PF FP TN
Noblis.12 751902 16270 3770 99.51% 0 0 680
Noblis.40 759303 10714 1925 99.75% 0 1 1420
Noblis.42 759957 6846 5139 99.33% 0 2 1136
Noblis.44 757518 7538 6886 99.11% 0 1 1786
Noblis.57 745115 9858 16969 97.80% 0 1 853
ncov_e_gene 763494 2982 5466 99.29% 361 15 53
ncov_n_gene 742875 21376 7691 99.00% 0 340 72
ncov_rdrp_1 1 758766 13175 98.29% 433 105 5967
ncov_rdrp_2 18 759274 12650 98.36% 2 73 5916
cdc_n1 737065 27906 6971 99.10% 0 2 381
cdc_n2 738615 25089 8238 98.93% 0 1 372
cdc_n3 737539 26645 7758 99.00% 1 353 36
China_N 377301 67268 327373 57.59% 0 2 691
China_ORF1ab 743986 7491 20465 97.35% 1 0 367
France_nCoV_IP2 753238 8580 10124 98.69% 1 0 367
France_nCoV_IP4 753143 16135 2664 99.65% 0 0 1190
HKU-N 741887 21156 8899 98.85% 327 33 34
HKU-ORF1b-nsp14 759029 6634 6279 99.19% 324 29 452
Japan_NIID_2019-nCOV_N 0 763112 8830 98.86% 0 0 384
Japan_NIID_WH-1_F24381 746657 11058 14227 98.16% 0 1 601
Japan_NIID_WH-1_F501 758595 7816 5531 99.28% 0 1 370
Japan_NIID_WH-1_F509 750562 15625 5755 99.25% 1 0 369
Japan_NIID_WH-1_Seq_F24383 746457 11174 14311 98.15% 0 0 606
Japan_NIID_WH-1_Seq_F519 747842 10190 13910 98.20% 0 1 369
Japan_WuhanCoV-spk1 750347 7330 14265 98.15% 1 1 594
Thailand_WH-NIC_N 749358 15508 7076 99.08% 0 2 375
Chan-N 740611 22101 9230 98.80% 4 3 353
Chan-ORF1ab 29 762175 9738 98.74% 0 5 2
Chan-S 748319 11443 12180 98.42% 5 0 2
Niu-E 757456 8908 5578 99.28% 26 359 0
Won-E 7 766909 5026 99.35% 0 384 1
Won-N 747979 16776 7187 99.07% 8 7 358
Won-ORF1ab 753654 4908 13380 98.27% 5 2 313
Won-S 741106 6212 24624 96.81% 5 1 0
Yip-ORF1ab 756396 6385 9161 98.81% 5 1 1
Young-N 1 764112 7829 98.99% 0 5 1
Young-ORF1ab 756776 11889 3277 99.58% 5 1 0
Young-S 549744 31094 191104 75.24% 4 1 0
C1_COV_ORF1a 738462 13273 20207 97.38% 0 0 0
C2_COV_ORF1b 744087 20333 7522 99.03% 0 0 0
C3_COV_ORF3a 657954 92924 21064 97.27% 0 0 0
C4_COV_ORF8 561543 191450 18949 97.55% 0 0 0
C5_COV_S_gene 747066 11032 13844 98.21% 0 0 0
48

Updated to 864,129 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 840401 17638 6090 99.30% 0 0 680
Noblis.40 847976 12014 4139 99.52% 0 1 1420
Noblis.42 848605 7971 7553 99.13% 0 2 1136
Noblis.44 846143 8571 9415 98.91% 0 1 1786
Noblis.57 832185 11279 20665 97.61% 0 1 853
ncov_e_gene 852789 3380 7960 99.08% 361 15 53
ncov_n_gene 830135 23466 10528 98.78% 0 340 72
ncov_rdrp_1 1 847038 17090 98.02% 433 105 5967
ncov_rdrp_2 22 847636 16471 98.09% 2 73 5916
cdc_n1 822947 31266 9916 98.85% 0 2 381
cdc_n2 823872 29373 10884 98.74% 0 1 372
cdc_n3 823441 30000 10688 98.76% 1 353 36
China_N 399624 78797 385708 55.36% 0 2 691
China_ORF1ab 830968 8556 24605 97.15% 1 0 367
France_nCoV_IP2 839447 11228 13454 98.44% 1 0 367
France_nCoV_IP4 838177 20933 5019 99.42% 0 0 1190
HKU-N 828840 23701 11588 98.66% 327 33 34
HKU-ORF1b-nsp14 847559 7380 9190 98.94% 324 29 452
Japan_NIID_2019-nCOV_N 0 852557 11572 98.66% 0 0 384
Japan_NIID_WH-1_F24381 833745 12688 17696 97.95% 0 1 601
Japan_NIID_WH-1_F501 847012 8975 8142 99.06% 0 1 370
Japan_NIID_WH-1_F509 837530 18211 8388 99.03% 1 0 369
Japan_NIID_WH-1_Seq_F24383 833552 12776 17801 97.94% 0 0 606
Japan_NIID_WH-1_Seq_F519 835643 11636 16850 98.05% 0 1 369
Japan_WuhanCoV-spk1 837969 8421 17739 97.95% 1 1 594
Thailand_WH-NIC_N 837128 16988 10013 98.84% 0 2 375
Chan-N 827434 24657 12038 98.61% 4 3 353
Chan-ORF1ab 31 851089 13009 98.49% 0 5 2
Chan-S 835700 14176 14253 98.35% 5 0 2
Niu-E 846012 10048 8069 99.07% 26 359 0
Won-E 7 856631 7491 99.13% 0 384 1
Won-N 835339 18800 9990 98.84% 8 7 358
Won-ORF1ab 841261 5592 17276 98.00% 5 2 313
Won-S 828696 7383 28050 96.75% 5 1 0
Yip-ORF1ab 844899 7263 11967 98.62% 5 1 1
Young-N 1 853370 10758 98.76% 0 5 1
Young-ORF1ab 845510 12966 5653 99.35% 5 1 0
Young-S 584432 35469 244228 71.74% 4 1 0
C1_COV_ORF1a 823809 15490 24830 97.13% 0 0 0
C2_COV_ORF1b 830198 23372 10559 98.78% 0 0 0
C3_COV_ORF3a 731586 107574 24969 97.11% 0 0 0
C4_COV_ORF8 597823 243087 23219 97.31% 0 0 0
C5_COV_S_gene 834391 12540 17198 98.01% 0 0 0
49

Updated to 944,000 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 917690 18808 7502 99.21% 0 0 680
Noblis.40 925263 13327 5410 99.43% 0 1 1420
Noblis.42 925947 9089 8964 99.05% 0 2 1136
Noblis.44 923309 9683 11008 98.83% 0 1 1786
Noblis.57 908109 12467 23424 97.52% 0 1 853
ncov_e_gene 930642 3835 9523 98.99% 361 15 53
ncov_n_gene 906131 25387 12482 98.68% 0 340 72
ncov_rdrp_1 1 924703 19296 97.96% 433 105 5967
ncov_rdrp_2 23 925343 18634 98.03% 2 73 5916
cdc_n1 897614 34467 11919 98.74% 0 2 381
cdc_n2 899056 32529 12415 98.68% 0 1 372
cdc_n3 898435 33138 12427 98.68% 1 353 36
China_N 416515 89120 438365 53.56% 0 2 691
China_ORF1ab 906737 9591 27672 97.07% 1 0 367
France_nCoV_IP2 915221 13471 15308 98.38% 1 0 367
France_nCoV_IP4 911708 25944 6348 99.33% 0 0 1190
HKU-N 905366 25336 13298 98.59% 327 33 34
HKU-ORF1b-nsp14 925154 7911 10935 98.84% 324 29 452
Japan_NIID_2019-nCOV_N 0 930708 13292 98.59% 0 0 384
Japan_NIID_WH-1_F24381 910062 13848 20090 97.87% 0 1 601
Japan_NIID_WH-1_F501 924278 9988 9734 98.97% 0 1 370
Japan_NIID_WH-1_F509 913880 20134 9986 98.94% 1 0 369
Japan_NIID_WH-1_Seq_F24383 909878 13928 20194 97.86% 0 0 606
Japan_NIID_WH-1_Seq_F519 912289 12988 18723 98.02% 0 1 369
Japan_WuhanCoV-spk1 914607 9243 20150 97.87% 1 1 594
Thailand_WH-NIC_N 913158 18815 12027 98.73% 0 2 375
Chan-N 903351 26880 13769 98.54% 4 3 353
Chan-ORF1ab 33 928668 15299 98.38% 0 5 2
Chan-S 910227 17335 16438 98.26% 5 0 2
Niu-E 923286 11086 9628 98.98% 26 359 0
Won-E 7 934973 9020 99.04% 0 384 1
Won-N 911451 20655 11894 98.74% 8 7 358
Won-ORF1ab 918334 6222 19444 97.94% 5 2 313
Won-S 905490 8338 30172 96.80% 5 1 0
Yip-ORF1ab 922076 8036 13888 98.53% 5 1 1
Young-N 1 931457 12542 98.67% 0 5 1
Young-ORF1ab 923202 13819 6979 99.26% 5 1 0
Young-S 612115 39958 291927 69.08% 4 1 0
C1_COV_ORF1a 898984 17613 27403 97.10% 0 0 0
C2_COV_ORF1b 905522 26105 12373 98.69% 0 0 0
C3_COV_ORF3a 797075 119259 27666 97.07% 0 0 0
C4_COV_ORF8 628777 289062 26161 97.23% 0 0 0
C5_COV_S_gene 910578 13726 19696 97.91% 0 0 0
50

Updated to 1,057,192 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt).

Algorithm updated 4/13/2021 to penalize Ns on the subject sequence.

assay PT TP FN Percent True PF FP TN
Noblis.12 1028325 20542 8325 99.21% 0 0 680
Noblis.40 1035399 16020 5773 99.45% 0 1 1420
Noblis.42 1037142 10391 9659 99.09% 0 2 1136
Noblis.44 1034163 11064 11965 98.87% 0 1 1786
Noblis.57 1015499 14692 27001 97.45% 0 1 853
ncov_e_gene 1042085 4344 10763 98.98% 361 15 53
ncov_n_gene 1015226 28772 13194 98.75% 0 340 72
ncov_rdrp_1 1 1035980 21211 97.99% 433 105 5967
ncov_rdrp_2 27 1036687 20478 98.06% 2 73 5916
cdc_n1 1004834 39319 13039 98.77% 0 2 381
cdc_n2 1007030 37156 13006 98.77% 0 1 372
cdc_n3 1005515 38103 13574 98.72% 1 353 36
China_N 434260 106553 516379 51.16% 0 2 691
China_ORF1ab 1016268 11093 29831 97.18% 1 0 367
France_nCoV_IP2 1024226 16518 16448 98.44% 1 0 367
France_nCoV_IP4 1016609 33520 7063 99.33% 0 0 1190
HKU-N 1014389 28203 14600 98.62% 327 33 34
HKU-ORF1b-nsp14 1035829 9211 12152 98.85% 324 29 452
Japan_NIID_2019-nCOV_N 0 1042146 15046 98.58% 0 0 384
Japan_NIID_WH-1_F24381 1015334 15551 26307 97.51% 0 1 601
Japan_NIID_WH-1_F501 1035167 10984 11041 98.96% 0 1 370
Japan_NIID_WH-1_F509 1023681 22161 11350 98.93% 1 0 369
Japan_NIID_WH-1_Seq_F24383 1015160 15621 26411 97.50% 0 0 606
Japan_NIID_WH-1_Seq_F519 1018188 14533 24471 97.69% 0 1 369
Japan_WuhanCoV-spk1 1020629 10209 26354 97.51% 1 1 594
Thailand_WH-NIC_N 1023410 20626 13156 98.76% 0 2 375
Chan-N 1012118 30301 14773 98.60% 4 3 353
Chan-ORF1ab 38 1040042 17112 98.38% 0 5 2
Chan-S 1018292 21245 17655 98.33% 5 0 2
Niu-E 1034200 12161 10831 98.98% 26 359 0
Won-E 7 1047277 9908 99.06% 0 384 1
Won-N 1020160 23987 13045 98.77% 8 7 358
Won-ORF1ab 1028461 7279 21452 97.97% 5 2 313
Won-S 1015126 9670 32396 96.94% 5 1 0
Yip-ORF1ab 1033026 9446 14720 98.61% 5 1 1
Young-N 1 1043455 13736 98.70% 0 5 1
Young-ORF1ab 1034522 14998 7672 99.27% 5 1 0
Young-S 654700 45248 357244 66.21% 4 1 0
C1_COV_ORF1a 1007061 20749 29382 97.22% 0 0 0
C2_COV_ORF1b 1014649 29291 13252 98.75% 0 0 0
C3_COV_ORF3a 893126 134309 29757 97.19% 0 0 0
C4_COV_ORF8 674434 354219 28539 97.30% 0 0 0
C5_COV_S_gene 1018242 15342 23608 97.77% 0 0 0
51

Updated to 1,228,199 COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 04/26/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1195139 22765 914 99.93%
Noblis.40 1199037 22097 641 99.95%
Noblis.42 1204605 12295 341 99.97%
Noblis.44 1201137 13011 732 99.94%
Noblis.57 1175565 18669 1305 99.89%
ncov_e_gene 1209760 5025 1661 99.86%
ncov_n_gene 1179744 32628 976 99.92%
ncov_rdrp_1 0 1204176 1183 99.90%
ncov_rdrp_2 33 1204985 377 99.97%
cdc_n1 1166643 46652 165 99.99%
cdc_n2 1168898 43869 561 99.95%
cdc_n3 1167043 45309 597 99.95%
China_N 455092 132114 623915 48.48%
China_ORF1ab 1179747 13446 1854 99.84%
France_nCoV_IP2 1188667 21067 688 99.94%
France_nCoV_IP4 1163929 55175 875 99.93%
HKU-N 1179262 31843 666 99.95%
HKU-ORF1b-nsp14 1202615 10956 406 99.97%
Japan_NIID_2019-nCOV_N 0 1210503 904 99.93%
Japan_NIID_WH-1_F24381 1176945 18071 6170 99.49%
Japan_NIID_WH-1_F501 1201666 13439 735 99.94%
Japan_NIID_WH-1_F509 1187262 27454 1178 99.90%
Japan_NIID_WH-1_Seq_F24383 1176691 18167 6393 99.47%
Japan_NIID_WH-1_Seq_F519 1177642 18775 19501 98.40%
Japan_WuhanCoV-spk1 1183035 11961 5827 99.51%
Thailand_WH-NIC_N 1190244 22940 206 99.98%
Chan-N 1175743 35062 1023 99.92%
Chan-ORF1ab 42 1205868 1425 99.94%
Chan-S 1177715 29737 3652 99.70%
Niu-E 1200461 14190 665 99.95%
Won-E 7 1216111 840 99.93%
Won-N 1184667 27857 495 99.96%
Won-ORF1ab 1194309 9288 423 99.96%
Won-S 1179638 11385 10531 99.12%
Yip-ORF1ab 1199321 11457 3608 99.70%
Young-N 1 1211927 345 99.97%
Young-ORF1ab 1201332 17282 639 99.95%
Young-S 703119 53672 447080 62.86%
C1_COV_ORF1a 1170522 24631 1597 99.87%
C2_COV_ORF1b 1179752 33446 510 99.96%
C3_COV_ORF3a 1042529 151375 4831 99.60%
C4_COV_ORF8 728696 466365 2645 99.78%
C5_COV_S_gene 1179784 18269 1241 99.90%
52

Updated to 1,011,071 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/04/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 987355 18700 214 99.98%
Noblis.40 986919 19473 90 99.99%
Noblis.42 996623 9734 25 100.00%
Noblis.44 995983 10284 33 100.00%
Noblis.57 993041 14943 102 99.99%
ncov_e_gene 1002017 3982 215 99.98%
ncov_n_gene 979433 26322 150 99.99%
ncov_rdrp_1 0 1005540 895 99.91%
ncov_rdrp_2 28 1006038 379 99.96%
cdc_n1 970106 36024 12 100.00%
cdc_n2 969226 36311 76 99.99%
cdc_n3 968441 37759 16 100.00%
China_N 350658 101313 552332 45.00%
China_ORF1ab 998615 7539 147 99.99%
France_nCoV_IP2 988320 17919 87 99.99%
France_nCoV_IP4 955144 51211 131 99.99%
HKU-N 979152 26143 62 99.99%
HKU-ORF1b-nsp14 997478 8909 62 99.99%
Japan_NIID_2019-nCOV_N 2 1005174 192 99.98%
Japan_NIID_WH-1_F24381 993111 14549 230 99.98%
Japan_NIID_WH-1_F501 994233 11753 74 99.99%
Japan_NIID_WH-1_F509 984296 21542 237 99.98%
Japan_NIID_WH-1_Seq_F24383 992903 13469 1538 99.85%
Japan_NIID_WH-1_Seq_F519 982455 15391 8244 99.18%
Japan_WuhanCoV-spk1 998282 9367 203 99.98%
Thailand_WH-NIC_N 987637 18482 66 99.99%
Chan-N 976309 28854 140 99.99%
Chan-ORF1ab 38 1005830 64 100.00%
Chan-S 980269 24956 404 99.96%
Niu-E 995796 10194 227 99.98%
Won-E 5 1006048 156 99.98%
Won-N 983658 22161 21 100.00%
Won-ORF1ab 998483 7934 12 100.00%
Won-S 995881 8201 689 99.93%
Yip-ORF1ab 996891 8979 62 99.99%
Young-N 1 1006230 47 100.00%
Young-ORF1ab 992298 13789 357 99.96%
Young-S 551310 42652 406416 59.37%
C1_COV_ORF1a 984120 17876 176 99.98%
C2_COV_ORF1b 978638 27732 19 100.00%
C3_COV_ORF3a 886664 117684 337 99.97%
C4_COV_ORF8 574988 428122 1573 99.84%
C5_COV_S_gene 993736 14337 82 99.99%
53

Updated to 1,081,736 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/7/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1055483 21103 220 99.98%
Noblis.40 1054779 22150 92 99.99%
Noblis.42 1066452 10448 25 100.00%
Noblis.44 1065821 10982 36 100.00%
Noblis.57 1062626 15899 103 99.99%
ncov_e_gene 1072331 4201 216 99.98%
ncov_n_gene 1048853 27418 152 99.99%
ncov_rdrp_1 0 1075806 1172 99.89%
ncov_rdrp_2 31 1076322 635 99.94%
cdc_n1 1038877 37797 12 100.00%
cdc_n2 1038361 37699 76 99.99%
cdc_n3 1036012 40716 16 100.00%
China_N 358119 106799 609811 43.26%
China_ORF1ab 1067798 8839 160 99.99%
France_nCoV_IP2 1058123 18653 90 99.99%
France_nCoV_IP4 1018659 58223 148 99.99%
HKU-N 1048396 27363 69 99.99%
HKU-ORF1b-nsp14 1067524 9406 64 99.99%
Japan_NIID_2019-nCOV_N 2 1075641 204 99.98%
Japan_NIID_WH-1_F24381 1062677 15456 234 99.98%
Japan_NIID_WH-1_F501 1063930 12551 75 99.99%
Japan_NIID_WH-1_F509 1053255 23074 242 99.98%
Japan_NIID_WH-1_Seq_F24383 1062389 14339 1659 99.85%
Japan_NIID_WH-1_Seq_F519 1050519 16955 9112 99.15%
Japan_WuhanCoV-spk1 1067642 10481 205 99.98%
Thailand_WH-NIC_N 1057475 19183 71 99.99%
Chan-N 1045151 30480 151 99.99%
Chan-ORF1ab 41 1076321 73 100.00%
Chan-S 1048072 27596 417 99.96%
Niu-E 1065378 11145 229 99.98%
Won-E 5 1076580 157 99.99%
Won-N 1052374 23961 23 100.00%
Won-ORF1ab 1068015 8946 12 100.00%
Won-S 1065942 8601 721 99.93%
Yip-ORF1ab 1066864 9538 63 99.99%
Young-N 1 1076761 48 100.00%
Young-ORF1ab 1061627 15000 358 99.97%
Young-S 563864 44899 461606 56.87%
C1_COV_ORF1a 1053672 18772 176 99.98%
C2_COV_ORF1b 1047179 29732 19 100.00%
C3_COV_ORF3a 953272 121543 346 99.97%
C4_COV_ORF8 589965 483426 1713 99.84%
C5_COV_S_gene 1063209 15402 84 99.99%
54

Updated to 1,185,134 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/17/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1155724 22678 222 99.98%
Noblis.40 1150262 28522 94 99.99%
Noblis.42 1167331 11396 31 100.00%
Noblis.44 1166570 12052 40 100.00%
Noblis.57 1163197 18119 110 99.99%
ncov_e_gene 1173766 4558 224 99.98%
ncov_n_gene 1146701 31317 174 99.99%
ncov_rdrp_1 0 1177536 1296 99.89%
ncov_rdrp_2 35 1178118 689 99.94%
cdc_n1 1137338 41167 19 100.00%
cdc_n2 1136718 41150 87 99.99%
cdc_n3 1134039 44491 18 100.00%
China_N 363658 118928 693614 41.03%
China_ORF1ab 1169105 9325 181 99.98%
France_nCoV_IP2 1157622 20989 95 99.99%
France_nCoV_IP4 1105699 72951 227 99.98%
HKU-N 1147909 29662 72 99.99%
HKU-ORF1b-nsp14 1167920 10863 64 99.99%
Japan_NIID_2019-nCOV_N 2 1177439 225 99.98%
Japan_NIID_WH-1_F24381 1164169 16723 254 99.98%
Japan_NIID_WH-1_F501 1164183 14129 79 99.99%
Japan_NIID_WH-1_F509 1152098 26057 253 99.98%
Japan_NIID_WH-1_Seq_F24383 1163797 15460 1909 99.84%
Japan_NIID_WH-1_Seq_F519 1147928 19471 11027 99.06%
Japan_WuhanCoV-spk1 1169458 11424 225 99.98%
Thailand_WH-NIC_N 1157768 20722 77 99.99%
Chan-N 1143755 33683 159 99.99%
Chan-ORF1ab 41 1178087 94 100.00%
Chan-S 1144316 34184 431 99.96%
Niu-E 1166294 12021 238 99.98%
Won-E 5 1178378 161 99.99%
Won-N 1150088 28002 29 100.00%
Won-ORF1ab 1167085 11730 12 100.00%
Won-S 1167839 9321 799 99.93%
Yip-ORF1ab 1167330 10843 71 99.99%
Young-N 1 1178564 55 100.00%
Young-ORF1ab 1161867 16601 363 99.97%
Young-S 590426 48252 533237 54.50%
C1_COV_ORF1a 1153187 20968 185 99.98%
C2_COV_ORF1b 1146761 32002 21 100.00%
C3_COV_ORF3a 1048420 128076 367 99.97%
C4_COV_ORF8 618586 556050 2047 99.83%
C5_COV_S_gene 1164343 17055 91 99.99%
55

Updated to 1,323,492 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/28/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1288871 25594 252 99.98%
Noblis.40 1278510 36349 144 99.99%
Noblis.42 1301684 13098 47 100.00%
Noblis.44 1300869 13816 65 100.00%
Noblis.57 1296740 20704 152 99.99%
ncov_e_gene 1309334 5008 269 99.98%
ncov_n_gene 1280038 33929 245 99.98%
ncov_rdrp_1 0 1313413 1554 99.88%
ncov_rdrp_2 47 1314043 877 99.93%
cdc_n1 1268555 46056 52 100.00%
cdc_n2 1268144 45377 167 99.99%
cdc_n3 1263742 51017 36 100.00%
China_N 376532 139139 798024 39.25%
China_ORF1ab 1303882 10515 196 99.99%
France_nCoV_IP2 1290867 23801 120 99.99%
France_nCoV_IP4 1221942 92737 285 99.98%
HKU-N 1280481 32751 285 99.98%
HKU-ORF1b-nsp14 1302598 12251 80 99.99%
Japan_NIID_2019-nCOV_N 2 1313059 308 99.98%
Japan_NIID_WH-1_F24381 1298051 18975 319 99.98%
Japan_NIID_WH-1_F501 1298268 16048 154 99.99%
Japan_NIID_WH-1_F509 1284458 29690 331 99.97%
Japan_NIID_WH-1_Seq_F24383 1297522 17690 2136 99.84%
Japan_NIID_WH-1_Seq_F519 1279397 22826 12243 99.07%
Japan_WuhanCoV-spk1 1304034 13004 286 99.98%
Thailand_WH-NIC_N 1291789 22782 95 99.99%
Chan-N 1275309 37758 234 99.98%
Chan-ORF1ab 42 1314058 241 99.99%
Chan-S 1272603 44049 1399 99.89%
Niu-E 1300602 13739 235 99.98%
Won-E 5 1314408 298 99.98%
Won-N 1283023 31015 109 99.99%
Won-ORF1ab 1295236 19657 25 100.00%
Won-S 1303465 10906 2532 99.81%
Yip-ORF1ab 1301352 12816 388 99.97%
Young-N 1 1314514 99 99.99%
Young-ORF1ab 1296251 18274 398 99.97%
Young-S 626128 54307 628418 51.99%
C1_COV_ORF1a 1286258 23711 224 99.98%
C2_COV_ORF1b 1279702 35114 51 100.00%
C3_COV_ORF3a 1172160 140157 661 99.95%
C4_COV_ORF8 660926 648983 2512 99.81%
C5_COV_S_gene 1297856 19658 175 99.99%
56

Updated to 1,415,475 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 06/08/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1379006 27207 259 99.98%
Noblis.40 1364879 41763 145 99.99%
Noblis.42 1392454 14102 53 100.00%
Noblis.44 1391559 14897 71 99.99%
Noblis.57 1387069 22129 161 99.99%
ncov_e_gene 1400826 5278 274 99.98%
ncov_n_gene 1367007 38182 287 99.98%
ncov_rdrp_1 0 1405108 1636 99.88%
ncov_rdrp_2 48 1405778 918 99.93%
cdc_n1 1357685 48617 57 100.00%
cdc_n2 1357351 47842 176 99.99%
cdc_n3 1351961 54524 40 100.00%
China_N 381708 155568 867964 38.23%
China_ORF1ab 1394965 11190 207 99.99%
France_nCoV_IP2 1380683 25745 129 99.99%
France_nCoV_IP4 1305290 101141 317 99.98%
HKU-N 1370215 34660 300 99.98%
HKU-ORF1b-nsp14 1393606 13023 85 99.99%
Japan_NIID_2019-nCOV_N 3 1404692 320 99.98%
Japan_NIID_WH-1_F24381 1388298 20391 343 99.98%
Japan_NIID_WH-1_F501 1388322 17644 164 99.99%
Japan_NIID_WH-1_F509 1373840 31944 357 99.97%
Japan_NIID_WH-1_Seq_F24383 1387693 18953 2389 99.83%
Japan_NIID_WH-1_Seq_F519 1368618 24802 12710 99.10%
Japan_WuhanCoV-spk1 1394779 13924 309 99.98%
Thailand_WH-NIC_N 1382094 24172 102 99.99%
Chan-N 1364669 40026 245 99.98%
Chan-ORF1ab 47 1405751 259 99.99%
Chan-S 1357925 50498 1499 99.89%
Niu-E 1391685 14414 243 99.98%
Won-E 5 1406170 310 99.98%
Won-N 1369388 35891 120 99.99%
Won-ORF1ab 1375505 31165 25 100.00%
Won-S 1394416 11587 2635 99.81%
Yip-ORF1ab 1391831 14076 400 99.97%
Young-N 1 1406187 111 99.99%
Young-ORF1ab 1385853 20446 403 99.97%
Young-S 654144 56697 688394 50.80%
C1_COV_ORF1a 1376415 25134 248 99.98%
C2_COV_ORF1b 1368614 37976 58 100.00%
C3_COV_ORF3a 1258645 145336 691 99.95%
C4_COV_ORF8 691359 709972 2624 99.81%
C5_COV_S_gene 1387728 21552 181 99.99%
57

Updated to 1,467,158 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 06/15/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1428580 28171 261 99.98%
Noblis.40 1410187 46998 146 99.99%
Noblis.42 1442321 14775 54 100.00%
Noblis.44 1441381 15606 75 99.99%
Noblis.57 1436632 23166 172 99.99%
ncov_e_gene 1451168 5450 284 99.98%
ncov_n_gene 1416576 39005 300 99.98%
ncov_rdrp_1 0 1455502 1782 99.88%
ncov_rdrp_2 53 1456202 1029 99.93%
cdc_n1 1406276 50549 59 100.00%
cdc_n2 1406225 49444 180 99.99%
cdc_n3 1400673 56336 42 100.00%
China_N 385695 171479 898540 38.27%
China_ORF1ab 1445057 11579 224 99.98%
France_nCoV_IP2 1429891 27059 137 99.99%
France_nCoV_IP4 1350568 106392 327 99.98%
HKU-N 1419280 36071 303 99.98%
HKU-ORF1b-nsp14 1443698 13470 88 99.99%
Japan_NIID_2019-nCOV_N 3 1455163 327 99.98%
Japan_NIID_WH-1_F24381 1438097 21126 362 99.98%
Japan_NIID_WH-1_F501 1438212 18234 172 99.99%
Japan_NIID_WH-1_F509 1422997 33253 383 99.97%
Japan_NIID_WH-1_Seq_F24383 1437450 19715 2422 99.83%
Japan_NIID_WH-1_Seq_F519 1417852 25623 13148 99.10%
Japan_WuhanCoV-spk1 1444755 14483 329 99.98%
Thailand_WH-NIC_N 1431790 25005 104 99.99%
Chan-N 1413915 41254 247 99.98%
Chan-ORF1ab 52 1456204 272 99.99%
Chan-S 1402833 56135 1541 99.89%
Niu-E 1441802 14812 250 99.98%
Won-E 5 1456689 324 99.98%
Won-N 1418985 36687 125 99.99%
Won-ORF1ab 1414489 42714 27 100.00%
Won-S 1445481 11994 2752 99.81%
Yip-ORF1ab 1441625 14801 411 99.97%
Young-N 1 1456685 115 99.99%
Young-ORF1ab 1435631 21200 405 99.97%
Young-S 675594 58563 712969 50.73%
C1_COV_ORF1a 1425815 26184 254 99.98%
C2_COV_ORF1b 1418115 38997 73 99.99%
C3_COV_ORF3a 1305249 149194 703 99.95%
C4_COV_ORF8 716173 735316 2789 99.81%
C5_COV_S_gene 1437520 22364 193 99.99%