{"id":465554,"date":"2024-10-20T10:41:06","date_gmt":"2024-10-20T10:41:06","guid":{"rendered":"https:\/\/pdfstandards.shop\/product\/uncategorized\/ieee-3168-2024-4\/"},"modified":"2024-10-26T19:42:28","modified_gmt":"2024-10-26T19:42:28","slug":"ieee-3168-2024-4","status":"publish","type":"product","link":"https:\/\/pdfstandards.shop\/product\/publishers\/ieee\/ieee-3168-2024-4\/","title":{"rendered":"IEEE 3168-2024"},"content":{"rendered":"

New IEEE Standard – Active. The natural language processing (NLP) services using machine learning have rich applications in solving various tasks and have been widely deployed and used, usually accessible by application programming interface (API) calls. The robustness of the NLP services is challenged by various well-known general corruptions and adversarial attacks. Inadvertent or random deletion, addition, or repetition of characters or words are examples of general corruptions. Adversarial characters, words, or sentence samples are generated by adversarial attacks, causing the models underpinning the NLP services to produce incorrect results. A method for quantitatively evaluating the robustness the NLP services is proposed by this standard. Under the method, different cases the evaluation needs to perform against are specified. Robustness metrics and their calculation are defined. With the standard, understanding of the robustness of the services can be developed by the service stakeholders including the service developer, service providers, and service users. The evaluation can be performed during various phases in the life cycle of the NLP services, the testing phase, in the validation phase, after deployment, and so forth.<\/p>\n

PDF Catalog<\/h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PDF Pages<\/th>\nPDF Title<\/th>\n<\/tr>\n
1<\/td>\nIEEE Std 3168\u2122-2024 Front cover <\/td>\n<\/tr>\n
2<\/td>\nTitle page <\/td>\n<\/tr>\n
4<\/td>\nImportant Notices and Disclaimers Concerning IEEE Standards Documents <\/td>\n<\/tr>\n
8<\/td>\nParticipants <\/td>\n<\/tr>\n
9<\/td>\nIntroduction <\/td>\n<\/tr>\n
10<\/td>\nContents <\/td>\n<\/tr>\n
11<\/td>\n1.\u2002Overview
1.1\u2002Scope
1.2\u2002Purpose
1.3\u2002Word usage <\/td>\n<\/tr>\n
12<\/td>\n2.\u2002Normative references
3.\u2002Definitions, acronyms, and abbreviations
3.1\u2002Definitions
3.2\u2002Acronyms and abbreviations <\/td>\n<\/tr>\n
13<\/td>\n4.\u2002Evaluation target
5.\u2002Evaluation cases for NLP services
5.1\u2002Overview of evaluation cases <\/td>\n<\/tr>\n
14<\/td>\n5.2\u2002General corruptions <\/td>\n<\/tr>\n
15<\/td>\n5.3\u2002Adversarial attacks
6.\u2002Robustness metrics of NLP services
6.1\u2002Metrics overview <\/td>\n<\/tr>\n
16<\/td>\n6.2\u2002Utility metrics
6.3\u2002Corruption resistant metrics
6.4\u2002Adversarial resistant metrics <\/td>\n<\/tr>\n
17<\/td>\n6.5\u2002Quality metrics
6.6\u2002Metrics calculation for NLP services <\/td>\n<\/tr>\n
24<\/td>\n7.\u2002Test cases
7.1\u2002Test cases for utility metrics <\/td>\n<\/tr>\n
25<\/td>\n7.2\u2002Test cases for general corruption
7.3\u2002Test cases for adversarial attacks <\/td>\n<\/tr>\n
27<\/td>\nAnnex\u00a0A (Informative) Defense against adversarial attacks <\/td>\n<\/tr>\n
28<\/td>\nAnnex\u00a0B (Informative) Bibliography <\/td>\n<\/tr>\n
29<\/td>\nBack cover <\/td>\n<\/tr>\n<\/table>\n","protected":false},"excerpt":{"rendered":"

IEEE Standard for Robustness Evaluation Test Methods for a Natural Language Processing Service That Uses Machine Learning (Approved Draft)<\/b><\/p>\n\n\n\n\n
Published By<\/td>\nPublication Date<\/td>\nNumber of Pages<\/td>\n<\/tr>\n
IEEE<\/b><\/a><\/td>\n2024<\/td>\n29<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"featured_media":465564,"template":"","meta":{"rank_math_lock_modified_date":false,"ep_exclude_from_search":false},"product_cat":[2644],"product_tag":[],"class_list":{"0":"post-465554","1":"product","2":"type-product","3":"status-publish","4":"has-post-thumbnail","6":"product_cat-ieee","8":"first","9":"instock","10":"sold-individually","11":"shipping-taxable","12":"purchasable","13":"product-type-simple"},"_links":{"self":[{"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/product\/465554","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/product"}],"about":[{"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/types\/product"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/media\/465564"}],"wp:attachment":[{"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/media?parent=465554"}],"wp:term":[{"taxonomy":"product_cat","embeddable":true,"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/product_cat?post=465554"},{"taxonomy":"product_tag","embeddable":true,"href":"https:\/\/pdfstandards.shop\/wp-json\/wp\/v2\/product_tag?post=465554"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}