{"corpus_id":23046847,"paper_sha":"6b3ec7f669025c7725695cf8e337ff8b4cb57a65","doi":"10.1109/ACCESS.2017.2649838","arxiv_id":null,"pmid":null,"pmcid":null,"mag_id":2570749312,"dblp_id":"journals/access/LinYLLL17","acl_id":null,"title":"Development of Novel Lip-Reading Recognition Algorithm","year":2017,"publication_date":null,"venue":"IEEE Access","journal":{"name":"IEEE Access","pages":"794-801","volume":"5"},"journal_issn":null,"journal_title":null,"publication_types":["JournalArticle"],"pubmed_pub_types":null,"s2_fields_of_study":["Medicine","Computer Science","Engineering"],"reference_count":26,"citation_count":21,"influential_citation_count":0,"is_open_access":false,"arxiv_categories":null,"arxiv_license":null,"arxiv_journal_ref":null,"mesh_headings":null,"chemicals":null,"comments_corrections":null,"source_flags":1,"s2_open_access_pdf_url":null,"s2_open_access_landing_url":null,"s2_open_access_license":null,"s2_open_access_status":null,"pmc_open_access_pdf_url":null,"pmc_open_access_landing_url":null,"pmc_open_access_license":null,"pmc_open_access_status":null,"unpaywall_open_access_pdf_url":null,"unpaywall_open_access_landing_url":null,"unpaywall_open_access_license":null,"unpaywall_open_access_status":null,"abstract":"Total laryngectomy is a common treatment for patients with advanced laryngeal and hypopharyngeal cancer, but it is also a result from the loss of the natural voice and directly affects the basic communication functions in daily life. Reconstructing the basic communication function is an important issue for these patients after total laryngectomy surgery. Recently, the image processing technique for lip-reading recognition has been widely developed and applied in various kinds of applications. It is also one of the possibly alternative approaches to reconstructing the basic communication function for these patients after total laryngectomy surgery. Although many human lip-reading recognition methods have been developed to detect lip contour precisely, detecting pronouncing lip contour effectively is still a difficult challenge. In this paper, a novel lip-reading recognition algorithm was proposed to recognize English vowels from the lip contour when speaking. Here, several criteria for detecting the mouth region of interest (ROI) were designed to reduce the error rate of detecting the mouth ROI and lip contour. Moreover, several lip parameters, including the width, height, contour points, area, and the ratio (width/height) of lips, were used to recognize the lip contour and English vowels when speaking. The advantages of the proposed method are that it could detect the mouth ROI automatically, reduce the influence of individual differences, such as the individual lip shape or makeup effect, and it also could perform a good performance without pretraining. Finally, the performance of lip-reading recognition under different backgrounds and individual differences was also tested, and the accuracy of the proposed algorithm on lip-reading recognition was over 80%.","claims":[{"public_id":"cl_d4bfd932345093d47a058a0af83e6681","status":"active","text":"A novel lip-reading recognition algorithm was proposed to recognize English vowels from lip contour during speech.","confidence":0.98,"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_d4bfd932345093d47a058a0af83e6681"},{"public_id":"cl_25d4183251969b7a63f7870f5e391155","status":"active","text":"Lip width, height, contour points, area, and the width-to-height ratio were used as features for lip-contour and English-vowel recognition.","confidence":0.96,"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_25d4183251969b7a63f7870f5e391155"},{"public_id":"cl_bf094013d5baeff8d89829d328fe3531","status":"active","text":"Lip-reading recognition accuracy was over 80% under different backgrounds and individual differences.","confidence":0.93,"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_bf094013d5baeff8d89829d328fe3531"},{"public_id":"cl_baa1376dc87d2999f42296f047748b40","status":"active","text":"Several criteria for detecting the mouth region of interest were designed to reduce mouth ROI and lip-contour detection errors.","confidence":0.92,"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_baa1376dc87d2999f42296f047748b40"},{"public_id":"cl_95f23da7ee6b692d10c8cc0afe305175","status":"active","text":"The method automatically detects the mouth region of interest, reduces the influence of individual differences such as lip shape and makeup, and performs well without pretraining.","confidence":0.95,"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_95f23da7ee6b692d10c8cc0afe305175"}],"concepts":[{"public_id":"co_15952f02046ccc87872e0d7dd6170726","status":"active","name":"width/height ratio","description":"The ratio of lip width to lip height used as a shape descriptor.","types":["measurement"],"aliases":["width-to-height ratio","lip ratio"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_15952f02046ccc87872e0d7dd6170726"},{"public_id":"co_1e8602294859e321de24c69d305308f4","status":"active","name":"individual differences","description":"Person-specific variations such as lip shape or makeup that affect visual recognition.","types":["confounding factor"],"aliases":["person-specific differences"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_1e8602294859e321de24c69d305308f4"},{"public_id":"co_20bbd7c0b6af27f6f56cc8405c0b8b3f","status":"active","name":"lip contour","description":"The outline and shape of the lips used as visual information for recognition.","types":["visual feature"],"aliases":["lip contour shape"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_20bbd7c0b6af27f6f56cc8405c0b8b3f"},{"public_id":"co_595b5162a3067761963638ae815ea466","status":"active","name":"width","description":"The horizontal extent of the lips used as a lip parameter.","types":["measurement"],"aliases":["lip width"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_595b5162a3067761963638ae815ea466"},{"public_id":"co_5da6563f573d33ba2cc88c2bfe8c7716","status":"active","name":"criteria for detecting the mouth region of interest","description":"Rules or conditions used to locate the mouth region accurately in the image.","types":["method component"],"aliases":["mouth ROI detection criteria"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_5da6563f573d33ba2cc88c2bfe8c7716"},{"public_id":"co_747390b0c2d62c7ec78d684db9480dcb","status":"active","name":"contour points","description":"Specific points sampled along the lip outline for describing its shape.","types":["measurement"],"aliases":["lip contour points"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_747390b0c2d62c7ec78d684db9480dcb"},{"public_id":"co_a0bc0bd1cb9e6cac4b998eae22705d5c","status":"active","name":"pretraining","description":"Prior training of the recognition model before use on the target task.","types":["training procedure"],"aliases":["pre-training"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_a0bc0bd1cb9e6cac4b998eae22705d5c"},{"public_id":"co_a616209fcd28174184264916d0557fae","status":"active","name":"area","description":"The measured area enclosed by the lip contour.","types":["measurement"],"aliases":["lip area"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_a616209fcd28174184264916d0557fae"},{"public_id":"co_a8ba31189d5e436137a66cdd342f6fff","status":"active","name":"different backgrounds","description":"Varying background scenes used to test robustness of the recognition algorithm.","types":["evaluation condition"],"aliases":["background variations"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_a8ba31189d5e436137a66cdd342f6fff"},{"public_id":"co_c9439bf10d6d88f33a8ebd742a52a96b","status":"active","name":"mouth region of interest","description":"The image region around the mouth selected for lip detection and recognition.","types":["image region"],"aliases":["mouth ROI","ROI"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_c9439bf10d6d88f33a8ebd742a52a96b"},{"public_id":"co_cc4263d996f017338ad02b65f6bf00af","status":"active","name":"lip parameters","description":"Quantitative measurements extracted from the lips for recognition, including geometric descriptors.","types":["feature set"],"aliases":["lip features"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_cc4263d996f017338ad02b65f6bf00af"},{"public_id":"co_dd75c0078aed7169543ce0cb346b6ae7","status":"active","name":"lip-reading recognition algorithm","description":"An image-processing algorithm that infers spoken content from visual mouth and lip movement.","types":["method"],"aliases":["lip-reading algorithm","lip-reading recognition method"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_dd75c0078aed7169543ce0cb346b6ae7"},{"public_id":"co_f075ffa3686b29c398b1bf9f1530b7c9","status":"active","name":"height","description":"The vertical extent of the lips used as a lip parameter.","types":["measurement"],"aliases":["lip height"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_f075ffa3686b29c398b1bf9f1530b7c9"},{"public_id":"co_fa5f18c0d319b2671c6a26327442f1d1","status":"active","name":"English vowels","description":"The set of English vowel sounds targeted for recognition from lip movement.","types":["linguistic unit","target class"],"aliases":["vowels"],"contributors":[{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["extraction"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_fa5f18c0d319b2671c6a26327442f1d1"}],"external_ids":{"DOI":"10.1109/ACCESS.2017.2649838","ArXiv":null,"PubMed":null,"PubMedCentral":null,"MAG":2570749312,"DBLP":"journals/access/LinYLLL17","ACL":null},"open_access":{"is_open_access":false,"pdf_url":null,"landing_url":"https://sah.borca.ai/papers/23046847","source":null,"pdf_url_source":null,"license":null,"reason":"pdf_url_not_indexed"},"reference_availability":{"status":"available","references_indexed":true,"full_text_available":false,"full_text_source":null,"count_basis":"semantic_scholar_metadata","extraction_status":"not_applicable","reason":null},"source":{"provider":"episteme2","base_corpus":"semantic_scholar_dump","freshness_mode":"unknown","basis":["semantic_scholar_metadata","postgres_metadata"],"limits":["paper metadata is based on indexed upstream scholarly datasets","claims and concepts are available only for extracted papers","absence of claims or concepts means no extracted graph data is available in this response"],"status":"available","degraded":false,"degraded_reasons":[],"diagnostics":{"status":"available","degraded":false,"degraded_reasons":[],"metadata_status":"available","graph_status":"available","abstract_status":"available"},"source_flags":1},"paper_id":633715,"paper_uid":"ca02e1fe-dcd6-4101-9786-f664d76cbe13","canonical_identity":{"paper_id":633715,"paper_uid":"ca02e1fe-dcd6-4101-9786-f664d76cbe13","identity_status":"available","lookup_basis":"semantic_scholar_external_id","compatibility_path":"corpus_id"},"url":"https://sah.borca.ai/papers/23046847"}