{"corpus_id":56058550,"paper_sha":"0c9098ae8342c28b8bf60e5cf1e792966af92902","doi":"10.1109/cvpr42600.2020.00101","arxiv_id":"1812.05583","pmid":null,"pmcid":null,"mag_id":3034902354,"dblp_id":"conf/cvpr/IzadiniaS20","acl_id":null,"title":"Scene Recomposition by Learning-Based ICP","year":2018,"publication_date":"2018-12-13","venue":"Computer Vision and Pattern Recognition","journal":{"name":"2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","pages":"927-936","volume":null},"journal_issn":null,"journal_title":null,"publication_types":["JournalArticle","Conference"],"pubmed_pub_types":null,"s2_fields_of_study":["Computer Science"],"reference_count":72,"citation_count":14,"influential_citation_count":0,"is_open_access":true,"arxiv_categories":["cs.CV","cs.LG"],"arxiv_license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","arxiv_journal_ref":null,"mesh_headings":null,"chemicals":null,"comments_corrections":null,"source_flags":1,"s2_open_access_pdf_url":"https://arxiv.org/pdf/1812.05583","s2_open_access_landing_url":"https://www.semanticscholar.org/paper/0c9098ae8342c28b8bf60e5cf1e792966af92902","s2_open_access_license":null,"s2_open_access_status":"GREEN","pmc_open_access_pdf_url":null,"pmc_open_access_landing_url":null,"pmc_open_access_license":null,"pmc_open_access_status":null,"unpaywall_open_access_pdf_url":null,"unpaywall_open_access_landing_url":null,"unpaywall_open_access_license":null,"unpaywall_open_access_status":null,"abstract":"By moving a depth sensor around a room, we compute a 3D CAD model of the environment, capturing the room shape and contents such as chairs, desks, sofas, and tables. Rather than reconstructing geometry, we match, place, and align each object in the scene to thousands of CAD models of objects. In addition to the fully automatic system, the key technical contribution is a novel approach for aligning CAD models to 3D scans, based on deep reinforcement learning. This approach, which we call Learning-based ICP, outperforms prior ICP methods in the literature, by learning the best points to match and conditioning on object viewpoint. LICP learns to align using only synthetic data and does not require ground truth annotation of object pose or keypoint pair matching in real scene scans. While LICP is trained on synthetic data and without 3D real scene annotations, it outperforms both learned local deep feature matching and geometric based alignment methods in real scenes. The proposed method is evaluated on real scenes datasets of SceneNN and ScanNet as well as synthetic scenes of SUNCG. High quality results are demonstrated on a range of real world scenes, with robustness to clutter, viewpoint, and occlusion.","claims":[{"public_id":"cl_5198b3257d3a99d0b706374d5e115874","status":"active","text":"A full automatic system computes a 3D CAD model of a room by matching, placing, and aligning room objects to thousands of CAD models instead of reconstructing geometry directly.","confidence":0.93,"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_5198b3257d3a99d0b706374d5e115874"},{"public_id":"cl_d0c31aac6719eab09d28018a57f05ce3","status":"active","text":"Learning-based ICP aligns CAD models to 3D scans using deep reinforcement learning.","confidence":0.95,"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_d0c31aac6719eab09d28018a57f05ce3"},{"public_id":"cl_d9ab35b1287e2f3fff46638bd1153d16","status":"active","text":"Learning-based ICP learns alignment from synthetic data without requiring ground truth annotation of object pose or keypoint pair matching in real scene scans.","confidence":0.94,"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_d9ab35b1287e2f3fff46638bd1153d16"},{"public_id":"cl_694a6dcb5c975e576f55c29ca8ffb684","status":"active","text":"Learning-based ICP outperforms learned local deep feature matching and geometric based alignment methods in real scenes, and is evaluated on SceneNN, ScanNet, and SUNCG.","confidence":0.9,"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_694a6dcb5c975e576f55c29ca8ffb684"},{"public_id":"cl_ef85c3b20dc885f4c9676da6e9c06acc","status":"active","text":"Learning-based ICP outperforms prior ICP methods by learning the best points to match and conditioning alignment on object viewpoint.","confidence":0.92,"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/claims/cl_ef85c3b20dc885f4c9676da6e9c06acc"}],"concepts":[{"public_id":"co_0522bf1695d320939fdcbe2fc000b719","status":"active","name":"real scene scans","description":"Scan data from real-world scenes used to assess alignment without requiring real-scene annotations for training.","types":["evaluation data"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_0522bf1695d320939fdcbe2fc000b719"},{"public_id":"co_13533e7e332a6fd3e24e00f5095b88f8","status":"active","name":"3D CAD model","description":"A three-dimensional computer-aided design representation of the room shape and contents.","types":["model"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_13533e7e332a6fd3e24e00f5095b88f8"},{"public_id":"co_14d00a3780f8bf27e14554ae591fe92d","status":"active","name":"depth sensor","description":"A sensing device moved around a room to capture depth information about the environment.","types":["sensor"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_14d00a3780f8bf27e14554ae591fe92d"},{"public_id":"co_4d434212e27fe2c092e3357fd629eb51","status":"active","name":"Learning-based ICP","description":"The proposed alignment approach for matching CAD models to 3D scans.","types":["method"],"aliases":["LICP"],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_4d434212e27fe2c092e3357fd629eb51"},{"public_id":"co_68d0aa22e0ee1ac91af8291c3eee96a9","status":"active","name":"CAD models of objects","description":"Computer-aided design object models used as matches for chairs, desks, sofas, tables, and other scene contents.","types":["model collection"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_68d0aa22e0ee1ac91af8291c3eee96a9"},{"public_id":"co_8f1ff4cf5096a8f9728e72546156ce0a","status":"active","name":"deep reinforcement learning","description":"A learning framework used to train the proposed CAD-to-scan alignment method.","types":["machine learning method"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_8f1ff4cf5096a8f9728e72546156ce0a"},{"public_id":"co_96f0b24e6bdf6f4cc7fac24775034683","status":"active","name":"learned local deep feature matching","description":"A learned feature-based matching approach used as a comparison method for real-scene alignment.","types":["baseline method"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_96f0b24e6bdf6f4cc7fac24775034683"},{"public_id":"co_ad14ba1b68310372042acb6c34c1b4ba","status":"active","name":"3D scans","description":"Three-dimensional scan data of scenes used as the target for CAD model alignment.","types":["data"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_ad14ba1b68310372042acb6c34c1b4ba"},{"public_id":"co_b688ced86ce1cdcb4afa7d5c6a651d97","status":"active","name":"synthetic data","description":"Computer-generated training data used to learn the alignment method.","types":["training data"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_b688ced86ce1cdcb4afa7d5c6a651d97"},{"public_id":"co_c85ac79b142efd0fc7432123c7f16150","status":"active","name":"object viewpoint","description":"The observed viewing orientation of an object used as conditioning information during alignment.","types":["alignment factor"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_c85ac79b142efd0fc7432123c7f16150"},{"public_id":"co_d52c01091eac27c297a4bae4be17e901","status":"active","name":"geometric based alignment methods","description":"Alignment methods based on geometric information used as comparison approaches.","types":["baseline method"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_d52c01091eac27c297a4bae4be17e901"},{"public_id":"co_eab33064fc789a712a53431a897854c4","status":"active","name":"prior ICP methods","description":"Existing iterative closest point alignment methods used as baselines for comparison.","types":["baseline method"],"aliases":[],"contributors":[{"id":35,"public_id":"b2adb6bfad","public_label":"Anonymous (b2adb6bfad)","roles":["extraction"],"url":"https://sah.borca.ai/u/b2adb6bfad"},{"id":2,"public_id":"4715169a40","public_label":"AK (4715169a40)","roles":["review"],"url":"https://sah.borca.ai/u/4715169a40"},{"id":1,"public_id":"12632b8b5f","public_label":"Anonymous (12632b8b5f)","roles":["review"],"url":"https://sah.borca.ai/u/12632b8b5f"}],"url":"https://sah.borca.ai/concepts/co_eab33064fc789a712a53431a897854c4"}],"external_ids":{"DOI":"10.1109/cvpr42600.2020.00101","ArXiv":"1812.05583","PubMed":null,"PubMedCentral":null,"MAG":3034902354,"DBLP":"conf/cvpr/IzadiniaS20","ACL":null},"open_access":{"is_open_access":true,"pdf_url":"https://arxiv.org/pdf/1812.05583","landing_url":"https://www.semanticscholar.org/paper/0c9098ae8342c28b8bf60e5cf1e792966af92902","source":"semantic_scholar","pdf_url_source":"semantic_scholar_open_access_pdf","license":null,"status":"GREEN","reason":null},"reference_availability":{"status":"available","references_indexed":true,"full_text_available":true,"full_text_source":"arxiv","count_basis":"semantic_scholar_metadata","extraction_status":"not_applicable","reason":null},"source":{"provider":"episteme2","base_corpus":"semantic_scholar_dump","freshness_mode":"unknown","basis":["semantic_scholar_metadata","postgres_metadata"],"limits":["paper metadata is based on indexed upstream scholarly datasets","claims and concepts are available only for extracted papers","absence of claims or concepts means no extracted graph data is available in this response"],"status":"available","degraded":false,"degraded_reasons":[],"diagnostics":{"status":"available","degraded":false,"degraded_reasons":[],"metadata_status":"available","graph_status":"available","abstract_status":"available"},"source_flags":1},"paper_id":631301,"paper_uid":"22ecf36c-50e3-4cae-9bb4-68a01f76e174","canonical_identity":{"paper_id":631301,"paper_uid":"22ecf36c-50e3-4cae-9bb4-68a01f76e174","identity_status":"available","lookup_basis":"semantic_scholar_external_id","compatibility_path":"corpus_id"},"url":"https://sah.borca.ai/papers/56058550"}