Hostname: page-component-55f67697df-2mk96 Total loading time: 0 Render date: 2025-05-09T20:50:39.732Z Has data issue: false hasContentIssue false
  • English
  • Français

A review of robotic grasp detection technology

Published online by Cambridge University Press:  22 September 2023

Minglun Dong Open the ORCID record for Minglun Dong [Opens in a new window]  and
Jian Zhang
Minglun Dong*
Affiliation:
School of Mechanical Engineering, Tongji University, Shanghai, China
Jian Zhang
Affiliation:
School of Mechanical Engineering, Tongji University, Shanghai, China
*
Corresponding author: Minglun Dong; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In order to complete many complex operations and attain more general-purpose utility, robotic grasp is a necessary skill to master. As the most common essential action of robots in factory and daily life environments, robotic autonomous grasping has a wide range of application prospects and has received much attention from researchers in the past decade. However, the accurate grasp of arbitrary objects in unstructured environments is still a research challenge that has not yet been completely overcome. A complete robotic grasp system usually involves three aspects: grasp detection, grasp planning, and control subsystem. As the first step, identifying the location of the object and generating the grasp pose is the premise of successful grasp, which is conducive to planning the subsequent grasp path and the realization of the entire grasp action. Therefore, this paper conducts a literature review focusing on grasp detection technology and concludes two significant aspects: the analytic and data-driven methods. According to the previous grasp experience of the target object, this paper divides the data-driven methods into the grasp of known and unknown objects. Then it describes in detail the typical grasp detection methods and related characteristics of each classification in the grasp of unknown objects. Finally, current research status and potential research directions in this field are discussed to provide some reference for related research.

Keywords

robotic handsgraspingcontrol of robotic systems
Type
Review Article
Information
Robotica , Volume 41 , Issue 12 , December 2023 , pp. 3846 - 3885
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Jiang, D., Li, G., Sun, Y., Hu, J., Yun, J. and Liu, Y., “Manipulator grabbing position detection with information fusion of color image and depth image using deep learning,” J. Ambient Intell. Humaniz. Comput. 12(12), 10809–10822 (2021). doi: 10.1007/s12652-020-02843-w.CrossRefGoogle Scholar
Du, G., Wang, K., Lian, S. and Zhao, K., “Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review,” Artif. Intell. Rev. 54(3), 1677–1734 (2020). doi: 10.1007/s10462-020-09888-5.CrossRefGoogle Scholar
Liu, S., Tian, G., Zhang, Y., Zhang, M. and Liu, S., “Service planning oriented efficient object search: A knowledge-based framework for home service robot,” Expert Syst. Appl. 187, 115853 (2022). doi: 10.1016/j.eswa.2021.115853.CrossRefGoogle Scholar
Sanchez, J., Corrales, J. A., Bouzgarrou, B. C. and Mezouar, Y., “Robotic manipulation and sensing of deformable objects in domestic and industrial applications: A survey,” Int. J. Robot. Res. 37(7), 688–716 (2018). doi: 10.1177/0278364918779698.CrossRefGoogle Scholar
Morrison, D., Corke, P. and Leitner, J., “Learning robust, real-time, reactive robotic grasping,” Int. J. Robot. Res. 39(2-3), 183–201 (2019). doi: 10.1177/0278364919859066.CrossRefGoogle Scholar
Antanas, L., Moreno, P., Neumann, M., De Figueiredo, R. P., Kersting, K., Santos-Victor, J. and De Raedt, L., “Semantic and geometric reasoning for robotic grasping: A probabilistic logic approach,” Auton. Robot. 43(6), 1393–1418 (2018). doi: 10.1007/s10514-018-9784-8.CrossRefGoogle Scholar
Mohammed, M. Q., Chung, K. L. and Chyi, C. S., “Review of deep reinforcement learning-based object grasping: Techniques, open challenges, and recommendations,” IEEE Access 8, 178450–178481 (2020). doi: 10.1109/access.2020.3027923.CrossRefGoogle Scholar
Gong, H. J., Ling, S., Dong, X. and Shotton, J., “Enhanced computer vision with microsoft kinect sensor: A review,” IEEE Trans. Cybern. 43(5), 1318–1334 (2013). doi: 10.1109/tcyb.2013.2265378.Google Scholar
Zabatani, A., Surazhsky, V., Sperling, E., Moshe, S. B., Menashe, O., Silver, D. H., Karni, Z., Bronstein, A. M., Bronstein, M. M. and Kimmel, R., “Intel® RealSenseTM SR300 coded light depth camera,” IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2333–2345 (2020). doi: 10.1109/tpami.2019.2915841.CrossRefGoogle Scholar
Kleeberger, K., Bormann, R., Kraus, W. and Huber, M. F., “A survey on learning-based robotic grasping,” Curr. Robot. Rep. 1(4), 239–249 (2020). doi: 10.1007/s43154-020-00021-6.CrossRefGoogle Scholar
Kumra, S. and Kanan, C., “Robotic Grasp Detection Using Deep Convolutional Neural Networks,” In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017) pp. 769–776. doi: 10.1109/iros.2017.8202237.CrossRefGoogle Scholar
Al-Gallaf, E., Allen, A. and Warwick, K., “A survey of multi-fingered robot hands: Issues and grasping achievements,” Mechatronics 3(4), 465–491 (1993). doi: 10.1016/0957-4158(93)90018-w.CrossRefGoogle Scholar
Shimoga, K. B., “Robot grasp synthesis algorithms: A survey,” Int. J. Robot. Res. 15(3), 230–266 (1996). doi: 10.1177/027836499601500302.CrossRefGoogle Scholar
Sahbani, A., El-Khoury, S. and Bidaud, P., “An overview of 3D object grasp synthesis algorithms,” Robot. Auton. Syst. 60(3), 326–336 (2012). doi: 10.1016/j.robot.2011.07.016.CrossRefGoogle Scholar
Bicchi, A. and Kumar, V., “Robotic Grasping and Contact: A Review,” In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings, vol. 1 (2000) pp. 348–353. doi: 10.1109/robot.2000.844081.Google Scholar
Du, G., Wang, K., Lian, S. and Zhao, K., “Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review,” Artif. Intell. Rev. 54(3), 1677–1734 (2021). doi: 10.1007/s10462-020-09888-5.CrossRefGoogle Scholar
Caldera, S., Rassau, A. and Chai, D., “Review of deep learning methods in robotic grasp detection,” Multimodal Technol. Interact. 2(3), 57 (2018). doi: 10.3390/mti2030057.CrossRefGoogle Scholar
Bohg, J., Morales, A., Asfour, T. and Kragic, D., “Data-driven grasp synthesis—A survey,” IEEE Trans. Robot. 30(2), 289–309 (2013). doi: 10.1109/tro.2013.2289018.CrossRefGoogle Scholar
Bicchi, A., “Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity,” IEEE Trans. Robot. Autom. 16(6), 652–662 (2000). doi: 10.1109/70.897777.CrossRefGoogle Scholar
Salisbury, J. K. and Roth, B., “Kinematic and force analysis of articulated mechanical hands,” J. Mech. Transm. Autom. Des. 105(1), 35–41 (1983). doi: 10.1115/1.3267342.CrossRefGoogle Scholar
Liu, Y. H., “Qualitative test and force optimization of 3-D frictional form-closure grasps using linear programming,” IEEE Trans. Robot. Autom. 15(1), 163–173 (1999). doi: 10.1109/70.744611.Google Scholar
Ding, D., Liu, Y. H. and Wang, S., “Computing 3-D Optimal Form-Closure Grasps,” In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings, vol. 4 (2000) pp. 3573–3578. doi: 10.1109/robot.2000.845288.Google Scholar
Prattichizzo, D. and Trinkle, J. C., “Grasping,” In: Springer Handbook of Robotics (Springer, Berlin/Heidelberg, 2008) pp. 671–700. doi: 10.1007/978-3-540-30301-5_29.CrossRefGoogle Scholar
Baker, B. S., Fortune, S. and Grosse, E., “Stable Prehension with Three Fingers,” In: Proceedings of the Seventeenth Annual ACM Symposium on Theory of Computing (1985) pp. 114–120. doi: 10.1145/22145.22158.Google Scholar
Markenscoff, X. and Papadimitriou, C. H., “Optimum grip of a polygon,” Int. J. Robot. Res. 8(2), 17–29 (1989). doi: 10.1177/027836498900800202.CrossRefGoogle Scholar
Nguyen, V. D., “Constructing force-closure grasps,” Int. J. Robot. Res. 7(3), 3–16 (1988). doi: 10.1177/027836498800700301.CrossRefGoogle Scholar
Ponce, J. and Faverjon, B., “On computing three-finger force-closure grasps of polygonal objects,” IEEE Trans. Robot. Autom. 11(6), 868–881 (1995). doi: 10.1109/70.478433.CrossRefGoogle Scholar
Cornella, J. and Suárez, R., “On Computing Form-Closure Grasps/Fixtures for Non-Polygonal Objects (ISATP 2005),” In: The 6th IEEE International Symposium on Assembly and Task Planning: From Nano to Macro Assembly and Manufacturing (2005) pp. 138–143. doi: 10.1109/isatp.2005.1511463.Google Scholar
Faverjon, B. and Ponce, J., “On Computing Two-Finger Force-Closure Grasps of Curved 2D Objects,” In: Proceedings. 1991 IEEE International Conference on Robotics and Automation (1991) pp. 424–429. doi: 10.1109/robot.1991.131614.CrossRefGoogle Scholar
Bicchi, A., “On the closure properties of robotic grasping,” Int. J. Robot. Res. 14(4), 319–334 (1995). doi: 10.1177/027836499501400402.CrossRefGoogle Scholar
Howard, W. S. and Kumar, V., “On the stability of grasped objects,” IEEE Trans. Robot. Autom. 12(6), 904–917 (1996). doi: 10.1109/70.544773.CrossRefGoogle Scholar
Trinkle, J. C., “On the stability and instantaneous velocity of grasped frictionless objects,” IEEE Trans. Robot. Autom. 8(5), 560–572 (1992). doi: 10.1109/70.163781.CrossRefGoogle Scholar
Zhu, X. Y. and Wang, J., “Synthesis of force-closure grasps on 3-D objects based on the Q distance,” IEEE Trans. Robot. Autom. 19(4), 669–679 (2003). doi: 10.1109/tra.2003.814499.Google Scholar
Liu, Y. H., “Computing N-Finger Force-Closure Grasps on Polygonal Objects,” In: Proceedings. 1998 IEEE International Conference on Robotics and Automation, vol. 3 (1998) pp. 2734–2739. doi: 10.1109/robot.1998.680759.Google Scholar
Ciocarlie, M., Miller, A. and Allen, P., “Grasp Analysis Using Deformable Fingers,” In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2005) pp. 4122–4128. doi: 10.1109/iros.2005.1545525.CrossRefGoogle Scholar
Rosales, C., Suárez, R., Gabiccini, M. and Bicchi, A., “On the Synthesis of Feasible and Prehensile Robotic Grasps,” In: 2012 IEEE International Conference on Robotics and Automation (2012) pp. 550–556. doi: http://doi.org/10.1109/icra.2012.6225238.CrossRefGoogle Scholar
Jia, P., Li, W.li, Wang, G. and Li, S. Y., “Optimal grasp planning for a dexterous robotic hand using the volume of a generalized force ellipsoid during accepted flattening,” Int. J. Adv. Robot. Syst. 14(1), 172988141668713 (2017). doi: 10.1177/1729881416687134.CrossRefGoogle Scholar
Lin, Q., Burdick, J. W. and Rimon, E., “A stiffness-based quality measure for compliant grasps and fixtures,” IEEE Trans. Robot. Autom. 16(6), 675–688 (2000). doi: 10.1109/70.897779.Google Scholar
Ferrari, C. and Canny, J., “Planning Optimal Grasps,” In: Proceedings 1992 IEEE International Conference on Robotics and Automation (1992) pp. 2290–2295. doi: 10.1109/robot.1992.219918.CrossRefGoogle Scholar
H., M. O., “Planning of grasping with multi-fingered hands based on the maximal external wrench,” J. Mech. Eng. 45(3), 258–262 (2009). doi: 10.3901/jme.2009.03.258.Google Scholar
Cutkosky, M. and Wright, P., “Modeling Manufacturing Grips and Correlations with the Design of Robotic Hands,” In: Proceedings. 1986 IEEE International Conference on Robotics and Automation, vol. 3 (1986) pp. 1533–1539. doi: 10.1109/robot.1986.1087525.CrossRefGoogle Scholar
Li, Z. and Sastry, S. S., “Task-oriented optimal grasping by multi-fingered robot hands,” IEEE J. Robot. Autom. 4(1), 32–44 (1988). doi: 10.1109/56.769.CrossRefGoogle Scholar
Pollard, N. S., “Parallel algorithms for synthesis of whole-hand grasps,” Proc. Int. Conf. Robot. Autom. 1, 373–378 (1997). doi: 10.1109/robot.1997.620066.CrossRefGoogle Scholar
Borst, C., Fischer, M. and Hirzinger, G., “Grasp Planning: How to Choose a Suitable Task Wrench Space,” In: IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04, vol. 1 (2004) pp. 319–325.Google Scholar
El-Khoury, S., de Souza, R. and Billard, A., “On computing task-oriented grasps,” Robot. Auton. Syst. 66, 145–158 (2015). doi: 10.1016/j.robot.2014.11.016.CrossRefGoogle Scholar
Deng, Z., Zheng, X., Zhang, L. and Zhang, J., “A learning framework for semantic reach-to-grasp tasks integrating machine learning and optimization,” Robot. Auton. Syst. 108, 140–152 (2018).CrossRefGoogle Scholar
Wiering, M. and van Otterlo, M., “Reinforcement learning, Adapt. Learn. Optim. 12(3), 729 (2012). doi: 10.1007/978-3-642-27645-3.Google Scholar
Bormann, R., de Brito, B. F., Lindermayr, J., Omainska, M. and Patel, M., “Towards Automated Order Picking Robots for Warehouses and Retail,” In: Lecture Notes in Computer Science (2019) pp. 185–198. doi: 10.1007/978-3-030-34995-0_18.Google Scholar
Miller, A. T., Knoop, S., Christensen, H. I. and Allen, P. K., “Automatic Grasp Planning Using Shape Primitives,” In: 2003 IEEE International Conference on Robotics and Automation, vol. 2 (2003) pp. 1824–1829. doi: 10.1109/robot.2003.1241860.Google Scholar
Miller, A. T. and Allen, P. K., “Graspit!: A Versatile Simulator for Grasp Analysis,” In: Proc. of the ASME Dynamic Systems and Control Division, vol. 2 (2000) pp. 1251–1258. doi: 10.1115/imece2000-2439.Google Scholar
Goldfeder, C., Allen, P. K., Lackner, C. and Pelossof, R., “Grasp Planning via Decomposition Trees,” In: Proceedings 2007 IEEE International Conference on Robotics and Automation (2007) pp. 4679–4684. doi: 10.1109/robot.2007.364200.CrossRefGoogle Scholar
Pelossof, R., Miller, A., Allen, P. and Jebara, T., “An SVM Learning Approach to Robotic Grasping,” In: IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04, vol. 4 (2004) pp. 3512–3518. doi: 10.1109/robot.2004.1308797.Google Scholar
Cortes, C., Vapnik, V. and Networks, S.-V., “Support-vector networks,” Mach. Learn. 20(3), 273–297 (1995). doi: 10.1007/bf00994018.CrossRefGoogle Scholar
Nieuwenhuisen, M., Stückler, J., Berner, A., Klein, R. and Behnke, S., “Shape-Primitive Based Object Recognition and Grasping,” In: ROBOTIK 2012; 7th German Conference on Robotics (2012) pp. 1–5.Google Scholar
Cutkosky, M. R., “On grasp choice, grasp models, and the design of hands for manufacturing tasks,” IEEE Trans. Robot. Autom. 5(3), 269–279 (1989). doi: 10.1109/70.34763.CrossRefGoogle Scholar
Kjellstrom, H., Romero, J. and Kragic, D., “Visual Recognition of Grasps for Human-to-Robot Mapping,” In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (2008) pp. 3192–3199. doi: 10.1109/iros.2008.4650917.CrossRefGoogle Scholar
Feix, T., Romero, J., Schmiedmayer, H. B., Dollar, A. M. and Kragic, D., “The grasp taxonomy of human grasp types,” IEEE Trans. Hum. Mach. Syst. 46(1), 66–77 (2016). doi: 10.1109/thms.2015.2470657.CrossRefGoogle Scholar
Cini, F., Ortenzi, V., Corke, P. and Controzzi, M. J. S. R., “On the choice of grasp type and location when handing over an object,” Sci. Robot. 4(27), (2019). doi: 10.1126/scirobotics.aau9757.CrossRefGoogle ScholarPubMed
Kang, S. B. and Ikeuchi, K., “Toward automatic robot instruction from perception-mapping human grasps to manipulator grasps,” IEEE Trans. Robot. Autom. 13(1), 81–95 (1997). doi: 10.1109/70.554349.CrossRefGoogle Scholar
Balasubramanian, R., Xu, L., Brook, P. D., Smith, J. R. and Matsuoka, Y., “Physical human interactive guidance: Identifying grasping principles from human-planned grasps,” IEEE Trans. Robot. 28(4), 899–910 (2012). doi: 10.1109/tro.2012.2189498.CrossRefGoogle Scholar
Ekvall, S. and Kragic, D., “Learning and Evaluation of the Approach Vector for Automatic Grasp Generation and Planning,” In: Proceedings 2007 IEEE International Conference on Robotics and Automation (2007) pp. 4715–4720. doi: 10.1109/robot.2007.364205.CrossRefGoogle Scholar
Ekvall, S. and Kragic, D., “Receptive Field Cooccurrence Histograms for Object Detection,” In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (2015) pp. 84–89. doi: 10.1109/iros.2005.1545588.Google Scholar
Ekvall, S. and Kragic, D., “Grasp Recognition for Programming by Demonstration,” In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation (2005) pp. 748–753. doi: 10.1109/robot.2005.1570207.CrossRefGoogle Scholar
Lin, Y. and Sun, Y., “Robot grasp planning based on demonstrated grasp strategies,” Int. J. Robot. Res. 34(1), 26–42 (2014). doi: 10.1177/0278364914555544.CrossRefGoogle Scholar
Deng, Z., Gao, G., Frintrop, S., Sun, F., Zhang, C. and Zhang, J., “Attention based visual analysis for fast grasp planning with a multi-fingered robotic hand,” Front. Neurorobot. 13, 60 (2019). doi: 10.3389/fnbot.2019.00060.CrossRefGoogle ScholarPubMed
Detry, R., Kraft, D., Kroemer, O., Bodenhagen, L., Peters, J., Krüger, N. and Piater, J., “Learning grasp affordance densities,” Paladyn J. Behav. Robot. 2(1), 1–17 (2011). doi: 10.2478/s13230-011-0012-x.CrossRefGoogle Scholar
Dehnad, K., “Density estimation for statistics and data analysis,” Technometrics 29(4), 495–495 (1987). doi: 10.1080/00401706.1987.10488295.CrossRefGoogle Scholar
Kroemer, O. B., Detry, R., Piater, J. and Peters, J., “Combining active learning and reactive control for robot grasping,” Robot. Auton. Syst. 58(9), 1105–1116 (2010). doi: 10.1016/j.robot.2010.06.001.CrossRefGoogle Scholar
Kober, J., Oztop, E. and Peters, J., “Reinforcement Learning to Adjust Robot Movements to New Situations,” In: Robotics: Science and Systems VI (2010). doi: 10.15607/rss.2010.vi.005.Google Scholar
Theodorou, E., Buchli, J. and Schaal, S., “A generalized path integral control approach to reinforcement learning,” J. Mach. Learn. Res. 11, 3137–3181 (2010).Google Scholar
Stulp, F., Theodorou, E., Kalakrishnan, M., Pastor, P., Righetti, L. and Schaal, S., “Learning Motion Primitive Goals for Robust Manipulation,” In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (2011) pp. 325–331. doi: 10.1109/iros.2011.6094877.Google Scholar
Stulp, F., Theodorou, E., Buchli, J. and Schaal, S., “Learning to Grasp under Uncertainty,” In: 2011 IEEE International Conference on Robotics and Automation (2011) pp. 5703–5708. doi: 10.1109/icra.2011.5979644.CrossRefGoogle Scholar
Dune, C., Marchand, E., Collowet, C. and Leroux, C., “Active Rough Shape Estimation of Unknown Objects,” In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (2008) pp. 3622–3627. doi: 10.1109/iros.2008.4651005.CrossRefGoogle Scholar
Detry, R., Ek, C. H., Madry, M., Piater, J. and Kragic, D., “Generalizing Grasps Across Partly Similar Objects,” In: 2012 IEEE International Conference on Robotics and Automation (2012) pp. 3791–3797. doi: 10.1109/icra.2012.6224992.CrossRefGoogle Scholar
Hsiao, K., Chitta, S., Ciocarlie, M. and Jones, E. G., “Contact-Reactive Grasping of Objects with Partial Shape Information,” In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (2010) pp. 1228–1235.CrossRefGoogle Scholar
Liu, Z., Gueta, L. B. and Ota, J., “Feature Extraction from Partial Shape Information for Fast Grasping of Unknown Objects,” In: 2011 IEEE International Conference on Robotics and Biomimetics (2011) pp. 1332–1337. doi: 10.1109/robio.2011.6181473.CrossRefGoogle Scholar
Morales, A., Sanz, P. J., Del Pobil, A. P. and Fagg, A. H., “Vision-based three-finger grasp synthesis constrained by hand geometry,” Robot. Auton. Syst. 54(6), 496–512 (2006). doi: 10.1016/j.robot.2006.01.002.CrossRefGoogle Scholar
Richtsfeld, M. and Vincze, M., “Grasping of Unknown Objects from a Table Top,” In: Workshop on Vision in Action: Efficient Strategies for Cognitive Agents in Complex Environments (2008).Google Scholar
Fischler, M. A. and Bolles, R. C. B., “Random sample consensus,” Commun. ACM 24(6), 381–395 (1981). doi: 10.1145/358669.358692.CrossRefGoogle Scholar
O’Rourke, J., Computational Geometry in C (Cambridge University Press, Cambridge, 1998).CrossRefGoogle Scholar
Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., Kragic, D. and Morales, A., “Mind the Gap - Robotic Grasping under Incomplete Observation,” In: 2011 IEEE International Conference on Robotics and Automation (2011) pp. 686–693. doi: 10.1109/icra.2011.5980354.CrossRefGoogle Scholar
Hinton, G. E. and Salakhutdinov, R. R., “Reducing the dimensionality of data with neural networks,” Science 313(5786), 504–507 (2006). doi: 10.1126/science.1127647.CrossRefGoogle ScholarPubMed
Bo, L., Ren, X. and Fox, D., “Unsupervised Feature Learning for RGB-D Based Object Recognition,” In: Experimental Robotics (2013) pp. 387–402. doi: 10.1007/978-3-319-00065-7_27.CrossRefGoogle Scholar
Socher, R., Huval, B., Bath, B., Manning, C. D. and Ng, A., “Convolutional-Recursive Deep Learning for 3D Object Classification,” In: Advances in Neural Information Processing Systems, vol. 25 (2012).Google Scholar
Zhou, M., Bai, Y., Zhang, W., Zhao, T. and Mei, T., “Look-into-Object: Self-Supervised Structure Modeling for Object Recognition,” In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) pp. 11774–11783. doi: 10.1109/cvpr42600.2020.01179.Google Scholar
Cai, W., Liu, D., Ning, X., Wang, C. and Xie, G., “Voxel-based three-view hybrid parallel network for 3D object classification,” Displays 69, 102076 (2021). doi: 10.1016/j.displa.2021.102076.CrossRefGoogle Scholar
Lenz, I., Lee, H. and Saxena, A., “Deep learning for detecting robotic grasps,” Int. J. Robot. Res. 34(4-5), 705–724 (2015). doi: 10.1177/0278364914549607.CrossRefGoogle Scholar
Zhang, J., Zhang, W., Song, R., Ma, L. and Li, Y., “Grasp for Stacking via Deep Reinforcement Learning,” In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (2020) pp. 2543–2549. doi: 10.1109/icra40945.2020.9197508.CrossRefGoogle Scholar
Quillen, D., Jang, E., Nachum, O., Finn, C., Ibarz, J. and Levine, S., “Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods,” In: 2018 IEEE International Conference on Robotics and Automation (ICRA) (2018) pp. 6284–6291. doi: 10.1109/icra.2018.8461039.CrossRefGoogle Scholar
Pedersen, O. M., Misimi, E. and Chaumette, F., “Grasping Unknown Objects by Coupling Deep Reinforcement Learning, Generative Adversarial Networks, and Visual Servoing,” In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (2020) pp. 5655–5662. doi: 10.1109/icra40945.2020.9197196.CrossRefGoogle Scholar
Qi, C. R., Yi, L., Su, H. and Guibas, L. J., “Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,” In: Advances in Neural Information Processing Systems, vol. 30 (2017).Google Scholar
Jiao, L. and Zhao, J., “A survey on the new generation of deep learning in image processing,” IEEE Access 7, 172231–172263 (2019). doi: 10.1109/access.2019.2956508.CrossRefGoogle Scholar
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J. and Quillen, D., “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” Int. J. Robot. Res. 37(4-5), 421–436 (2017). doi: 10.1177/0278364917710318.CrossRefGoogle Scholar
Saxena, A., Driemeyer, J., Kearns, J. and Ng, A. Y., “Robotic Grasping of Novel Objects,” In: Advances in Neural Information Processing Systems, vol. 19 (2006). doi: 10.7551/mitpress/7503.003.0156.Google Scholar
Saxena, A., Driemeyer, J. and Ng, A. Y., “Robotic grasping of novel objects using vision,” Int. J. Robot. Res. 27(2), 157–173 (2008). doi: 10.1177/0278364907087172.CrossRefGoogle Scholar
Rao, D., Le, Q. V., Phoka, T., Quigley, M., Sudsang, A. and Ng, A. Y., “Grasping Novel Objects with Depth Segmentation,” In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (2010) pp. 2578–2585. doi: 10.1109/iros.2010.5650493.CrossRefGoogle Scholar
Jiang, Y., Moseson, S. and Saxena, A., “Efficient Grasping from RGB-D Images: Learning Using a New Rectangle Representation,” In: 2011 IEEE International Conference on Robotics and Automation (2011) pp. 3304–3311.CrossRefGoogle Scholar
Ten Pas, A., Gualtieri, M., Saenko, K. and Platt, R., “Grasp pose detection in point clouds,” Int. J. Robot. Res. 36(13-14), 1455–1473 (2017). doi: 10.1177/0278364917735594.CrossRefGoogle Scholar
Gualtieri, M., ten Pas, A., Saenko, K. and Platt, R., “High Precision Grasp Pose Detection in Dense Clutter,” In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016) pp. 598–605. doi: 10.1109/iros.2016.7759114.CrossRefGoogle Scholar
Ten Pas, A. and Platt, R., “Using Geometry to Detect Grasp Poses in 3D Point Clouds,” In: Springer Proceedings in Advanced Robotics (2017) pp. 307–324. doi: 10.1007/978-3-319-51532-8_19.Google Scholar
Wei, J., Liu, H., Yan, G. and Sun, F., “Robotic grasping recognition using multi-modal deep extreme learning machine,” Multidim. Syst. Signal Process. 28(3), 817–833 (2016). doi: 10.1007/s11045-016-0389-0.CrossRefGoogle Scholar
Guo, D., Sun, F., Liu, H., Kong, T., Fang, B. and Xi, N., “A Hybrid Deep Architecture for Robotic Grasp Detection,” In: 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017) pp. 1609–1614. doi: 10.1109/icra.2017.7989191.CrossRefGoogle Scholar
Zeiler, M. D. and Fergus, R., “Visualizing and Understanding Convolutional Networks,” In: Computer Vision – ECCV 2014 (2014) pp. 818–833. doi: 10.1007/978-3-319-10590-1_53.CrossRefGoogle Scholar
Redmon, J. and Angelova, A., “Real-Time Grasp Detection Using Convolutional Neural Networks,” In: 2015 IEEE International Conference on Robotics and Automation (ICRA) (2015) pp. 1316–1322. doi: 10.1109/icra.2015.7139361.CrossRefGoogle Scholar
Zhang, Q., Qu, D., Xu, F. and Zou, F., “Robust robot grasp detection in multimodal fusion,” MATEC Web of Conf. 139, 00060 (2017). doi: 10.1051/matecconf/201713900060.CrossRefGoogle Scholar
Depierre, A., Dellandréa, E. and Chen, L., “Scoring Graspability Based on Grasp Regression for Better Grasp Prediction,” In: 2021 IEEE International Conference on Robotics and Automation (ICRA) (2021) pp. 4370–4376.CrossRefGoogle Scholar
Basalla, M., Ebert, F., Tebner, R. and Ke, W., Grasping for the Real World (Greifen mit Deep Learning) (2017). https://www.frederikebert.de/abgeschlossene-projekte/greifen-mit-deep-learning Google Scholar
Watson, J., Hughes, J. and Iida, F., “Real-World, Real-Time Robotic Grasping with Convolutional Neural Networks,” In: Towards Autonomous Robotic Systems (2017) pp. 617–626. doi: 10.1007/978-3-319-64107-2_50.CrossRefGoogle Scholar
Chu, F-J., Xu, R. and Vela, P. A., “Real-world multiobject, multigrasp detection,” IEEE Robot. Autom. Lett. 3(4), 3355–3362 (2018). doi: 10.1109/lra.2018.2852777.CrossRefGoogle Scholar
Zhang, H., Zhou, X., Lan, X., Li, J., Tian, Z. and Zheng, N., “A real-time robotic grasping approach with oriented anchor box,” IEEE Trans. Syst. Man Cybern. Syst. 51(5), 3014–3025 (2019). doi: 10.1109/tsmc.2019.2917034.CrossRefGoogle Scholar
Anđelić, N., Car, Z. and Šercer, M., “Prediction of robot grasp robustness using artificial intelligence algorithms,” Tehnicki Vjesnik - Technical Gazette 29(1), 101–107 (2022). doi: 10.17559/tv-20210204092154.Google Scholar
Chen, L., Huang, P., Li, Y. and Meng, Z., “Edge-dependent efficient grasp rectangle search in robotic grasp detection,” IEEE/ASME Trans. Mechatron. 26(6), 2922–2931 (2020). doi: 10.1109/tmech.2020.3048441.CrossRefGoogle Scholar
Li, T., Sun, X., Shu, X., Wang, C., Wang, Y., Chen, G. and Xue, N., “Robot grasping system and grasp stability prediction based on flexible tactile sensor array,” Machines 9(6), 119 (2021). doi: 10.3390/machines9060119.CrossRefGoogle Scholar
Seita, D., Pokorny, F. T., Mahler, J., Kragic, D., Franklin, M., Canny, J. and Goldberg, K., “Large-Scale Supervised Learning of the Grasp Robustness of Surface Patch Pairs,” In: 2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR) (2016) pp. 216–223. doi: 10.1109/simpar.2016.7862399.CrossRefGoogle Scholar
Mahler, J., Pokorny, F. T., Hou, B., Roderick, M., Laskey, M., Aubry, M., Kohlhoff, K., Kroger, T., Kuffner, J. and Goldberg, K., “Dex-NET 1.0: A Cloud-Based Network of 3D Objects for Robust Grasp Planning Using a Multi-Armed Bandit Model with Correlated Rewards,” In: 2016 IEEE International Conference on Robotics and Automation (ICRA) (2016) pp. 1957–1964. doi: 10.1109/icra.2016.7487342.CrossRefGoogle Scholar
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J. and Goldberg, K., “Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics,” In: Robotics: Science and Systems XIII (2017). doi: 10.15607/rss.2017.xiii.058.Google Scholar
Gariépy, A., Ruel, J. C., Chaib-Draa, B. and Giguere, P., “GQ-STN: Optimizing One-Shot Grasp Detection Based on Robustness Classifier,” In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2019) pp. 3996–4003. doi: 10.1109/iros40897.2019.8967785.CrossRefGoogle Scholar
Jaderberg, M., Simonyan, K. and Zisserman, A., “Spatial Transformer Networks,” In: Advances in Neural Information Processing Systems, vol. 28 (2015).Google Scholar
Fang, K., Zhu, Y., Garg, A., Kurenkov, A., Mehta, V., Fei-Fei, L. and Savarese, S., “Learning task-oriented grasping for tool manipulation from simulated self-supervision,” Int. J. Robot. Res. 39(2-3), 202–216 (2020). doi: 10.1177/0278364919872545.CrossRefGoogle Scholar
Šegota, S. B., Anđelić, N., Z., C. A. R. and Šercer, M., “Prediction of robot grasp robustness using artificial intelligence algorithms,” Technical Gazette 29(1), 101–107 (2022). doi: 10.17559/tv-20210204092154.Google Scholar
Kaggle, “Grasping Dataset,” In: Ugocupcic (2017). https://www.kaggle.com/datasets/ugocupcic/grasping-dataset Google Scholar
Zhang, F., Leitner, J., Milford, M., Upcroft, B. and Corke, P., “Towards vision-based deep reinforcement learning for robotic motion control,” ArXiv Preprint (2015).Google Scholar
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S. and Hassabis, D., “Human-level control through deep reinforcement learning,” Nature 518(7540), 529–533 (2015). doi: 10.1038/nature14236.CrossRefGoogle ScholarPubMed
Zeng, A., Song, S., Yu, K-T., Donlon, E., Hogan, F. R., Bauza, M., Ma, D., Taylor, O., Liu, M., Romo, E., Fazeli, N., Alet, F., Chavan Dafle, N., Holladay, R., Morona, I., Nair, P. Q., Green, D., Taylor, I., Liu, W., Funkhouser, T. and Rodriguez, A., “Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching,” Int. J. Robot. Res. 41(7), 690–705 (2022). doi: 10.1177/0278364919868017.CrossRefGoogle Scholar
Jang, E., Vijayanarasimhan, S., Pastor, P., Ibarz, J. and Levine, S., “End-to-end learning of semantic grasping,” ArXiv Preprint (2017).Google Scholar
Wu, C., Chen, J., Cao, Q., Zhang, J., Tai, Y., Sun, L. and Jia, K., “Grasp proposal networks: An end-to-end solution for visual learning of robotic grasps,” Adv. Neural Inf. Process. Syst. 33, 13174–13184 (2020).Google Scholar
Chen, I. M. and Burdick, J. W., “Finding antipodal point grasps on irregularly shaped objects,” IEEE Trans. Robot. Autom. 9(4), 507–512 (1993). doi: 10.1109/70.246063.CrossRefGoogle Scholar
Alliegro, A., Rudorfer, M., Frattin, F., Leonardis, A. and Tommasi, T., “End-to-end learning to grasp via sampling from object point clouds,” IEEE Robot. Autom. Lett. 7(4), 9865–9872 (2022). doi: 10.1109/lra.2022.3191183.CrossRefGoogle Scholar
Alliegro, A., Valsesia, D., Fracastoro, G., Magli, E. and Tommasi, T., “Denoise and Contrast for Category Agnostic Shape Completion,” In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021) pp. 4629–4638. doi: 10.1109/cvpr46437.2021.00460.Google Scholar
Yi, L., Kim, V. G., Ceylan, D., Shen, I-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A. and Guibas, L., “A scalable active framework for region annotation in 3D shape collections,” ACM Trans. Graph. 35(6), 1–12 (2016). doi: 10.1145/2980179.2980238.CrossRefGoogle Scholar
Saxena, A., Wong, L. L. and Ng, A. Y., “Learning grasp strategies with partial shape information,” AAAI 3(2), 1491–1494 (2008).Google Scholar
Farahani, A., Voghoei, S., Rasheed, K. and Arabnia, H. R., “A Brief Review of Domain Adaptation,” In: Advances in Data Science and Information Engineering (2021) pp. 877–894. doi: 10.1007/978-3-030-71704-9_65.CrossRefGoogle Scholar
Wang, M. and Deng, W., “Deep visual domain adaptation: A survey,” Neurocomputing 312, 135–153 (2018). doi: 10.1016/j.neucom.2018.05.083.CrossRefGoogle Scholar
Wang, B., Jiang, L., Li, J. W., Cai, H. G. and Liu, H., “Grasping Unknown Objects Based on 3D Model Reconstruction,” In: Proceedings, 2005 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (2005) pp. 461–466. doi: 10.1109/aim.2005.1511025.CrossRefGoogle Scholar
Goodfellow, I., Lee, H., Le, Q., Saxe, A. and Ng, A., “Measuring Invariances in Deep Networks,” In: Advances in Neural Information Processing Systems, vol. 22 (2009).Google Scholar
Cambria, E. and Huang, G., “Extreme learning machines-representational learning with ELMs for big data,” IEEE Intell. Syst. 28(6), 30–59 (2013).CrossRefGoogle Scholar
Krizhevsky, A., Sutskever, I. and Hinton, G. E., “ImageNet classification with deep convolutional neural networks,” Commun. ACM 60(6), 84–90 (2017). doi: 10.1145/3065386.CrossRefGoogle Scholar
Simonyan, K. and Zisserman, A., “Very deep convolutional networks for large-scale image recognition,” ArXiv Preprint (2014).Google Scholar
He, K., Zhang, X., Ren, S. and Sun, J., “Deep Residual Learning for Image Recognition,” In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) pp. 770–778. doi: 10.1109/cvpr.2016.90.CrossRefGoogle Scholar
Schwarz, M., Schulz, H. and Behnke, S., “RGB-D Object Recognition and Pose Estimation Based on Pre-Trained Convolutional Neural Network Features,” In: 2015 IEEE International Conference on Robotics and Automation (ICRA) (2015) pp. 1329–1335. doi: 10.1109/icra.2015.7139363.CrossRefGoogle Scholar
Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y. and Zheng, N., “Fully Convolutional Grasp Detection Network with Oriented Anchor Box,” In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2018) pp. 7223–7230. doi: 10.1109/iros.2018.8594116.CrossRefGoogle Scholar
Ren, S., He, K., Girshick, R. and Sun, J., “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). doi: 10.1109/tpami.2016.2577031.CrossRefGoogle ScholarPubMed
Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J. and Scholkopf, B., “Support vector machines,” IEEE Intell. Syst. Appl. 13(4), 18–28 (1998). doi: 10.1109/5254.708428.CrossRefGoogle Scholar
Fukunaga, K. and Narendra, P. M., “A branch and bound algorithm for computing K-nearest neighbors,” IEEE Trans. Comput. 100(7), 750–753 (1975). doi: 10.1109/t-c.1975.224297.CrossRefGoogle Scholar
Ruczinski, I., Kooperberg, C. and LeBlanc, M., “Logic regression,” J. Comput. Graph. Stat. 12(3), 475–511 (2003). doi: 10.1198/1061860032238.CrossRefGoogle Scholar
Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P. and Dollar, A. M., “Benchmarking in manipulation research: Using the Yale-CMU-Berkeley object and model set,” IEEE Robot. Autom. Mag. 22(3), 36–52 (2015). doi: 10.1109/mra.2015.2448951.CrossRefGoogle Scholar