Patent
System and method for automation of surgical pathology processes using artificial intelligence
| العنوان: | System and method for automation of surgical pathology processes using artificial intelligence |
|---|---|
| Patent Number: | 12014,830 |
| تاريخ النشر: | June 18, 2024 |
| Appl. No: | 17/722567 |
| Application Filed: | April 18, 2022 |
| مستخلص: | This invention provides a system for automation of a pathology process, which includes a processor having trained artificial intelligence (AI) modules operating in association therewith, adapted to receive image data from camera images of whole tissue acquired by a camera assembly and whole slide images (WSIs) of inked and segmented tissue samples. A mask produces image results for tissue with holes and free of holes, and a filter provides filtered image results to the AI modules, detecting tumors and macroarchitecture features. A quality assessment process produces quality score outputs for tumors and macroarchitecture features. A report generator provides reports with one or more parameters to a user via an interface. More particularly, the report generator automatically creates a pathology report, having a written description and pictorial diagram relative to the gross images of the tissue integrating the outputs of the AI modules used to analyze the whole slide digital images. |
| Inventors: | Mary Hitchcock Memorial Hospital, for itself and on behalf of Dartmouth-Hitchcock Clinic (Lebanon, NH, US) |
| Assignees: | Mary Hitchcock Memorial Hospital, for itself and on behalf of Dartmouth-Hitchcock Clinic (Lebanon, NH, US) |
| Claim: | 1. A system for automation of a pathology process comprising; a processor having a neural network, having trained artificial intelligence (AI) modules operating in association therewith, adapted to receive image data from images of whole tissue acquired by an camera assembly and whole slide images (WSIs) of inked and segmented tissue samples; a mask that produces image results for tissue with holes and free of holes; a filter that provides filtered image results to the AI modules, which thereby detects tumors and macroarchitecture features; a quality assessment process that produces quality score outputs for tumors and macroarchitecture features; a report generator that provides reports with one or more parameters to a user via an interface; a tissue determination process that determines a size and a description of tissue automatically based upon characteristics in the images acquired by the camera assembly; a tissue grossing and inking process that automatically generates a grossing and inking scheme based upon a user input of desired percentage of tissue margins analyzed and the tissue size; and an ink detection and orientation process that operates on the filtered images of the tissue samples and delivers results thereof to the report generator, and wherein the macroarchitecture features are defined by holes, fat, edges, dermis and epidermis information. |
| Claim: | 2. The system as set forth in claim 1 wherein the AI modules include a tumor inflammation convolutional neural network (CNN), a macroarchitecture hole CNN, a tumor detection graphical neural network (GNN) and a macroarchitecture detection GNN. |
| Claim: | 3. The system as set forth in claim 1 wherein the report generator receives results from the quality assessment process, and wherein the quality assessment process acts on results from a tumor detection GNN and a macroarchitecture detection GNN. |
| Claim: | 4. The system as set forth in claim 1 , further comprising, a visual generation and 3D stitching process that provides results to the report generator. |
| Claim: | 5. The system as set forth in claim 1 , wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue. |
| Claim: | 6. The system as set forth in claim 2 wherein the filters include at least one of a Sobel filter and a gradient-based filter. |
| Claim: | 7. The system as set forth in claim 3 wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue. |
| Claim: | 8. The system as set forth in claim 4 wherein the visual generation process receives information from a cell nuclei detection process. |
| Claim: | 9. The system as set forth in claim 8 wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue. |
| Claim: | 10. A method for automation of a pathology process comprising the steps of: providing a processor with a neural network having artificial intelligence (AI) modules operating in association therewith, which receive image data from camera images of whole tissue acquired by an camera assembly and whole slide images (WSIs) of inked and segmented tissue samples; producing image results for tissue with holes and free of holes with a mask; providing filtered image results to the AI modules, and detecting tumors and macroarchitecture features therewith; generating quality score outputs for tumors and macroarchitecture features; generating a report with one or more parameters to a user via an interface; determining a size and a description of tissue automatically based upon characteristics in the images acquired by the camera assembly; automatically generating a grossing and inking scheme based upon a user input of desired percentage of tissue margins analyzed and the tissue size; and operating an ink detection and orientation process on the filtered images of the tissue samples that delivers results thereof to the report generator, and wherein the macroarchitecture features are defined by holes, fat, edges, dermis and epidermis information. |
| Claim: | 11. The method as set forth in claim 10 , wherein the step of generating the report includes automatically creating a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue. |
| Claim: | 12. The system as set forth in claim 10 , further comprising, defining the AI modules to include a tumor inflammation convolutional neural network (CNN), a macroarchitecture hole CNN, a tumor detection graphical neural network (GNN) and a macroarchitecture detection GNN. |
| Claim: | 13. The method as set forth in claim 10 , further comprising, wherein the step of generating includes receiving results from a quality assessment process acting on results from a tumor detection GNN and a macroarchitecture detection GNN. |
| Claim: | 14. The method as set forth in claim 10 , further comprising, wherein the step of generating includes operating a visual generation and 3D stitching process that provides results. |
| Claim: | 15. The method as set forth in claim 12 , wherein the step of providing filtered image results includes operating at least one of a Sobel filter and a gradient-based filter. |
| Claim: | 16. The method as set forth in claim 14 , further comprising, receiving by the visual generation process, information from a cell nuclei detection process. |
| Claim: | 17. The method as set forth in claim 16 , wherein the step of generating the report includes automatically creating a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue. |
| Patent References Cited: | 3975762 August 1976 van den Bosch 5075214 December 1991 Connor 5218529 June 1993 Meyer 5544650 August 1996 Boon 5784162 July 1998 Cabib 5976885 November 1999 Cohenford 5991028 November 1999 Cabib 6146897 November 2000 Cohenford 6463438 October 2002 Veltri 6690817 February 2004 Cabib 7693334 April 2010 Ogura 8351676 January 2013 Dai 9518982 December 2016 Sood 9786050 October 2017 Bhargava 9983195 May 2018 King 10013760 July 2018 Bhargava 10013781 July 2018 Gammage 10521901 December 2019 Ikemoto 10935773 March 2021 Johnson 11504103 November 2022 Johnson 11508045 November 2022 Amthor 11526984 December 2022 Barnes 11614610 March 2023 Johnson 11620751 April 2023 Sarkar 11621058 April 2023 Gurcan 11631171 April 2023 Leng 11633146 April 2023 Leng 11672425 June 2023 Pyun 11675178 June 2023 Wirch 11681418 June 2023 Wirch 11684264 June 2023 Bryant-Greenwood 11751903 September 2023 Knowlton 11751904 September 2023 Knowlton 11759231 September 2023 Knowlton 11776124 October 2023 Behrooz 11776681 October 2023 Godrich 11783603 October 2023 Stumpe 20010018659 August 2001 Koritzinsky 20030026762 February 2003 Malmros 20060210153 September 2006 Sara 20070127022 June 2007 Cohen 20070135999 June 2007 Kolatt 20080015448 January 2008 Keely 20080166035 July 2008 Qian 20080273199 November 2008 Maier 20080319324 December 2008 Maier 20090002702 January 2009 Maier 20090024375 January 2009 Kremer 20090319291 December 2009 Noordvyk 20110080581 April 2011 Bhargava 20110182490 July 2011 Hoyt 20110286654 November 2011 Krishnan 20120034647 February 2012 Herzog 20120052063 March 2012 Bhargava 20120092663 April 2012 Kull 20120143082 June 2012 Notingher 20120200694 August 2012 Garsha 20120212733 August 2012 Kodali 20120226644 September 2012 Jin 20120290607 November 2012 Bhargava 20130022250 January 2013 Nygaard 20140235487 August 2014 McDevitt 20140270457 September 2014 Bhargava 20140336261 November 2014 Chin 20150268226 September 2015 Bhargava 20150374306 December 2015 Gelbman 20160042511 February 2016 Chukka 20160272934 September 2016 Chander 20160335478 November 2016 Bredno 20170160171 June 2017 Tsujikawa 20170169567 June 2017 Chefd'hotel 20170322124 November 2017 Barnes 20170358082 December 2017 Bhargava 20170372471 December 2017 Eurèn 20180232883 August 2018 Sethi 20190188446 June 2019 Wu 20200302603 September 2020 Barnes 20200372235 November 2020 Peng 20200372635 November 2020 Veidman 20200394825 December 2020 Stumpe 20210103797 April 2021 Jang 20210295507 September 2021 Nie 20220146418 May 2022 Bauer 20230249175 August 2023 Linnes 20230279512 September 2023 Masters 20230394716 December 2023 de Haan 2020180755 September 2020 |
| Other References: | Akbar, 2019, Scientific Reports, pp. 1-9. cited by examiner Arunachalam, 2019, PLOS ONE, pp. 1-19. cited by examiner Corvo, 2017 IEEE Workshop on Visual Analytics in Healthcare, pp. 77-83. cited by examiner McCann, IEEE Signal Processing Magazine, 2015, pp. 78-87. cited by examiner Rivenson, 2020, BMEF, pp. 1-11. cited by examiner Taieb, 2019, ArXiv, pp. 1-58. cited by examiner Grala, 2009, pp. 587-592. cited by examiner U.S. Appl. No. 16/679,133, entitled System and Method for Analyzing Cytological Tissue Preparations, Louis J. Vaickus, filed Nov. 8, 2018. cited by applicant |
| Primary Examiner: | Ezewoko, Michael I |
| Attorney, Agent or Firm: | Loginov & Associates, PLLC Loginov, William A. |
| رقم الانضمام: | edspgr.12014830 |
| قاعدة البيانات: | USPTO Patent Grants |
| الوصف غير متاح. |