Registration of a generic 3D hand model on 2D radiographs using x-ray simulation and CMA-ES

Written by: Tianci Wen and Radu P. Mihail, and Franck P. Vidal

Published on: April, 2021

Summary

Radiographs of the hand are useful in diagnosing and staging diseases such as rheumatoid arthritis (RA) and other musculoskeletal diseases. Radiographs are projections of the 3D anatomy, with the useful information such as pose and pathology becoming lost in the process. We propose a 3D hand pose recovery method for radiographs of hands using a novel hybrid image registration method. Our pose recovery pipeline consists of aligning a simulated X-ray (digitally reconstructed radiograph) of an articulated phantom mesh model to a real hand radiograph using Covariance Matrix Adaptation Evolution Strategy. Early results demonstrate that our approach works well. Further inquiry is required to evaluate the applicability of our registration approach to other articulated musculoskeletal anatomy.

Award

3^rd place: Vidal, F. P., Afshari, S., Albiol, A., Albiol, F., Bellot, A., Brun, A. L., Chou, C.-Y., Desbarats, P., García, M., Giovannelli, J.-F., Hatton, C., Henry, A., Kelly, G., Michelet, C., Mihail, R. P., Racy, M., Rouwane, A., Seznec, H., Sújar, A., Tugwell-Allsup, J., and Villard, P.-F.: "X-ray simulations with gVirtualXray in medicine and life sciences". Dirk Bartz Prize for Visual Computing in Medicine and Life Sciences 2025 (Eurographics Medical Prize).

Publications

Vidal, F. P., Afshari, S., Albiol, A., Albiol, F., Bellot, A., Brun, A. L., Chou, C.-Y., Desbarats, P., García, M., Giovannelli, J.-F., Hatton, C., Henry, A., Kelly, G., Michelet, C., Mihail, R. P., Racy, M., Rouwane, A., Seznec, H., Sújar, A., Tugwell-Allsup, J., and Villard, P.-F. (2025). Meuschke, M. & Kuhlen, T. W. (Eds.). “X-ray simulations with gVirtualXray in medicine and life sciences”. Dirk Bartz Prize for Visual Computing in Medicine and Life Sciences 2025 (Eurographics Medical Prize). DOI: 10.2312/evm.20251974
Wen, T., Mihail, Radu P., & Vidal, F. P. (2021). 3D-2D registration using x-ray simulation and CMA-ES. International Conference on the Applications of Evolutionary Computation, evo*2021 (pp. 458–468). doi:10.1007/978-3-030-72699-7_29
Wen, T., Mihail, R. P., Al-Maliki, S., Létang, J.-M., & Vidal, F. P. (2019). Tam, G. K. L., & Roberts, J. C. (Eds.). Registration of 3D triangular models to 2D X-ray projections using black-box optimisation and X-ray simulation. Computer Graphics and Visual Computing (CGVC) (pp. 105–113). The Eurographics Association. doi:10.2312/cgvc.20191265

Dissertation

Wen, T. (2023 , July). Registration of 3D models to 2D X-ray images using fast X-ray simulation and global optimisation algorithms (Doctoral dissertation). Bangor University, Bangor, UK.

Citations

@INPROCEEDINGS{Vidal2025DirkBartz,
  author = {F. P. Vidal and S. Afshari and A. Albiol and F. Albiol and 
    A. Corb\'i Bellot and A. L. Brun and C.-Y. Chou and P. Desbarats and 
    M. Garc\'ia and J.-F. Giovannelli and C. Hatton and A. Henry and 
    G. Kelly and C. Michelet and Radu P. Mihail M. Racy and A. Rouwane and
    H. Seznec and A. S\'ujar and J. Tugwell-Allsup and P.-F. Villard},
  booktitle = {Dirk Bartz Prize for Visual Computing in Medicine and Life Sciences 2025 (Eurographics Medical Prize)}, 
  title = {{X-ray simulations with gVirtualXray in medicine and life sciences}}, 
  year=2025,
  editor = {Monique Meuschke and Torsten W. Kuhlen},
  publisher = {The Eurographics Association},
  ISBN = {978-3-03868-281-3},
  DOI = {10.2312/evm.20251974}
}

@inproceedings{Wen2021EvoSTAR,
    title     = "{3D-2D} Registration using X-ray Simulation and {CMA-ES}", 
    author    = {Tianci Wen and Mihail, Radu P. and Franck P. Vidal},
    year      = 2021,
    doi       = "10.1007/978-3-030-72699-7_29",
    pages     = "458--468",
    booktitle = "International Conference on the Applications of 
        Evolutionary Computation, evo*2021",
}

 @inproceedings {Wen2019CGVC,
    booktitle = {Computer Graphics and Visual Computing (CGVC)},
    editor    = {G. K. L. Tam and J. C. Roberts},
    title     = {Registration of {3D} triangular models to {2D} {X-ray} projections using 
        black-box optimisation and {X-ray} simulation},
    author    = {T. Wen and R. P. Mihail and S. Al-Maliki and {J.-M.} L\'etang and 
        F.\,P. Vidal},
    year      = 2019,
    publisher = {The Eurographics Association},
    pages     = {105-113},
    isbn      = {978-3-03868-096-3},
    doi       = {10.2312/cgvc.20191265},
}

@PhDThesis{Wen2023PhD,
    author  =     {Tianci Wen},
    title   =     {{Registration of 3D models to 2D X-ray images using fast X-ray simulation and global optimisation algorithms}},
    school  =     {Bangor University},
    address =     {Bangor, UK},
    year    =     2023,
    month   = jul,
}