Background The prognostic value of histone γ-H2AX and 53BP1 proteins to predict the radiotherapy (RT) outcome of patients with rectal carcinoma (RC) was evaluated in a prospective study. High expression of the constitutive histone γ-H2AX is indicative of defective DNA repair pathway and/or genomic instability, whereas 53BP1 (p53-binding protein 1) is a conserved checkpoint protein with properties of a DNA double-strand breaks sensor. Methods Using fluorescence microscopy, we assessed spontaneous and radiation-induced foci of γ-H2AX and 53BP1 in peripheral blood mononuclear cells derived from unselected RC patients (n = 53) undergoing neoadjuvant chemo- and RT. Cells from apparently healthy donors (n = 12) served as references. Results The γ-H2AX assay of in vitro irradiated lymphocytes revealed significantly higher degree of DNA damage in the group of unselected RC patients with respect to the background, initial (0.5 Gy, 30 min) and residual (0.5 Gy and 2 Gy, 24 h post-radiation) damage compared to the control group. Likewise, the numbers of 53BP1 foci analyzed in the samples from 46 RC patients were significantly higher than in controls except for the background DNA damage. However, both markers were not able to predict tumor stage, gastrointestinal toxicity or tumor regression after curative RT. Interestingly, the mean baseline and induced DNA damage was found to be lower in the group of RC patients with tumor stage IV (n = 7) as compared with the stage III (n = 35). The difference, however, did not reach statistical significance, apparently, because of the limited number of patients. Conclusions The study shows higher expression of γ-H2AX and 53BP1 foci in rectal cancer patients compared with healthy individuals. Yet the data in vitro were not predictive in regard to the radiotherapy outcome. Electronic supplementary material The online version of this article (doi:10.1186/s12885-015-1890-9) contains supplementary material, which is available to authorized users.
