Mexican drug lord Joaquin "El Chapo" Guzman is returning to a Brooklyn courtroom Friday, a day after a judge rejected his request to be allowed in the general inmate population.

The 59-year-old defendant famous for twice escaping from prison in Mexico lost his bid Thursday to relax the terms of his confinement at a lower Manhattan lockup when U.S. District Judge Brian Cogan concluded that solitary confinement was appropriate.

Cogan said the U.S. government had good justifications for applying tough jail conditions on a man who escaped twice, including once through a mile-long tunnel stretching from the shower in his cell. But Cogan relaxed the restrictions known as Special Administrative Measures enough for Guzman to communicate with his wife through written questions and answers.

His lawyers said in a statement that it was "devastating" for Guzman and his wife that they will not be allowed jail visits.

Guzman was brought to the U.S. in January to face charges that he oversaw a multi-billion dollar international drug trafficking operation responsible for murders and kidnappings. He has pleaded not guilty.