In 1955 Hubert T. Delany lost his position as Children's Court Justice. One of Polier's trusted colleagues, Delany was also one of the New York civil-rights movement's most outspoken leaders. A member of the NAACP Board of Directors, he was associated with the National Lawyers Guild, which the House Un-American Activities Committee (HUAC) suspected was a Communist front. Yet even at the height of the postwar Red scare, Delany refused to pull his punches. In the early fifties, he routinely condemned police brutality against African Americans and declared that New York's housing and public schools were racially segregated. Some saw the antiracist causes he championed as subversive. Still he persisted. He openly criticized politicians and Roman Catholic leaders who brandished anticommunist rhetoric in defense of racial segregation and racism. When the State Department alleged that the NAACP founder W. E. B. DuBois was a Communist and revoked his passport in 1951, Delany defended him. Delany's courage proved to be his undoing. When his term came up for renewal, Mayor Robert F. Wagner Jr.—a liberal Democrat—did not reappoint him, claiming that Delany's “left-wing views” disqualified him from the job.

One of Delany's liberal African American allies, Dr. Kenneth B. Clark, wrote the former judge in September, expressing his sympathy. A fellow member of the Urban League's committee on education, Clark was a City College of New York psychologist and cofounder of Harlem's Northside Center for Child Development. The Clarks founded this private child-guidance facility on Harlem's West 150th Street with the support of Delany and many of the same judges, psychiatrists, racial-justice activists, and philanthropists behind the Harlem Project and the Wiltwyck School for Boys. And in 1954 Clark had helped secure a major victory for the national civil-rights movement through his participation in the landmark Brown v. Board of Education Supreme Court case. Clark lamented the sacrifice of this “eminently qualified and humane judge” on the altar of anticommunism, informing Justice Delany that “I have admired your courage and insistence upon the fundamental rights of equality and democracy.” Despite Clark's admiration for Delany, this was the end of their association. Certainly by the 1950s activists identified as too left-leaning had become a liability to those seeking to both combat racism and shield themselves from accusations of Communist alliance.