After giving his Beyond Vietnam speech, Martin Luther King Jr. lost some of his allies who had once been strong supporters of his cause. Not everyone who supported equality was opposed to the Vietnam War. However, Martin Luther King Jr. supported equality and opposed the Vietnam War. This is where people started to break away.
These people allied with King and strongly supported him when he was enacting these boycotts and sit-ins to allow both white and black people to have access to the same facilities. Then King changed and started to oppose the Vietnam War. His speeches were geared toward that. When this took place, some of his allies stopped supporting him because they supported aspects of the war.