Since you have the ability to Add to List, you also need a way to remove people from lists.
Find and select the List you want the people to be removed from.
That’s it!