If you like, you can create a backup of your conversation before removing it:
Go to the conversation
Open the conversation you want to delete (1).
Open the delete process
Open the group info (1), click the group settings (2) and select "Delete Group" (3) from the dropdown menu.
Now simply enter the group name (1) and confirm the deletion by hitting the delete button (2)