When your data privacy is on the line, you should be taking every step available to protect yourself.
Even with the location setting turned off, and even with the app only running in the background, it's still tracking your every move.
You are free to use the service, but you must agree to their terms and conditions. It would be nice if you could keep all of the rights to your data, but that just isn’t possible.
Personal details can be used against you in very creative ways. We also hold the confidence of other people, and their details can become exposed if the listening switch gets turned on.