Maybe if you don't tell the others?

I'd rather not have you or your brothers get hurt again because of me, so I'd rather not have you guys along, but I know I can't stop you.

[There was a chance that Leo would want to come along too if he knew, and Anemone would really like to minimize the potential turtle suffering if she can.]